ESP8266 Duet web interface for any board

Hey, community.

I just had a very productive evening and "ported" the Duet web interface to ESP8266 chip.
ESP8266 is a neat little arduino-like stamp-sized module that has 80Mhz 32-bit CPU with WiFi onboard. It has serial port, WiFi and 2-4 MB flash disk. Well, here comes a web panel for it!

Means, now anyone can have an awesome web panel (credits to Christian Hammacher!), not only those who have Duet board!

To set it up you'll need to solder some resistors onto your ESP8266 and connect it to auxiliary port on your 3d printer board. For RADDS that's four topmost pins on long right AUX header.
And you'll need a Arduino set up for ESP8266 (yup, they even have that). Anyways, I put some instructions with pictures on the github repo at [github.com]

Here is a picture of my module that happily thinks he's a Duet.


Beware that some stuff simply does not work. You'll be kicked out of web panel if you'll try to upload something on poor 2MB flash (that is 58% full anyways). So don't do that.

I know, I know, it is pretty complicated of a setup for that little of a module that works for like ~80%. But hey, it's ~$3 shipped to your door! And WiFi! Aaaand... You want that!
So think of it as an electronic project. You have to solder, fight unstable software and flash firmwares. But that's what reprappers do, right? And we're good at it!

Please fork on github and make contributions. There's much to work on and I probably won't be able to spend a lot of time on it.
Licence: GPLv2.

Really cool! I've got to try this

Nice, i have now seen a few examples og web interfaces for 3d printer and must say that its fun to see the development

He-he. I think I just fixed the file uploads. I thought it is not possible since UART is real slow, but then I read somewhere that Arduino can do UART on up to 2 Mbit. So this combined with HTML5 File API allows you to stream gcode right through ESP8266 without saving it on intermediate flash. Downsides: you will be basically DDOSing your poor ESP with ~100 requests per second. Assuming you're the only client it'l do just fine.
As I mentioned before the upload speed is real slow. But it is something! Yay!

Test, anybody?..

Hi, sorry for being a bit slow, but I haven't quite understood what this can do. Can I use this to remote monitor a mega2560/ramps based printer running marlin? That would be cool.

This currently works with only RepRapFirmware boards. That is Duet, RADDS and maybe any other Arduino DUE-based boards.
But since "does it works with marlin" is #1 question to this project I should probably do something about it?..

If Marlin supported the M408 command, then I think it would probably work with Marlin, at least up to a point. That's the same command that would be needed to support a PanelDue. If using 5V printer electronics, you would need to use level shifters between the ESP8266 and the main board.

Edited 1 time(s). Last edit at 11/12/2015 04:53PM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Can you adjust it to send a file to the sd card on the controller instead of the 8266? And it doesn't work for RAMPs?

@dc42 There are several commands that are expected to come in JSON format. And as far as I know only RepRapFirmware does that. We could code some workarounds though (like storing macros in ESP flash, lol).
And BTW, ESP8266 has 5V-tolerant inputs, so you'll only need a voltage regulator (or a 3V line which is present on RADDS), no need to shift all the levels.

@madmike8 it does not store anything on ESP itself. It reads and sends files to and from the SD card that is plugged to the main board. And as we speak it does not support RAMPS since RepRapFirmware does not support it. But I am brainstorming what can be done. No promises though

Edited 1 time(s). Last edit at 11/12/2015 08:18PM by toxuin.

Quote
toxuin
@dc42 There are several commands that are expected to come in JSON format. And as far as I know only RepRapFirmware does that. We could code some workarounds though (like storing macros in ESP flash, lol).
And BTW, ESP8266 has 5V-tolerant inputs, so you'll only need a voltage regulator (or a 3V line which is present on RADDS), no need to shift all the levels.

The commands you need with json-format responses are M408 (which only ever returns json format because it is defined that way), M20 S2 (where the S2 specifies that a json response is expected, otherwise it returns the file list in the usual text format), and M36 (which again has only ever been defined as returning json format). The M20 S2 response would be trivial to add to Marlin without breaking backwards compatibility. For M408 and M36, sufficient support could easily be added to Marlin to keep the web interface happy. In most cases, if a json field is not present, the web interface will use a sensible default or just not display the associated information.

Arthur has done some work on supporting these responses in Smoothieware too.

The beauty of json format is that is self-describing, so you can add and remove fields without breaking the software that receives it - rather like XML but less verbose. That's why we used it in these responses. I've seen Pronterface fail more than once because it was expecting a response in a particular text format, and it couldn't cope when the format returned by the firmware was changed slightly.

Edited 1 time(s). Last edit at 11/13/2015 04:39AM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

I've got a few of these. I'm wanting to make a basic web server that you select a gcode file and it transfers it to the Ramps Controllers SD card and then starts the print from the printer's SD card after successful upload. If I'm able to get that to work, and then I might try to have it look for the M140 gcode and start preheating the bed before transfering the whole file.

All else fails, I'll use a Toshiba Flashair SD and transfer the file that way, and use the ESP8266 to just remotely start the print.

Edited 1 time(s). Last edit at 11/13/2015 03:31PM by madmike8.

I do not have any Smoothieware-enabled devices, sadly. Would be great if someone could test it and tell what doesn't work. It'd be great to have more than one firmware supported :-D

@madmike8I hear you. But unless we add the JSON replies to Marlin and\or Repetier it won't work with RAMPS 1.4. I was looking at Repetier source this friday but got too lazy and went to see new South Park episode. I'll try again soon.

@all: dc42 is right, there are only 3 G-Code commands that could be added to basically any firmware to make it work with ESP8266. If you got skills – add it to your favourite firmware and make a pull request. Everyone wins!

These things look pretty neat. I've ordered one to play with. Am I out of my mind, or would it be possible to port the entire printer control software to one of these things? There seems to be just about enough pins for a single extruder setup, although I haven't got far enough to know if it's flexible enough (can you have 2 ADC inputs for example?). There definitely isn't enough to have an lcd, but maybe with a web interface that wouldn't be such a big problem. It just seems so darn cute (and cheap!).

I like your thinking. Unfortunately, there are only ~15 GPIO pins on ESP8266 12E and just ONE ADC pin :-D Even with external devices to give more ADC you'll be losing pins (and AFAIK ESP8266 doesn't have hardware I2C). You could probably control stuff like Shapeoko with it though. No endstops, no heaters, only 3 axis. Yay!

So I've made changes to the latest Repetier-Firmware (dev version) and it should support required commands now. I've submitted a pull request for Repetier, but seeing how many pull requests are there for Repetier-Firmware I don't think it going to be approved soon. Anyways, if anyone wants to test it out on hardware, here's a repo link: [github.com] (make sure you're checking out the work092 branch!).

Here's a peek at ESP8266 web interface working with Repetier 0.92.6:


Since Repetier-Firmware does not support on-the-fly serial baudrate change, file transfer speed is VERY limited. I get ~3 Kb/s when testing. And since Repetier saves files in some magic binary compressed unreadable-by-humans weird format, there is no info like filament needed and layer height for files uploaded through serial.

Oh, and I adjusted the ESPrinter firmware so it can haz icons (just like original Duet web interface, yaaay.). Glyphicons font is not stored on ESP8266 itself, it is loaded from CDN. If you can't see icons – you have no network connection!

Now I can claim it works with RAMPS, but answer to "does it work with Marlin" is still no.

Edited 4 time(s). Last edit at 11/18/2015 02:16AM by toxuin.

Good news, everyone!

Repetier was kind enough and accepted my pull request (sorta, anyways). He also mentioned that support for JSON-style responses is coming to the next version of host software be it Repetier-Host of Repetier-Server.

Hi Toxuin,

I'm new to using arduino on the ESP, so I need some more info. I'm a bit more used to running LUA.

How do I do this?
"You have to be able to upload SPIFFS data to it. Do not ignore that part or you'll get constant 404's."

I have googled for SPIFFS and ESP but results are conflicting about it having been added to the IDE, i think, then removed then being pending etc. Any advice?

Thank you,
Errol

@errolt When you add ESP8266 board support to your Arduino IDE you use a link in the board manager. There are two links on Arduino for ESP8266 project github: one is stable, other one is staging. You have to use the staging version - only staging has SPIFFS support.
Then you need to be able to upload files in SPIFFS format. For that you'll need to install a SPIFFS uploader tool in your Arduino IDE. They have tutorial about that on this page: [github.com] at the "Uploading files to file system" section. Just follow it and you'll be able to upload SPIFFS.
Let me know how it goes.

Hey toxuin,

I think I made it to the end with this but I 've hit an obstacle I can't seem to overcome...
The page loads without css/js and looking at the console it says:

GET http: // a.b.c.d/ ugly.min.css.gz net::ERR_CONTENT_DECODING_FAILED
GET http: //a.b.c.d/ ugly.min.js.gz net::ERR_CONTENT_DECODING_FAILED

I 'll admit web development is not my thing
Any idea on how to solve this?

Edited 1 time(s). Last edit at 11/28/2015 06:25PM by mandrav.

Thanks for your observations! Apparently something has changed in how ESP8266 webserver handles .gz files – now it detects gzipped files and sends "Content-Encoding: gzip" on its own. ESPRINTER code was doing the same job so there were two content-encoding headers in every response and Chrome seems to not like it (Safari is okay with that though).
Anyways, I've pushed some small fixes to the repo and tested the thing with Chromium 45.0.2454.101 and Firefox 26. Seems to work.

Hi toxuin,

Thank you for the assistance with the SPIFFS upload.

I had the same problem as mandrav, but as I have a 4M part I just unzipped the CSS an is files and was able to load the page. Will get the fix and use compressed files again.

I don't know if I broke something somewhere though because, if I press Connect while the module is still connected to the PC, the module crashes with a full stack dump.

I have traced the problem line to the line that prints the IP to the printer LCD. I don't know if the WiFi.localip() has also changed and does not like to be concatenated to a string. I have not had time to investigate further.

Thank you,
Errol

Oh, forgot to mention. I don't know if the WiFi access point name that is returned during setup should be passed through the urldecode function(can't remember the function name and I'm not at my pc) as any space in the AP name gets replaced with a '+' char and saved to the eeprom as such.

Thank you,
Errol

I pushed quick fixes for these issues to github. I haven't tested it on real hardware yet, but there is nothing substantial.
Apparently I was using some weird WiFi library version (either ahead of what's in staging or awfully outdated) that had the .toSting() method and concatenation overloads. For me having no 3d printer connected to the module and clicking "Connect" was just producing an obvious error in UI, but not a module crash. I'll force-update my environment.

Quote
toxuin
Thanks for your observations! Apparently something has changed in how ESP8266 webserver handles .gz files – now it detects gzipped files and sends "Content-Encoding: gzip" on its own. ESPRINTER code was doing the same job so there were two content-encoding headers in every response and Chrome seems to not like it (Safari is okay with that though).
Anyways, I've pushed some small fixes to the repo and tested the thing with Chromium 45.0.2454.101 and Firefox 26. Seems to work.

It works now, thanks toxuin!
I will now connect this to my printer and see about the functionality

Success

A couple of things I noticed:

1. I get this error: "Uncaught TypeError: Cannot read property 'axesHomed' of undefined". Doesn't look like it affects anything other than displaying that the axes are always not homed.
2. The button for uploading system files (e.g. config.g) is not available in settings. I don't know if this is intentional.

Promising work though!

Are you using RepRapFirmware? Homing axis should work, but I'll check it on my printer. I remember this error showed up on Repetier.

At first, ESPrinter could not upload files at all so I've deleted (or hide) everything clickable that doesn't work. I've removed the whole UI block that is responsible for uploading the /sys and /www files as there was no point in it at that time: uploading something to /www dir wouldn't change the actual UI on the ESPrinter. I didn't have any need to upload the config.g file (and I am a bit scared, to be honest: file uploads are kinda unstable and config.g is a big deal) I'll put that block back then.

In the meanwhile, I am experimenting with remote firmware update. Not for the printer, just for ESP8266 module. My goal is to make it so that you don't have to go through all the hassle with git, SPIFFS tool and board managers every time I push an update. It would detect your flash chip size and just download new firmware that would fit your module. And in case your ESP is connected to no-internet access point, you can just upload a newer firmware file through browser.

Yes, I 'm using RepRapFirmware with RADDS.
Having the ability to upload /sys files is of great help when tuning the printer because you can iterate different settings easily w/out having to remove the card, load it in a pc, update it, unload it and then put it back. Worst case scenario, the sd gets corrupted from web uploading for some reason and I have to take it out and copy all files again. That's not so scary . Btw, I always keep my sdcard files in a folder on my pc and first update those and then copy them to sd. This way I always have the current copy available on my hard disk.

As for remote firmware updating of esp8266, that sounds exciting. Waiting to see what you come up with!

You can iterate different settings without uploading a new config.g every time. Just 'print' the new config.g. The network settings won't get updated, and you will get an error from the M563 command(s) unless you delete the existing tool(s) first using e.g. M365 P0 D-1 H-1, but everything else should work.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42
You can iterate different settings without uploading a new config.g every time. Just 'print' the new config.g. The network settings won't get updated, and you will get an error from the M563 command(s) unless you delete the existing tool(s) first using e.g. M365 P0 D-1 H-1, but everything else should work.

I 'm aware that I can "print" the settings. As a matter of fact, when testing stuff I usually change one setting at a time (e.g. delta params) so I run the command directly in the gcode console. I 'm mostly interested though in updating the actual file on the sd (config.g), when I 'm confident about the changes, so my settings survive reboots

Edited 1 time(s). Last edit at 11/30/2015 09:58AM by mandrav.

The /sys upload block is back, you can use it now.

Also, errolt added SSDP. That's a windows-compatible counterpart of mDNS: your printer will now show up in network neighbourhood and if you double-click it you'll be taken to the web interface.