New experimental firmware 0.59c-dc42 and web interface 0.79

I have put new experimental firmware binary 059c-dc42 at [github.com] and web interface 0.79 at [github.com]. There are no intentional changes to printing or the handling of gcodes since 0.59b. The changes are all in the web interface, in particular file uploading.

Firmware changes:

* Rewrote much of the web server as a state machine to improve performance.

* Added support for persistent connections.

* Added success/fail return status for all SD card file I/O operations.

* Added new web server commands and associated responses to handle file uploading and deleting. The new commands allow the Duet to report the file upload status to the web server. If an upload fails, this is reported to the web interface, and the file is deleted instead of leaving an incomplete file on the SD card. When the upload is complete, the file size is checked against the value provided by the sender.

* Improved robustness in the presence of network errors by fixing a number of bugs in the interface to lwip (the TCP/IP stack).

Web interface changes:

* Use the new upload commands supported by 0.59b firmware to improve upload speed and track upload status.

* Report failed uploads to the user. Previously, failed uploads were not detectable and usually led to the uploaded data being printed instead of saved on the SD card.

* If the web interface is reconnected to the Duet, any pending file upload is cancelled.

* Use HTML5 progress bars. The progress bars previously used by the web interface failed to update in a timely fashion at high upload speeds.

* Filenames that are too long to fit in the width available in the SD card listing are now broken and wrapped, so that they and the associated trash cans are displayed properly. This has been tested in Chrome, FireFox, and even IE!

* .js, .css, .htm, .png and font files for the web interface can now be uploaded. So you should never need to take the SD card out of the Duet, as long as you maintain compatibility between the web interface files on the SD card and the firmware.

* Upload speed for all file types is now almost 5Mb/min on my test system, using Windows 7 and Chrome.

If you decide to try these, please note that: web interface 0.79 requires firmware 0.59c. Firmware 0.58c requires either web interface 0.79, or a much older version e.g. 0.75 or earlier (but file uploading will be slower). Please report positive/negative experiences with these updates on this thread.

Edited 1 time(s). Last edit at 05/23/2014 06:21PM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

dc42, your modifications look very nice, in fact the networking code is already looking a lot more robust than before. I also like the new progress bars, they look much better than the old ones IMO. But I've noticed you're trying to store a word in an unsigned byte in line 55 in Network.cpp - I guess it's not tragic because the preprocessor takes care of that, but I'd change the type of windowSize to int just to be safe.

TBH I was a little worried about not having persistent connections a few hours ago when I implemented some changes for FTP and Telnet, but your update has saved me some time (again) I hope I can contribute something new next week.

Your current firmware is working well so far on Windows 8.1 - thanks again for your work.

Edited 1 time(s). Last edit at 05/23/2014 08:56PM by zombiepantslol.

dc42, great set of updates, you are working through the update list at a fast pace and we are all benefiting.

Thank you
Paul

---

appjaws - Core XYUV Duet Ethernet Duex5
firmware 3.1.1 Web Interface 3.1.1
Ormerod 1-converted to laser engraver, Duet wifi
OpenSCAD version 2020.07
slic3r-1.3.0, Simplify3D 4.1.2, Cura-4.4.1

Quote
zombiepantslol
But I've noticed you're trying to store a word in an unsigned byte in line 55 in Network.cpp

Good catch! It probably means that the window size is effectively 152 bytes instead of the intended 1432, causing files to be sent back to the host more slowly. But the page load time still looks find on my system. I'll fix it in the next release. I'm surprised the compiler doesn't give a warning.

Quote

Your current firmware is working well so far on Windows 8.1 - thanks again for your work.

Thanks for confirming this.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42

* Upload speed for all file types is now almost 5Mb/min on my test system, using Windows 7 and Chrome.

First I should say I do not have a Ormerod yet. (Plan to get one though, why I registred to this forum

5Mb[it]/min
is this a misstype?
thats little faster than old dialup modems from mid 90s.
(5000 / 60 = 83Kbit/s)

If not what is the bottleneck?

It's bytes not bits so 8 times faster. Still a bit slow for a network connection but it has been improved a lot from the first versions.

ok so 5MB/min 83KB/s
Is it the ARM / NIC chip or SD card that is the bottleneck?
or just software?

Just to clarify I'm not bashin anything here.
Im just curious

Edited 2 time(s). Last edit at 05/24/2014 11:52AM by Lejoni.

It's because the web interface is using Javascript to encode and pack the file into 950 byte blocks and sending them using GET. Now that I have sorted out the error handshaking, I should be able to switch to using PUT and do less of the work in Javascript. But it's not high on my priority list any more because I find 5MB/min tolerable. I'll be working on the movement algorithm instead, to finish implementing combined XYZ moves and sort out acceleration issues.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42
[...]
I'll be working on the movement algorithm instead, to finish implementing combined XYZ moves and sort out acceleration issues.

This sounds good to me! Installed your latest this afternoon and seems to be running very nicely on a reasonably big print. (Note that Ive never used the web interface though so can't comment about that)

regards
Andy

---

Ormerod #318
www.zoomworks.org - Free and Open Source Stuff

I've installed the latest firmware and web interface, Dave, and it's getting slicker all the time. Well done!

However, I have had a problem uploading/updating files to/in the sys directory of the SD card. Out of umpteen attempts, it's only worked once. After updating just one file, taking the SD card out and putting it in my computer to see what's happened, more often than not the original file is unchanged, and the new file has been written with a filename "undefined" (without any extension).

At one stage, I deleted all of the files in the sys directory except config.g, and then tried re-loading them one by one using the web interface. They all got uploaded, but with filenames jumbled up so that, e.g., homeall.g contained the gcodes for homex.g, and with one of the files named "undefined."

I was using XP and Chrome.

Ken.

Quote
KenF
However, I have had a problem uploading/updating files to/in the sys directory of the SD card. Out of umpteen attempts, it's only worked once. After updating just one file, taking the SD card out and putting it in my computer to see what's happened, more often than not the original file is unchanged, and the new file has been written with a filename "undefined" (without any extension).

Hi Ken,

I've just been looking into this after another user PM'd me. What I'm finding is that the first time I upload config.g in a new instance of the web interface, it reports success but the file hasn't been updated. Subsequent attempts work OK. Is this what you find too?

[EDIT: I also found a file called "undefined" in the /sys directory. So I guess that it passed this as the filename the first time I uploaded.]

Edited 2 time(s). Last edit at 05/25/2014 04:12AM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

I found the problem. I haven't time to do a release just now, but the following should fix it. In reprap.js change line 478 from:

uploadFile(action, fName, sysDir + gFilename, "");

to:

uploadFile(action, fName, sysDir + fName, "");

You can upload the new reprap.js via the web interface. The problem affects any file you up load using the "Upload config file" button.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42
... What I'm finding is that the first time I upload config.g in a new instance of the web interface, it reports success but the file hasn't been updated. Subsequent attempts work OK. Is this what you find too? ...

I've just tried this with a different SD card (in case there was a problem with the first one):

• Delete all files in SD sys directory, and install SD in Duet.
• With new and only Chrome window, upload config.g.
• Reboot Duet. Nothing reported for under settings tab for config.g.
• Remove SD card and check. Only file in sys directory is called "Undefined", but it contains the content of the config.g file I tried uploading.
• Reinstall SD in Duet. Same browser window. Nothing reported for config.
• Upload config.g again. Reboot Duet. Same browser. Nothing reported for config.
• Close browser. Reboot Duet. Open new browser. Nothing reported for config.
• Upload config.g. Reboot Duet. Same browser. Nothing reported for config.
• Close browser. Open new browser. Nothing reported for config.
• Remove SD card and check. As before, only file in sys directory is the one called "Undefined".

Hope that helps.

Ken.

Quote
dc42
I found the problem...

Ah, we crossed. I'll give it a try. Thanks Dave.

I've tried that. Hope I got it right, but all I have to edit the reprap.js file is Notepad. Think we're getting there, but not quite all the way.

I uploaded the modified reprap.js using the upload www facility. Then I uploaded a modified homex.g file using the upload sys facility and ran it. Still performed the unmodified action. Uploaded modified homex.g again, and that time it did what it should.

Then has a look at the SD card. The reprap.js and homex.g files were as they should be. But it also had a file called reprap.js in the sys folder and containing the content of the modified homex.g file! So something's got a bit screwed up.

Ken.

Ken, after uploading reprap.js you need to close the browser tab and open a new one, to pick up the new reprap.js in the browser.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

I've just released web interface 0.80 to [github.com]. As with 0.79, this uses the new file upload protocol and so needs firmware 0.59c-dc42. Changes in this release:

* Fixed bug whereby the "Upload config file" button would store the uploaded file using the name of the previously uploaded file

* The temperature control buttons now remain enabled during an SD card print

* Small improvement in file upload speed for large files

If you are already running web interface 0.79 and firmware 0.59c-dc42, then all you need to do is upload the new reprap.htm and reprap.js files using the "Upload web file" button. Then close the browser tab, open a new one and connect to the Duet, so that the new files get picked up by the browser.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Can't wait to get my Ormerod!

This morning I found my Ormerod (with 0.59c / 0.79) had crashed mid-print. Does this output (M111 S2) actually mean anything? Was printing a 4MB gcode file (not the largest one I ever printed), seems to have gone about 3/4 of the way.

07:42:22 Program static ram used: 39852 Dynamic ram used: 39004 Recycled dynamic ram: 3064 Current stack ram used: 1288 Maximum stack ram used: 2236
Never used ram: 14148 Last reset 06:21:28 ago, cause: software Last software reset code & available RAM: 0x1002, 13768 Error status: 0
Bed probe heights: 0.000 0.000 0.000 0.000 Free file entries: 10

Also a bunch of these in the log, not sure what those signify:

07:44:38 Warning! Ormerod webserver is probably broken, power cycle/reset your Duet Board 

Edit: Line breaks for readability


Edit again: Hah. It actually worked! Using gcode.ws I could identify exactly where in the print it had died. I then took the original file and cut out everything before it (except the necessary startup stuff), and X/Y/Z homing is apparently exact enough that it could just pick up where it left and keep printing.

Edited 3 time(s). Last edit at 05/26/2014 02:11AM by jstck.

Hi jstck, the 0x1002 software reset code indicates that the network locked up. The web interface log confirms that. What network equipment is there between the Duet and the PC?

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

The Ormerod is connected to a little wifi bridge, so it at least always has link up (and that part has so far been very reliable). However, the time when it crashed probably matches when my laptop (which had an open browser window to the web ui) went to sleep. So I guess one always needs to disconnect the web ui before leaving a print running? Is it possible to do something about that? After all when printing from SD the web ui is little more than just a status display, and the Duet should ideally keep going no matter what happens to the network connection.

Quote
jstck
The Ormerod is connected to a little wifi bridge, so it at least always has link up (and that part has so far been very reliable). However, the time when it crashed probably matches when my laptop (which had an open browser window to the web ui) went to sleep. So I guess one always needs to disconnect the web ui before leaving a print running? Is it possible to do something about that? After all when printing from SD the web ui is little more than just a status display, and the Duet should ideally keep going no matter what happens to the network connection.

I agree, it should not matter if the laptop goes top sleep during a print. The network code is generally very reliable now and survives the network being disconnected. But while testing the most recent version, I managed to get a 0x1002 reset just once, and now you have seen that too. So something is still not quite right in the network code. It's a complicated area because there is both a standard library (lwip) and Duet-specific code. The lwip documentation unfortunately doesn't give enough detail about how lwip interacts with its clients, and there is a lot of code to wade through to find out, which I don't have time to do.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

The firmware bin file has been growing in size gradually as more features are added getting up to 185kbyte in 59a. Then for 59b and 59c it suddenly increased by nearly 64kbyte to 254kbyte.

I have been doing my own builds from this source using a later arduino library but I didn't see this dramatic change in size going from a to b.

I don't understand what caused this big change or whether I am missing some extra piece.

Hi Bob,

I changed the optimization settings in gcc++. Previously it was set to -Os i.e. optimize for space, and I changed it to -O3 to improve the upload speed. The firmware still occupies only about half of the available space. I may switch to -O2 in the future, as the extra improvement given by -O3 is probably not worth the space.

I didn't check the project file in, because unfortunately a lot of settings in it depend on the build path and the path to your Arduino installation.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42
Hi Bob,

I changed the optimization settings in gcc++. Previously it was set to -Os i.e. optimize for space, and I changed it to -O3 to improve the upload speed. The firmware still occupies only about half of the available space. I may switch to -O2 in the future, as the extra improvement given by -O3 is probably not worth the space.

I didn't check the project file in, because unfortunately a lot of settings in it depend on the build path and the path to your Arduino installation.

Thanks. That explains it. I changed the optimise to 3 and got a similar size. There is some serious in-lining of the functions there.

I also tried -O2. Results I got on a 4Mbyte file were

-Os size 188k speed = 4.02Mbyte/min
-O2 size 222k speed = 4.53Mbyte/min
-O3 size 255k speed = 4.63Mbyte/min

So O2 might be quite a good compromise between speed and firmware size.

I was checking why I couldn't get height, filament, progress on latest version.

I think a check on filetype was introduced in GetFileInfo. It is checking for .gcode or .gc or .gco. Unfortunately my giles had a .g extension so got missed out.

I tried renaming a file to .gc and that restored the functionality and then I added .g as an allowable filetype and that worked OK.

Hi,
Until now have just used pronterface but I downloaded 0.59c firmware and installed plus web interface 0.79 and copied files onto card. My router shows that 192.168.1.14 is on my local network. But Chrome keeps reporting that it cannot connect to it. I must have missed something so any help would be appreciated. I sort of remember some time ago that you needed to have files on a local server not just on the Duet. Is that still the case ?

The original firmware and web interface did need the web support files (js,css etc) to be externally hosted and this was either from the original iamburny github site or from your own local server. However, recent versions of the firmware now allow the files to be hosted direct from the Duet so that putting the whole set of web files including the sub-folders should be OK.

I did occasionally and fairly rarely have a problem with this. If the browser requests all the files simultaneously then I think the Duet could still have a problem. Normally the files get cached so that requesting a few times sorts that out. It is also possible to still serve the support files locally by editing the 10 links in the reprap.htm to point to a local server and putting the files there. I was still doing that until recently to avoid having to take the SD card out when updating all the web files to newer versions. The latest firmware/web does now support remotely updating all the files on the Duet so this should not be needed any more even to do maintenance.

First step to trouble shoot is to make sure you can ping the Duet. Then you can turn on Tools/Developer mode in Chrome to see the web files being loaded and check that is OK.

Persist with this. Once you have got it going you won't want to go back to Pronterface.

Edited 1 time(s). Last edit at 05/26/2014 03:58PM by bobtidey.

Quote
bobtidey

Quote
dc42
Thanks. That explains it. I changed the optimise to 3 and got a similar size. There is some serious in-lining of the functions there.

I also tried -O2. Results I got on a 4Mbyte file were

-Os size 188k speed = 4.02Mbyte/min
-O2 size 222k speed = 4.53Mbyte/min
-O3 size 255k speed = 4.63Mbyte/min

So O2 might be quite a good compromise between speed and firmware size.

I forgot to mention that I changed to -O3 for both the C++ and the C compiler. I guess you did that too.

Thanks for running those timing tests. Since -O3 is slightly faster than -O2 and at present there is plenty of free space, I'll stick with -O3 for now.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
bobtidey
I was checking why I couldn't get height, filament, progress on latest version.

I think a check on filetype was introduced in GetFileInfo. It is checking for .gcode or .gc or .gco. Unfortunately my giles had a .g extension so got missed out.

I tried renaming a file to .gc and that restored the functionality and then I added .g as an allowable filetype and that worked OK.

Yes, I spotted that one yesterday evening. The .gc should have been .g. I'll correct it in the next release

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].