New experimental firmware (0.65e-dc42) with FTP support

Hello everyone,
I've just finished working on my firmware which is based off dc42's excellent development branch (now firmware version 0.65e, many thanks to him). I've added an FTP server for easier SD card file management, prepared a Telnet port and rewrote parts of the networking code.

If you're interesed in trying it out, you can download a precompiled firmware binary here. All other changes can be found in my GitHub repository. Please read dc42's release notes first, otherwise some things will NOT work when upgrading from an older firmware version.

The FTP server has been designed to work with FileZilla, but I guess any other FTP program with PASV support should work. FTP active mode is currently not supported.
When setting up a FTP connection profile, you can specify any username you like, but the password should match the one in your config.g file (usually "reprap").

I haven't tested everything yet, so be prepared to revert to dc42's (or RRP's) firmware in case anything goes wrong.

Feedback is welcome!

Edited 3 time(s). Last edit at 06/05/2014 04:44AM by zombiepantslol.

Very interesting. Especially the telnet part, is the telnet prompt supposed to behave as the USB-serial interface would?

Yes, I will add something similar in a future firmware release I think Telnet might be useful for debugging purposes as well.

I've just merged in dc42's latest changes for his 0.65e firmware relelase, so if anyone is interested to give it a try, download it here:
[github.com]

Awesome

I'll give it a go and attempt to provide feedback asap

Matt

---

Limited Edition Red RS Ormerod 1 #144 of 200 - RRP 1.09fw
iamburnys Ormerod Upgrades Github
Follow me on ThingiVerse My Designs

Zpl, what file upload speed to the Duet does FTP provide?

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Nice work zpl, here is some feedback on the FTP service.

FTP Client : FlashFXP 4.4.4 (build 2043)
Connection Mode : PASV mode
List Command : LIST (without -al)

Directory Navigation: All fine
Delete: Worked fine
Upload: SinePot1_0.2.gcode 19.37 MB in 4 minutes 31 seconds (73.3 KB/s)
Download: SinePot1_0.2.gcode 19.37 MB in 1 minute 14 seconds (267.4 KB/s)

I may have a look at coding a simple FTP client into the Web interface (time depending ofc), Looking forward to telnet too!

Matt

---

Limited Edition Red RS Ormerod 1 #144 of 200 - RRP 1.09fw
iamburnys Ormerod Upgrades Github
Follow me on ThingiVerse My Designs

Hi Matt, if you're back in business, I'll gladly pass development of the web interface back to you! Latest version compatible with my firmware is at [github.com]. Would you like me to send you a pull request?

Hi Zpl, I think it should be possible to improve the FTP upload speed, as I am already achieving 107kbytes/sec using HTTP GET requests, which involves URI encoding and decoding. Are you writing to the SD card file using the file block write interface that I added in 0.59e?

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Hi dc,

Sadly I'm still time restricted but cant resist the odd fiddle here and there, I don't think I can commit enough time to keep up with you speedy FW developments, however I will happily pull in your changes to my repo it seems sensible to keep the masters in sync and then I can so I can make the occasional tweak too.

Matt

---

Limited Edition Red RS Ormerod 1 #144 of 200 - RRP 1.09fw
iamburnys Ormerod Upgrades Github
Follow me on ThingiVerse My Designs

Hi Matt,

Sorry to hear you are still time-restricted. I have created the pull request.

I know you wanted to retain the direct web print function, however it got to the state where it needed separate code from the other functions, too much testing in relation to its value, and too much development to keep it up to date e.g. to establish the total filament required for the print. I hope you don't mind me removing it.

I had the following changes in mind for the web interface in the not-too-distant future:

1. Change the print completion time estimate that is based on filament consumption to use the consumption rate of the last 5 layers (or as much as has been printed if less) rather than the average consumption rate since starting. I think this should provide a more accurate estimate in most cases.

2. Provide only one estimate based on layer height instead of three, so that there are just two estimates of print completion time. Use the average print time for the last 5 layers, or as many layers have been completed if less than 5.

3. Add support for multiple extruders, because I may play with this in the near future. I have already done the preparatory work, i.e. changed the poll protocol to support variable numbers of extruders and heaters.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Hello Guys,

I see you developed a FTP communication protocol for Marlin. So I think you are expert enough to answer a simple question.
I hope you have time and will to help me ^_^
Right now, I'm using repetier firmware, and I'm no sure if I want use it or not in the next future. do you have any advice?
Anyhow I have a task to accomplish, and I'm not sure about what I have to do.

I'd like to insert some custom gcode in my firmware to implement some features like add a door to it.
I can do it in two way, add a case in the list of commands, or create a new char.
So, let me say I want to open a door, I could add a case, for instance g50 that is free, then add my custom code inside this case.
I'm able to do it, it is pretty easy ^_^

The original firmware version is the 0.95F on [www.repetier.com]
Instead I'm not able to add a custom letter, for example "K" and start a new list of command, so use K1 to open my door.
Basically I modified the file:
1) gcode.h
2) gcode.cpp
3) command.cpp

1) I added a method called hask() copied from hasM method.
2) I attached my gcode.cpp all the change are pointed by "!!!!!!!!!!!!!!"
3) The same as point 2.

I thank you in advance, for your help.

Thanks

Thanks for your feedback so far! iamburny's readings match my observations, but I haven't been able to increase the upload speeds yet.

dc42: I do use your new buffer write method to write incoming upload data. However, I believe the reason why upload speed is still kind of low is the flow of incoming RequestStates in readyTransactions. With the current code to flush incoming data, it can take up to 16 Webserverpin calls before an incoming transmission is fully written to the SD card. As a consequence, I tried to increase the buffer write length in the flush code to 512 bytes to speed this up, but this caused only HTTP uploads to malfunction which I don't really understand yet - FTP uploads still worked after I had made this change. I know FileStore::Write will use a buffer size of 256 bytes, but this does not explain why increasing the buffer size causes problems with HTTP but not with FTP, especially because FileStore::Write doesn't seem to use its own upload buffer at all when data is written block-wise. In theory I could modify the Webserver's flush code depending on the protocol, but I'd rather like to find out what's causing this issue first.

Another reason why HTTP uploads are still faster than FTP uploads is the fact that the JS code automatically throttles upload speed because it waits for an HTTP response before it sends any other data. For FTP, I assume this could only be achieved physically using LWIP calls. I know too little about LWIP though, so I haven't investigated any further yet. I believe it would be best to patch the low-level networking code to send ACKs for the connection PCB only whenever tcp_recved() is called. This way I could move the tcp_recved() call to RequestState::FreePbuf which would likely reduce the number of pending transactions. I figure this could speed up FTP uploads as well, and I could get rid of the rather dirty workaround around ethernet_task() in Network:pin().

If you have any thoughts on this, please let me know. I'm really not sure how to address these issues at the current time.

steob86; I'm sorry, but I don't know the Repetier firmware at all and from what I can see, it looks very different to the RepRap firmware we're running on the Duet. I can't help you on that, but you said you could do your changes by adding a custom G-Code. So why don't you try this?

Edited 1 time(s). Last edit at 06/06/2014 11:15AM by zombiepantslol.

Hi zombiepantslol,

well I already did it, using a custom code. This means I'm able to use the command G50 to open my door.
My point is to use the command K1.
The difference is subtle I know, but the letter K is not part of the standard characters.
Maybe this method is more complicated that what I need, but still I'd like to achieve this target.

Moreover I love your idea of a FTP support, and I'm really interested in it. Unfortunately I can't help so far
becasue my knowledge of the topic is almost zero.
I will keep reading this topic, before or later, maybe I can help ^_^

Thanks for your replay ^_^

Hope someone else that read this post could/want to help me.

Thanks again!!!

Hi zpl,

I've got to dash off, but here is a quick response:

1. If the FileStore buffer size matters, then you are not using the block write call that I added, you are using the single-character write..

2. Yes the tcp_recved() call should be moved to where the pbuf is freed. I intended to do this but didn't get around to it.

3. In Webserver:pin() i deliberately only write a limited number of bytes and not the whole received fragment, precisely so that lwip gets called and can send acks and the previous response sooner.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Hi dc42,
thanks for your reply.

Quote
dc42
1. If the FileStore buffer size matters, then you are not using the block write call that I added, you are using the single-character write..

Trust me, I am writing all upload buffers block-wise. That's why I don't understand this problem either. If you don't believe me, check the source code.

Quote
dc42
2. Yes the tcp_recved() call should be moved to where the pbuf is freed. I intended to do this but didn't get around to it.

I will move this call and see if I find anything else in Lwip.

Quote
dc42
3. In Webserver:pin() i deliberately only write a limited number of bytes and not the whole received fragment, precisely so that lwip gets called and can send acks and the previous response sooner.

Yes, I didn't say limiting the buffer write length is bad, but at the moment it is still too low IMO. I just don't understand why increasing it causes a firmware crash..

Zpl, what exactly is the parameter you are referring to, that causes a firmware crash if you increase it?

I may get a chance to look at your code on more detail today.

Edited 1 time(s). Last edit at 06/07/2014 02:34AM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].