LPC port of RepRapFirmware

@dc42 @sdavi many thanks for the feedback, very interesting.

Before I move on to try and get the rest of my configuration sorted I thought I'd run a few more tests.

I ran M122 P104 and got a result of 0.13MB/s and I re-ran the upload tests (using a 10MB file) and the speed I get is 0.094 MB/s (I ran these tests a few times from different browsers/computers with the same result). My understanding is that on my LPC1768 board the sd card SPI is running at 25MHz (which I think is the default) and the esp8266 spi interface is running at 6MHz. It is curious that I get a slower speed than @sdavi, I wonder if my SD card is slowing things down?

If either of you have the time I'm curious about the memory issues with the LPC devices. Do you have any figures for actual overall memory usage of the various features? Does using the esp8266 module save much compared to using wired networking (on boards that provide access to it)? Does the reprap firmware make much use of dynamic allocations? Is there an easy way to track memory usage?

Oh and thanks for the heads up on the potential PWM issues I'll certainly keep an eye out for that. Though it will probably be a while before I have things at the state that I'm hooking this board up to my actual printer.

I'm a little surprised that more people are not using this firmware on the LPC boards (which are now very popular).

I think the SPI interface to the SD card is the bottleneck in your machine, because the file write speed is low and the upload speed only a little lower. You can try different brands of SD card, but I doubt it will make much difference. Does the LPC port use DMA to do SPI transfers?

RRF allocates a lot of dynamic memory at boot time, some more when certain configuration commands are run (e.g. M563 allocates a Tool object), and none at all during normal printing. The Never Used RAM in the M122 report shows you how much headroom you have.

The ESP interface uses buffers to send and receive SPI data (2 * 2kb AFAIR) but against that it doesn't need the packet buffers that wired networking uses. Using a Wiz820io module to provide wired networking via SPI doesn't need the SPI buffers, so it should be more economical on RAM.

Edited 3 time(s). Last edit at 01/08/2020 04:45PM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Since I have a few sd cards kicking around I tried a different one, I picked the one that looked likely to be fastest (SanDisk Ultra 16Gb - it was red!). With this I get 0.44 MB/s for the M122 P104 test and 0.22 MB/s for the actual upload test (which is I think more in line with what @sdavi was getting). So yes the SD card is/was the problem and it looks like a different SD card can help!

I checked the never used ram result and it was 3040, so not a lot to play with. This was after performing the above upload test.

On the LPC, with WIFI compiled in there is 1 file write buffer of 1k and in the standard LPC build is only 1 file write buffer of 0.5k. You can test with this in storage/FileWriteBuffer.h (if you have enough memory). As comparison to the Duets which has 2 write buffers of 8k.

The LPC manual says that when running SSP in slave mode, the master must not exceed 1/12 of the SSP peripheral clock. When SSP PCLK configured to run at CCLK, this gives us a max speed of 8MHz on the LPC1768 to talk to the ESP. I was getting some errors when running at 8MHz and even 7MHz but 6MHz seems to be working fine for me.

In RRF3, SSP now uses DMA which uses the GPDMA on the LPC.

To get the built-in ethernet to fit MTU was reduced to 586 bytes (the minimum +TCP allows to use DHCP) and only 5x 616 byte buffers allocated for the EMAC. MSS was reduced to 512, each TCP socket only uses 2x MSS buffers and there are 2 sockets configured. Plus memory used by +TCP and memory for the EMAC and +TCP RTOS stacks. The network buffers in RRF was further reduced to 2x 512 bytes.

The wifi build uses more memory than the built-in ethernet version as I didn't need the extra memory for LCD, so it uses with the larger FileWriteBuffer 1k and also the larger networking buffers of 2x 2k. As David mentioned, the wifi uses 2x 2k buffers (one send and one receive) for the SPI transfers.

The LPC only has i think only 1/2 the memory of the duet2 boards and a some compromises was made to get everything including networking and lcd to all fit using a single binary. Plus the memory isn't a contiguous 64k which makes it a bit harder to manage too (and also some buffers must be in placed in AHB ram, i.e. for ethernet to use dma). I started putting up some info on the wiki on github and there is a page about the flash and memory layout used in the LPC port: [github.com]

Edited 1 time(s). Last edit at 01/09/2020 12:08AM by sdavi.

@dc42
Did anything ever come of the guy working on an STM port of RRF?

---

Based in Darlington, North East

Quote
jay_s
@dc42
Did anything ever come of the guy working on an STM port of RRF?

I have emailed him, linking to this post.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
jay_s
@dc42
Did anything ever come of the guy working on an STM port of RRF?

I've had this response:

Quote

The port for stm32f4 in is almost ready, and successfully works on several printers, but still there are some bugs, besides it is necessary to clean up the code. By the end of the month, I'll post it on github and send a link. I planned to finish it earlier, but I don't have enough free time

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42

Quote
jay_s
@dc42
Did anything ever come of the guy working on an STM port of RRF?

I've had this response:

Quote

The port for stm32f4 in is almost ready, and successfully works on several printers, but still there are some bugs, besides it is necessary to clean up the code. By the end of the month, I'll post it on github and send a link. I planned to finish it earlier, but I don't have enough free time

Sounds very promising. I have an STM32F103 based board that I can use for testing if he needs any extra feedback

---

Based in Darlington, North East

question about the ESP-Wifi integration:

Is the ESP unit working as a bridge to communicate to the network or is it working as an integrated piece of hardware with the LPC processor?

What if any code runs on the ESP unit?

Perhaps I misunderstand, do we need both Ethernet AND Wifi modules compiled in to the FW?

Quote
sinned
question about the ESP-Wifi integration:

Is the ESP unit working as a bridge to communicate to the network or is it working as an integrated piece of hardware with the LPC processor?

What if any code runs on the ESP unit?

Perhaps I misunderstand, do we need both Ethernet AND Wifi modules compiled in to the FW?

The built-in ethernet uses the onboard EMAC which receives frames from the ethernet. It then requires a TCP/IP stack (in the LPC port uses FreeRTOS +TCP) to process the frames and it also handles ARP and DHCP etc. The RRF networking code then reads the data from the sockets to process the HTTP requests etc.

The ESP8826 WIFI in RRF essentially moves the TCP/IP stack etc off the main processor to the ESP (freeing up processing time on the mainboard CPU) and runs a firmware also written by David: [github.com] that uses its own messaging format to talk between each other over SPI. The ESP does all connecting to access points, processing packets, connecting sockets and has memory for buffers etc. Data from the sockets are sent to the RRF networking code on the mainboard to handle the serving and parsing HTTP etc.

My current RRF3 build script produces 2 firmwares: a Wifi version and an ethernet version.

Edited 2 time(s). Last edit at 01/11/2020 02:36AM by sdavi.

sdavi -

thanks for explaining it so well. I have forgotten so much of this since college.

1. so by moving the tcp/ip stack off the LPC,does it gain us any more memory to the system as well? is this desirable to do anyhow?

2. what pins does the esp/wifi unit need to work? will it free up more pins on the LPC boards then?

3. can you compare performance overall between Eth0 and Wifi? is it measurable really?

4. Is DWC then running on the ESP or on the main CPU?

You are really making it hard not to migrate to RRF3!

Having just wired up an esp I can try and help with answering number 2. The pins you need are....
1. Access to all of the pins for SPI0 so MISO0, MOSI0, SCK0, SSEL0. Note that on many LPC boards these pins pins are used to perform I/O to the external SD card reader (or LCD), so are on the EXP1/2 header.
2. 3 additional GPIO pins (for reset and data ready in/out) one of which must be on port 0 or port 2 (because interrupt on change is needed).
3. If you want to be able to update the ESP firmware via the LPC board then you also need to connect the TX/RX lines of a hardware UART. I'm not sure if this works at the moment as I've not hooked them up (and I'm not sure which UART device on LPC is needed).
4. Power gnd and either 3.3V or 5V depending upon which esp module you use.

So basically you need 4 connections for the SPI, 3 GPIO connections and two power, plus the optional 2 UART connections. Most of these can probably be hooked up to the pins normally used to attach a display (certainly on the SKR boards). Given that it does not seem currently possible to operate both the display and WiFi (due to memory I think), this is probably not a big deal.

gloomyandy

Did you add the resistors etc as per the duet schematic here? [github.com]

---

Based in Darlington, North East

Yes I did, I'm not sure if they are strictly needed or not, but I had them, so I put them all in place, same values as used on the duet. At the moment I have all of this wired up on a breadboard, but at some point I'll probably build a small adaptor board that plugs into the EXP1/EXP2 headers.

Great.
I appear to have the firmware running correctly (RRF3).
I've not built it into anything, but my next task is to get the wifi work and then push my additional SKR 1.4 code to the repo. I've just ordered an ESP-07 so I can have the external antenna
Have you had a look into the SPI/UART at all for the drivers?

Edited 2 time(s). Last edit at 01/16/2020 05:54AM by jay_s.

---

Based in Darlington, North East

Quote
sinned
sdavi -

thanks for explaining it so well. I have forgotten so much of this since college.

1. so by moving the tcp/ip stack off the LPC,does it gain us any more memory to the system as well? is this desirable to do anyhow?

2. what pins does the esp/wifi unit need to work? will it free up more pins on the LPC boards then?

3. can you compare performance overall between Eth0 and Wifi? is it measurable really?

4. Is DWC then running on the ESP or on the main CPU?

You are really making it hard not to migrate to RRF3!

1. Memory at the moment is more with wifi as I've left all the big 2K buffers and increased the buffer for the SDCard. Offloading the tcp/ip frees cycles on the main cpu which is a good thing. Memory is pretty tight with wifi in the latest 3.01 so some of those buffers probably will get slashed.
2. In addition to @gloomyandy answer above, boards such as the rearm that don't have phy chip onboard will free up the ENET pins which can be used as GPIO.
3. In the current configuration, wifi gets around 210K/s and eth (in rrf3 using DMA for SDCard writes) gets up to around 140K/s.
4. In both the eth and wifi, the http processing etc is all done on the main CPU.

Quote
gloomyandy
Given that it does not seem currently possible to operate both the display and WiFi (due to memory I think), this is probably not a big deal.

The WIFI is also running as a SPI Master.

@jay_s for SPI/UART driver stuff see my recent comment on the RepRap V3 thread (I think it makes more sense to have the comments there, as these changes are based on V3).

Quote
sdavi

Quote
gloomyandy
Given that it does not seem currently possible to operate both the display and WiFi (due to memory I think), this is probably not a big deal.

The WIFI is also running as a SPI Master.

Ah yes but I was thinking that for some boards (like some of the SKR boards) that do not use the hardware SPI to drive the LCD it might be possible to still operate the display (but no external SD card reader) at the same time as using the WiFi, but memory may be the bigger problem.

I'd actually rather like to be able to have a minimal LCD menu system (just the main status screen, with perhaps the ability to pause/abort a job), if it was possible to fit it all it. That way I could have a local display on the printer that showed basic status. But I suspect that memory is just going to get tighter as RepRap V3.1 etc. comes out.

@gloomyandy

Why not do it with an ESP and OLED display?
Here's some basic monitoring code. [github.com]

---

Based in Darlington, North East

@jay_s Mainly because I've got a bunch of standard displays kicking around already and I'd not really considered it that much! :-)

Quote
gloomyandy
Ah yes but I was thinking that for some boards (like some of the SKR boards) that do not use the hardware SPI to drive the LCD it might be possible to still operate the display (but no external SD card reader) at the same time as using the WiFi, but memory may be the bigger problem.

I'd actually rather like to be able to have a minimal LCD menu system (just the main status screen, with perhaps the ability to pause/abort a job), if it was possible to fit it all it. That way I could have a local display on the printer that showed basic status. But I suspect that memory is just going to get tighter as RepRap V3.1 etc. comes out.

I think the LCD uses somewhere around 2.5K or so, but both should fit by tweaking the wifi buffers (see the v3 thread).

I think that a 1264 GLCD is nice too have for BASIC machine control. Other than that, one of the plugins i used to use with octoprint was a 4 line display that parsed the state change strings and would output on the LCD. It would work in conjunction with Octoprint, ATX-Pi breakout hat, and PSU_Control.

i miss that having switched to LPC port. It would give the state, the temps, percent complete and time estimate.



I want to try and see if this can can be rolled in somehow, eventually.

Quote
sinned
i miss that having switched to LPC port. It would give the state, the temps, percent complete and time estimate.

Those can all be displayed. [duet3d.dozuki.com]

Edited 1 time(s). Last edit at 01/17/2020 03:30PM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

dc42: are you saying that the LPC subsystem is already spitting out the datastream ?

what I was refering to was an LCD connected to the raspi. the data stream parsed by Octoprint and processed through the raspi/GPIO pins to a 4 line 20x4 HD44780 display.

[github.com]

there was another one that did it with I2C -

[plugins.octoprint.org]

see in my pic that the 12864 is connected to the RAMPs? the 20x4 is connected to the pi3B+.

Quote
sinned
dc42: are you saying that the LPC subsystem is already spitting out the datastream ?

what I was refering to was an LCD connected to the raspi. the data stream parsed by Octoprint and processed through the raspi/GPIO pins to a 4 line 20x4 HD44780 display.

[github.com]

there was another one that did it with I2C -

[plugins.octoprint.org]

see in my pic that the 12864 is connected to the RAMPs? the 20x4 is connected to the pi3B+.

You can also get access to all that info from M408: [duet3d.dozuki.com] which has access to a whole lot of data about the machine, print progress etc etc all packaged in a nice JSON format. Or if using networking, can be retrieved from rr_status as DWC does (also in a nice JSON format).

I have a Pi3 + touchscreen to display relevant info for my CNC that is running LPC port of RRF. I made a custom gui that uses the rr_status from networking and just pick out the values I want to display like progress, positions etc.

Edited 1 time(s). Last edit at 01/17/2020 07:12PM by sdavi.

sure, that works!

but the thing is you need some sort of device to parse it and direct it to a display.

Since I longer use the raspi with RRF, what is missing is the computer to process the M408 data and a means to display it.

I am curious how you are parsing the M408 data and how far back a data stream you archive

Quote
sinned
sure, that works!

but the thing is you need some sort of device to parse it and direct it to a display.

Since I longer use the raspi with RRF, what is missing is the computer to process the M408 data and a means to display it.

I am curious how you are parsing the M408 data and how far back a data stream you archive

There are heaps of JSON parsers out there for most languages these days.

I didn't want a computer out with the dust from the CNC, so wanted something where I could display info, do some movements and zero (or set) axes, select files to run from SDCard etc. I run raspbian on the pi3 modified to run as a readonly filesystem and wrote the code in c++ using QT5 with EGLFS so there is no need for X or anything so it boots up fast. It makes use of the multitouch LCD so you have to hold a "modifier" button down for larger movements when jogging the axes to prevent accidentally hitting i.e. 100mm and hitting something or snapping the endmill etc. I just have a timer that grabs the JSON data periodically and use the JSON parser within QT to parse it and extract the data I want. It displays it as the data comes in, the only thing i keep past data of is the "console" messages (like it does in DWC). This is one of the things i like about RRF, is being able to grab the raw data in one simple request which makes it easy to create custom GUIs etc.

The M409 command in RRF3.01beta1 provides even more information.

Edited 1 time(s). Last edit at 01/18/2020 02:46AM by dc42.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Dear all! Can you tell me in short your opinion is it worth compiling and running RRF3 instead latest RRF2 binary?

Edited 3 time(s). Last edit at 01/20/2020 08:23AM by PB.