Compiling RepRapFirmware on Linux without Eclipse

Hi!

I wrote a script to compile the Duet branch of the RepRapFirmware,
you only need to run 1 script instead of hacking until Eclipse works:

- It downloads a 32-bit or 64-bit Arduino 1.5.4
- It asks which firmware repository you want
- It applies dc42's ArduinoCorePatches folder
- It compiles all .c and .cpp files to object files
- It creates the binary in the Release folder

I don't know if anybody else has a need for this script,
but you never know until you publish it on some forum.

I tried to write a Makefile first, but I could not make it work,
probably because I know almost nothing about Makefiles.

In my second attempt I wrote this make.sh bash script,
it downloads the needed parts and compiles the firmware.

Most of the compiler options were copied from the
output of the Eclipse console, a few months ago.

The script uses dc42's recent -O3 speed optimization.

It would be awesome if somebody could convert this
script to a working Makefile, this project deserves it

Have fun!

Edited 1 time(s). Last edit at 05/28/2014 04:10PM by 3D-ES.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

Nice work! From a quick look at your script it looks like this builds as a standard C++ app ? ( rather than as occurs with the avr mcus, where the Arduino IDE mungs everything into one file. Yeeurch! ) I hadnt realised that. If we are back in the realm of standard C++ .. Makes it a much more attractive prospect to work on.

regards
Andy

---

Ormerod #318
www.zoomworks.org - Free and Open Source Stuff

Nice work indeed!

Do the RRP firmware sources work with my Arduino core patches? I was under the impression that they wouldn't, because one of the patches is to disable the startup code that turns off the watchdog. This means that you have to configure or disable the watchdog in the Duet startup code instead - and I didn't think the RRP source included code to do that.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

@kwikius

I have tried to mimic the output that was generated during the Eclipse compilation,
so I use the same tools with the same options to produce the same ELF executable.

In the last step this ELF file is converted to a binary that is suitable for the Duet board.

@dc42:

If the script finds a folder called ArduinoCorePatches it copies the files to the Arduino folder,
since the RepRapPro version of the firmware does not contain this folder, it will not be copied.

One thing that is different (and untested) is that it also compiles the RRP version with the -O3 option.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

Awesome. This might be what I need to finally be able to compile firmware on Mac. I have spent too many hours with Eclipse banging my head against the wall trying to make it work, and I would rather just do without it. Some path fiddling, and I suspect this shall work well on OS X too.

Quote
3D-ES
If the script finds a folder called ArduinoCorePatches it copies the files to the Arduino folder,
since the RepRapPro version of the firmware does not contain this folder, it will not be copied.

Great! Thanks for taking the trouble to support my fork. I'm still hoping that we can get back to a single RepRapFirmware sometime.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
dc42
I'm still hoping that we can get back to a single RepRapFirmware sometime.

So am I! But it's out of my hands...

Ian
RepRapPro tech support

Thank you very much 3D-ES I will be making use of this once I get my Ormerod

I have updated the script a bit:

- It only compiles changed files for faster recompilation.
- It shows less output to keep the compilation result clean.
- Added command line parameter 'clean' to clean project.
- Added command line parameter 'verbose' to show output.
- Removed the (not so interesting) ELF file statistics.
- Probably supports the MacOS platform for jstck...

@jstck:

Can you try to compile the firmware with this version?

It is probably able to detect and handle your platform,
since I don't own a Mac, I am unable to test this for you.

The MacOS Arduino archive contents is different,
I hope my move-and-rename-folders hack works...

Update: Probably repaired the OS detection...

Edited 1 time(s). Last edit at 05/30/2014 09:51AM by 3D-ES.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

Update: implemented uploading of the binary.

- The script should detect the programming port.
- The script tells you when & how to erase the board.
- The script waits until bossac *finally* finds the board.
- The board is erased, written, verified and reset.

To install the firmware you can type:

./make
./make install

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

I just tried it on Mac, and it successfully built the firmware. I haven't actually installed it yet, but at least I got a binary out of it which is more than Eclipse ever did for me... I just had to change this on line 226:

cp -r --preserve=timestamps ${FIRMWARE}/ArduinoCorePatches/sam ${ARDUINO}/hardware/arduino

to

cp -r -p ${FIRMWARE}/ArduinoCorePatches/sam ${ARDUINO}/hardware/arduino

This as the --preserve option does not exist on Mac. -p does a bit more (preserves more attributes than just timestamps), but seems to work. I suspect the whole thing would run fine even without that flag though.

Again, thanks for making this. I have very little patience for cranky IDE's and compilers, and this gets me right by the whole thing.

Awesome

That -p option is nice, thanks!

When I copy the patches, it updates the timestamps,
without that -p option it keeps compiling these files.

But indeed, it does no harm, it still produces a binary.

To install the firmware my script scans the list of connected devices.
I understand that your computer is not equipped with the 'lsusb' tool.

I found this project: [github.com]

Can you try to add this lsusb script to your computer?

Edited 2 time(s). Last edit at 05/31/2014 06:44AM by 3D-ES.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

I tried the supplied lsusb command, seems to work fine. Here's its output:

$ ./lsusb
Bus 026 Device 002: ID 8087:0024 Intel Corporation Hub
Bus 026 Device 003: ID 05ac:8510 Apple Inc. FaceTime HD Camera (Built-in)  Serial: CC2D8N0C8GND7F6B
Bus 029 Device 002: ID 8087:0024 Intel Corporation Hub
Bus 029 Device 003: ID 0424:2512 SMSC Hub
Bus 029 Device 005: ID 0a5c:4500 Broadcom Corp. BRCM20702 Hub
Bus 029 Device 006: ID 2341:003e 2341 Composite Device
Bus 029 Device 008: ID 05ac:828c Apple Inc. Bluetooth USB Host Controller
Bus 029 Device 004: ID 05ac:0259 Apple Inc. Apple Internal Keyboard / Trackpad
Bus 010 Device 001: ID 05ac:8007 Apple Inc. XHCI Root Hub USB 2.0 Simulation
Bus 026 Device 001: ID 05ac:8006 Apple Inc. EHCI Root Hub Simulation
Bus 010 Device 001: ID 05ac:8007 Apple Inc. XHCI Root Hub USB 2.0 Simulation
Bus 029 Device 001: ID 05ac:8006 Apple Inc. EHCI Root Hub Simulation

It's a MacBook Pro with nothing connected to it except the Duet (which would the "composite device" above). Also tried flashing the fw I built, and it went fine:

$ ./make.sh install
Erase your board to start the upload.
Please press MCU_ERASE and then RESET.

Found an empty board, starting upload.
Bossac is searching, this can take a while...
Bossac is searching, this can take a while...
Bossac is searching, this can take a while...
Device found on cu.usbmodem1d111
Erase flash
Write 242240 bytes to flash
[==============================] 100% (947/947 pages)
Verify 242240 bytes of flash
[==============================] 100% (947/947 pages)
Verify successful
Set boot flash true
CPU reset.

Though I usually just run the bossac command manually and probably will continue to do so, it's really nice that it works "automagically" as well.

One slightly odd thing is that I have seen the same Duet board with at least two different descriptions in lsusb:
Bus 029 Device 006: ID 2341:003e 2341 Composite Device
Bus 029 Device 006: ID 03eb:6124 Atmel Corporation Communication Device

Same bus/device, but different ID and strings. Does it present itself as different things depending on what stage it is in (erased or not, or some such)?

When the board contains no program, the device identifies itself as an Atmel Corporation device,
when you compile the firmware, variable USB_OPT contains the Arduino vendor and product ID,
so the device ID changes after flashing the firmware and the script knows the state of the board.

My PC shows the following output:

$ sudo lsusb -v -d 2341:003e

Bus 002 Device 115: ID 2341:003e Arduino SA
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x2341 Arduino SA
idProduct 0x003e
bcdDevice 1.00
iManufacturer 1 Arduino LLC
iProduct 2 Arduino Due
iSerial 0
bNumConfigurations 1

---8<---

Edited 1 time(s). Last edit at 05/31/2014 02:33PM by 3D-ES.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

Thank you for testing the 'lsusb' script! I have added it to my script.
If it installs Arduino on a Mac it also downloads the 'lsusb' script.

Edited 1 time(s). Last edit at 05/31/2014 03:39PM by 3D-ES.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

I found why bossac had so much trouble finding my board...

[bugs.launchpad.net]

There is a package called 'modemmanager' installed on my Linux PC,
when the Duet is attached, the manager tries to detect if it is a modem.

While this check takes place, bossac is unable to open the port!

I have uninstalled modem-manager and the delay is gone...

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

Update: added a check to prevent compilation of the ArduinoCorePatches folder

When I tried to compile RepRapFirmware-059e-dc42 there were error messages,
the compiler did not like the redefinitions of the names in the core patches folder...

Since the patched files are already compiled from their official location,
there is no need to also compile them when they wait in the patch folder.

---

RS-Online Ormerod #263, Kossel mini with Minitronics, Prusa i3 MK2

Struggle with Eclipse for few days with my Macbook until I found this one. Thanks for sharing 3D-ES

Hello everyone !

This script looks amazing, thank you guys ! I'm new on this forum, but I found many great things that's really nice
I'm building a big printer and I tried to modify the build dimensions in the firmware (in file plateform.h => AXIS_MAXIMA {520, 500, 850}) and then to compile it and upoad it to my board, but the printer doesn't work with my modified firmware.
Here is my github :
[github.com]

My new firmware is 235kb, but the original firmware is 206kb... It should have the same size, right ? (I only changed 3 values...). Maybe this is because of the new version of the firmware (1.09) and the script doesn't work, what do you think ?
I send my firmware if you want to test it.

If anyone could help me or compile my firmware that would be really nice, I spent some hours trying to fix it...

Thank you for your help !

Hello pelepicier,

You don't need to recompile the firmware to change the axis maxima, there is M208 for this purpose - the values in Platform.h are only default values.

But if you want to do other changes in the code, have a look at my repository: [github.com] There you can find a Makefile for both Windows and Linux, and you can find the necessary board files for the Duet in the Hardware directory.

Thank you very much !
Yes it works, it's perfect... I will send some pictures very soon

Very nice. It took me 3 evenings of pain to get Eclipse set up enough so that I could compile DC42's branch on Windows. If I get some free time in the next few weeks, I will look into creating a proper CMake build, based off your makefile. Maybe that will make it easier for people in future to compile the firmware - the current setup really is atrocious from a development perspective

Good news! It looks like experimental Arduino 1.6.5 support has been added to the Arduino Eclipse plugin again, so I'll update the Eclipse build instructions ASAP.

[edit]Updated instructions for Eclipse can be found here, but be aware that Arduino 1.6.4+ requires nightly builds of the Arduino Eclipse Plugin - YMMV![/edit]

Edited 1 time(s). Last edit at 07/09/2015 09:21PM by zombiepantslol.

Hi @3D-ES,

After spending two days struggling, and so far failing, to get Eclipse working on the Mac the discovery of your Make script came as a breath of fresh air. Really! Until, of course, I discovered it no longer works with the latest codebase. To me, a Make file seems a much better approach to the problem rather than installing (and then using) the behemoth that is Eclipse not to mention the broken Arduino plugin.

Is there anything I can do to help get this running again? I'm using a Mac but have Linux available too.

Best regards
Andy Bower

Quote
jstck
Again, thanks for making this. I have very little patience for cranky IDE's and compilers, and this gets me right by the whole thing.

i bought one of these boards because i like the idea of having a faster ARM Cortex (i normally use STM32Fs with libopencm3, and love the combination). i also liked the description of the RepRapFirmware being c++-based, it sounded extremely well-designed. i had *no idea* that there would be people working on it who don't bother to provide Makefiles - the absolute bedrock standard way of compiling and managing a project. that was a bit of a shock.

it's not just "patience" - it's complete insanity to think that it's ok to force people to install 150mbyte of crap runtime for a language that they've despised ever since it first came out [who thinks it's ok to have an OO language with only single-inheritance, and what the hell is it with being able to only pass arrays of *integers* by reference - anything else gets copied, i mean.. wtf??], then another bunch of crap that's again tens of megabytes... which does nothing more than *exactly* the same job as a standard set of command-line tools taking up a tenth of the space which have been proven to be highly effective and efficient at their tasks for several decades.

i'm running on a space-limited machine and constantly having to delete stuff. the thought of installing 300mb of crapware to compile up an executable that's *four* orders of magnitude smaller than the environment used to build it is just.... i *genuinely* don't understand why people think that's okay.

so, thank you 3D-ES - you've saved me several days of work - time that i don't have to spare. especially appreciated is that you've added support for dc42's fork, as i have designed and built a CoreXY printer.

[update only a few minutes later]

well, i ran the script. it took longer to download arduino-1.5.4 than it did to actually execute and compile the code. a *lot* longer. even so, it quite literally took.. shock-horror... *minutes* to run. aside from having to visit the hospital due to blowing blood-vessels and/or being committed to an asylum for going into rage-induced standing-alpha-wave EEG patterns over downloading jre, then downloading, installing and then *learning* Eclipse, the time comparison between typing "./make" and "go download some crapware which is clicky-point when you factor in how slow it would run on an already-overloaded computer"... i just... could someone please explain to me, in simple language, why any developer would want to work on such small source code projects with such _completely_ over-engineered IDEs? i just... i just *genuinely* genuinely don't understand.

Edited 2 time(s). Last edit at 04/07/2016 09:12PM by lkcl.

The reason why we use IDEs is that RepRapFirmware is no longer a simple project. There is no way I would be without some of the features found in all modern IDEs (except Arduino, which doesn't deserve to be called an IDE IMO), in particular the "Go to definition" and "Find all references" features. Generating makefiles for C and C++ projects is also a complex process, especially when there are multiple build configurations as there now are in some forks of RepRapFirmware.

I am not especially fond of Eclipse, I prefer Microsoft Visual Studio, or Atmel Software Framework. But those are Windows-only IDEs, and the community expects to be able to build firmware under Linux and Mac as well. Eclipse has the advantage of being cross-platform, and have have learned to live with it. Now that I have eliminated the need for the Arduino plug-in, it is fairly straightforward to set up an Eclipse build environment for RepRapFirmware.

I can see that users who want to make just small changes to RepRapFirmware may prefer to use just an editor and a make program, so I am happy that others are providing suitable makefiles. Perhaps one day I will get into the habit of checking in the makefiles that Eclipse generates.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Quote
lkcl
i had *no idea* that there would be people working on it who don't bother to provide Makefiles - the absolute bedrock standard way of compiling and managing a project. that was a bit of a shock.

See my firmware fork at [github.com], it provides a typical Makefile and a build script for Windows. Just make sure you install my Arduino Duet board before you attempt to compile it (requires the latest Arduino version). I managed to get it to compile with dc42's fork as well, but I haven't tried that with his latest firmware version yet.

Quote
lkcl
i'm running on a space-limited machine and constantly having to delete stuff. the thought of installing 300mb of crapware to compile up an executable that's *four* orders of magnitude smaller than the environment used to build it is just....

[www.amazon.co.uk]
[www.amazon.co.uk]
HTH

Dave

Edited 1 time(s). Last edit at 04/08/2016 07:43AM by dmould.

Quote
dc42
The reason why we use IDEs is that RepRapFirmware is no longer a simple project. There is no way I would be without some of the features found in all modern IDEs (except Arduino, which doesn't deserve to be called an IDE IMO), in particular the "Go to definition" and "Find all references" features.

exuberant-ctags and vim (or emacs - not a fan but i know people are) have exactly those features: after running "ctags -R", in vim you simply press ctrl-] and, regardless of the programming language, you get automatically taken to the definition of the function [or struct, or class, or variable]. typing ":tn" takes you to the next definition if there are several. i think emacs might actually have built-in definition-search and "find-all-references" - it is an enormous lisp program, after all.

Quote

Generating makefiles for C and C++ projects is also a complex process, especially when there are multiple build configurations as there now are in some forks of RepRapFirmware.

that's a problem that's been solved for almost thirty years [without requiring 300mb of java]. i'm a fan of autoconf because i was one of the major contributors to samba back in 1996-2000, but i know that many people are fond of scons, cmake, and so on. yes you're absolutely right - the practice of storing configuration info about the OS in Makefiles themselves was something that created absolute dog's dinner messes in the Unix community for 15+ years... then, thank god, autoconf came along.

in this case however, the advantages of autoconf [etc.] are slightly taken away because this is cross-compiling, not native compiling. autoconf [etc.] do host-level checking (what's the size of an int, what's the size of long, etc. etc.] which used to be the major cause of compile-headaches when there were 20+ different kinds of Unix around...

Quote

I am not especially fond of Eclipse, I prefer Microsoft Visual Studio, or Atmel Software Framework. But those are Windows-only IDEs, and the community expects to be able to build firmware under Linux and Mac as well. Eclipse has the advantage of being cross-platform, and have have learned to live with it. Now that I have eliminated the need for the Arduino plug-in, it is fairly straightforward to set up an Eclipse build environment for RepRapFirmware.

I can see that users who want to make just small changes to RepRapFirmware may prefer to use just an editor and a make program, so I am happy that others are providing suitable makefiles. Perhaps one day I will get into the habit of checking in the makefiles that Eclipse generates.

six of one, half a dozen of the other, as it's a compile-target (auto-generated) it's generally bad practice to check in "downstream" auto-generated files... but in this case as people (such as chrishamm and 3d-es) are actually creating substitute systems including makefiles, it might help.... i'm hand-waving, here the only big software libre project i know that's broken this rule is python: they checked in the "configure" script into the repository (which is a bad idea, it's auto-generated from configure.in) - but the justification there was to be able to keep a close eye on any changes (made by an upgrade to autoconf *or* by any contributors).

dc42, if i get into ReprapFirmware in a big way, i'll help out, ok?

Quote
dmould
[www.amazon.co.uk]
[www.amazon.co.uk]
HTH

Dave

appreciated, dave - i have a macbook pro running Debian, with a 250gb SSD, i'm a little reluctant to open it up. i used to build my own PCs [i've done around 50] - but laptops... i've seen the horror stories of people puncturing the laptop battery from apple macbooks.... i'm really reluctant to take the risk. nice thought, though.