Electronics and software combination questions

I am looking at making a 3d printer, and have read a lot and watched a lot of youtube vids on it.
I originally was going to make a CNC router, and the electronics and software seem to be the same.
My job is working with and programming for AutoCad/Bricscad at a civil enginerring firm.
As I approach the 3d printer world, I want to understand these things:

1) Is there complete separation between the model data, and the software that reads it and sends messages to motors and other items when printing? I ask because it seems to me the industry attempts to make the data be g-code, maybe wrapped in an stl file or other. Are there flavors of g-code that force the use of certain "slicer" tools for a given controller setup?

2) I notice many electronics use an arduino, then a shield with other stuff like motor drivers. I am not that interested in the drivers part, as that is not something I can mess much with. The arduino part is my interest, as that means there is a program that reads the g-code, then sends signals to the board to send voltages to the outputs. I understand arduinos fairly well.
My question is how open source that software is, and if the better ones are not open source.
I basically would like to port the tools to microsoft .net, as that is my language of familiarity (C#).
I want to use a netduino instead of arduino. I also want to understand previous software efforts better to re-use what has been done and share back.

Part of all this is I want to tie together AutoCad/Bricscad with the controller. I may want to make things that follow your hand movement and so on. There is a project at netduino.com that has code for interpreting g-code and controlling steppers.
Any links I should read, or comments?
thanks

I noticed the sanguino site is open source, I will read up on that as it seems to be a very popular setup.
So much stuff available I seemed to have missed the obvious.

another reply to myself, this link is helpful:
[www.reprap.org]

looks like most firmware (the part that interprets gcode) is free.
This is unexpected, as my research for CNC a while back did not have many free firmwares.
I recall one was mach3, not cheap.
I think the current state of firmware is easier to deal with, many options.

Firmware running on Arduino, open source Marlin firmware,
ingests G-Code and makes motors and heaters do what is necessary to print.

A set of software programs running on desktop computer generates G-Code and controls printer

Slicer software, Skeinforge, Kissslicer, Slic3r, takes model and produces layers then G-code
to be feed to firmware to print item.

From the printer control software, Repetier-Host, one can click on arrows on screen
and drive printer to any x, y, z position.

If you want something to follow your CAD/CAM stick with a robot.

???? AutoCAD solid to real movement of a machine ---- "Beam me up Scotty"

Perhaps a 3doodle in your hand while watching AutoCAD screen

[www.the3doodler.com]

Netduino is one of the lesser known platforms, so I guess you will find very little support for it around RepRap. All the Reprap software is open source, either C or C++, so you could try porting that to C#, but I think I would use it as a guide and start from scratch.

I would be interested to see what sort of step rates can be achieved with the .NET framework, that would be my main concern.

Netduino is 3.3V platform, so you would need a 3.3V compatible printer shield, and there appears to be no Netduino with a Mega type layout, so you might run short of pins.

If you are up for a challenge, developing Netduino based hardware and software would be a good project, but if you want to get things done more quickly then perhaps adopting a ready to go platform would be better.

great responses, thak so far. I love the Space Balls reference
3DDoodler! now that is something I could bring to a boring meeting.

I totally agree the existing tools are excellent and not in need of re-inventing. I was expecting them to be non-open and expensive but I was off there. Any re-doing would be to educate me, but possibly useful for CNC projects too.

Its actually generally very easy to port sketch code to C# for netduino, not sure if C code is as easy.
I'll check on the shield issue with the netduino folks.

jmaeding Wrote:
-------------------------------------------------------
> I was expecting
> them to be non-open and expensive but I was off
> there.

Yep. The RepRap project was built around free and open source.

jmaeding Wrote:
----------------------------------------------------

> I totally agree the existing tools are excellent
> and not in need of re-inventing. I was expecting
> them to be non-open and expensive but I was off
> there. Any re-doing would be to educate me, but
> possibly useful for CNC projects too.


Reprap is all about open source. For more comprehensive electronics packages that could be used for either printers or CNC check out Smoothie board [smoothieware.org] (though it's an ARM chip) or for Atmel/Arduino RAMBo [reprap.org] They basically add a stepper and some mosfets and sensor inputs.

A list of some of the options is at [reprap.org]

1) Yes. The differences of G-code flavours is small and can be handled by all slicers.

2) Single board ATmega based RepRap electronics are Arduino compatible, to the extents they use Arduino IDE for uploading the firmware. Everything you can run on an Arduino runs on a RepRap electronics, too. Just a bit faster in case of a Gen7.

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

I'm also in the process of building a 3D printer.

Just out of curiosity why is it necessary for the Arduino to interpret G-Code. Surely it would make more sense to do that on a PC and send simpler data to the Arduino to reduce the amount of work it has to do?

...R

Robin2 Wrote:
-------------------------------------------------------
> I'm also in the process of building a 3D printer.
>
>
> Just out of curiosity why is it necessary for the
> Arduino to interpret G-Code. Surely it would make
> more sense to do that on a PC and send simpler
> data to the Arduino to reduce the amount of work
> it has to do?
>
> ...R

GCode is not designed to be a communication protocol, but it does provide a standardised set of commands.

GCode is pretty simple already. If you did not use GCode, what simpler data would you send?

there are several non standard formats to use. some are directly controlled by c++, it all depends on the vendor, and use of the tool. gcode is just a standard. if you did not use gcode you might use snap as a gen1 example [reprap.org]

gcode is a protocol used to know what a tool needs to do for a numeric controller. [en.wikipedia.org]
it is used to communicate with a computer.

by using a standard we can reduce the learning curve tremendously. this is the same for use of the arduino. we are using a standard platform, that has had hundreds of thousands of man hours into its development and use.

The reason the platforms are used is simple it saves us all the hassle of failure, because the protocols and platforms have most of the bugs already fixed. this reduces our work tremendously to just the 3d printing aspects which are complex enough to last 6 months or more for learning curve.

If someone wants to still use gcode, and go down to just motor control, it can be done but i would focus on using gcode and interpreting it to something else, and only doing that simpler task of sending motor controls, because that simplifies design process. then it is up to the community to decide if such a process deserves merit for development.

many beginners use reprap to learn how to program and use electronics out of necessity, and really like the fact that no matter what they use, it seems to magically work together in the end.

thx for further responses all!
I had thought netduino platform might be worth pursuing simply because I program in C# already.
Also, there is a .net API for AutoCad, and the two naturally go together (or should).

Of all the reprap things I am focused on though, the main ones are the ability to make the printable area whatever size I want, and the development of the hotends. Seems like there is a lot of innovation in the hotend area, as well as room for improvement.
Then there is the whole DIY extruder thing going on to make cheaper filament.
I'd like to get a printer going sooner than doing netduino would allow, so will just keep my eyes on possibilities as I go.
fun stuff.

I see the value in using GCode as a standard interface between the drawings and the moves that the tool needs to make.

However the GCode to move along a straight line to a destination (and all moves reduce to that) must be converted by the Arduino into a series of coordinated motor steps. It seems to me it would be much easier to work out the steps and the coordination on the PC, and it wouldn't have to be done in real time.

...R


bobc Wrote:

> GCode is pretty simple already. If you did not use
> GCode, what simpler data would you send?

Quote

Surely it would make more sense to do that on a PC and send simpler data to the Arduino to reduce the amount of work it has to do?

Has been discussed and tried many times and the result is to keep G-code.

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

That's interesting. I would love to read some of that stuff. Can you give me any links to it?

Thanks

...R


Traumflug Wrote:
-------------------------------------------------------

>
> Has been discussed and tried many times and the
> result is to keep G-code.

I recall there being latency issues if you used USB. So if the firmware is doing the controlling, its right there so no delays.

jmaeding Wrote:
-------------------------------------------------------

> I'd like to get a printer going sooner than doing
> netduino would allow, so will just keep my eyes on
> possibilities as I go.
> fun stuff.

You should get a kit or source the parts and have at it. . That's the best way to learn. There is a learning curve with the machine itself not counting any of the mods/enhancements you want to try. By building a know printer and applying your mods from there will provide you with a good foundation.

I don't see netduino gaining traction with the open source community. Since you can code in C# knowing programming fundamentals it will take you less time to learn C++ and Arduino than to port the firmware to netduino. You'll be able to participate in current projects and have more people participate in yours. Though if you want to port it because you can, that's cool too. It's what open source is all about.

You are quite right that USB latency causes problems if you try to drive stepper motors directly with it in the same way that you would with an old-style parallel printer port.

That means that an Arduino (or equivalent) is necessary to act as a buffer between the PC USB port and the stepper motors. I just don't understand if or why it's necessary to give the Arduino the burden of interpreting GCode when the PC could do that and send pre-processed commands to the Arduino thereby reducing the amount of work the Arduino has to do. Programming on the PC is much easier than on an Arduino with limited memory and a slow processor.

...R

jmaeding Wrote:
-------------------------------------------------------
> I recall there being latency issues if you used
> USB. So if the firmware is doing the controlling,
> its right there so no delays.

when looking into the delay realize it is for check sum return. sub has a 1ms cycle where it checks for incoming data. if buffered and asynchronous, then it should not matter. especially with error correction like what is used in satellite communication. the idea is it is ok to send more data as bandwidth is less of an issue than resending data.

this was what i was talking about in 2011. [reprap.org]

and a method to do so discussed here: [forums.reprap.org]

Robin2 Wrote:
-------------------------------------------------------
> That means that an Arduino (or equivalent) is
> necessary to act as a buffer between the PC USB
> port and the stepper motors. I just don't
> understand if or why it's necessary to give the
> Arduino the burden of interpreting GCode when the
> PC could do that and send pre-processed commands
> to the Arduino thereby reducing the amount of work
> the Arduino has to do. Programming on the PC is
> much easier than on an Arduino with limited memory
> and a slow processor.

I think you are right in principle, a PC has a 64 bit CPU running at 3GHz, 4GB RAM, and Arduino has 8 bit running at 20 MHz with 8KB. At first, I also thought it was quite crazy to send data to the Arduino for it to process, given the difference in processing power.

To an extent, the Gcode can be reduced to a series of "wait n; pulse [x|y|z|e]" type steps, which could be precalculated in advance. I'll call this "Scode". There is some need for real time control, for example, dealing with endstops, and waiting for extruder to reach desired temperature. There may also be user interaction such as pausing, or changing the speed. Those could probably be dealt with.

However, if you have 80 steps per mm, then one line of gcode could become hundreds of SCode lines. That could be compressed in various ways, for example if the time period does not change then a "repeat x;" operations could be used.

If you generated Scode, then it must build in parameters of the printer, like steps/mm, so you would need to be careful to regenerate the Scode if any parameter changes, or for different printers.

So the Scode still needs some interpretation by the Arduino, but worse, the volume of data transferred becomes bigger. This may exceed the bandwith of the comms link, which can already saturate for a lot of small moves eg. in arcs. Plus, you have a machine specific Scode which is not transferable.

With that in mind, it is now less obvious that using SCode would have significant benefit, and might even be worse, i.e. requiring a bigger processor on the printer side and a faster comms link.

One factor that makes optimisation somewhat redundant is that processors become bigger and faster every year. The early RepRaps used smaller processors. Now we can get 32 bit 100MHz CPU for around the same price as 8 bit 20Mhz CPU. I am sure that in a couple of years that will change again. We may even have micros that can do their own slicing. The trend is clearly towards the printer becoming smarter, not simpler.

On the history of RepRap. It is not a necessary feature, but does seem to be common, that open source projects tend to be more evolutionary rather than starting with a known spec and developing to a spec. I guess that is because OS are developed by volunteers and academics, who don't need to make a profit, as opposed to businesses that need to meet a production deadline. They need to sell a product or go bust!

Additionally, a closed source product must have a "theory and operating guide", since there is no other way for the user to see how the product works. With OS, all the design files are freely available, so the user can simply look at those. Plus writing a guide takes time and effort, especially if the design is continually changing.

Unfortunately, that does mean some of the design decisions become a bit lost in time, so it is worth asking occasionally "why is it done like this?" to see if assumptions no longer apply and if things can be done better,

With OS, the standard response if someone says "there is a better way to do this" is "go ahead and do it!". If it is better, people will (probably) adopt it. This sort of selection of ideas is also part of the evolutionary nature of OS projects.

Wise words from BOBC, as always his comments are based on wisdom, I always listen to what he says, its usually right.

---

Random Precision

jamesdanielv Wrote:
-------------------------------------------------------
> this was what i was talking about in 2011.
> [reprap.org]
>
> and a method to do so discussed here:
> [forums.reprap.org]
> g-75207

Did it ever get out of Phase 1? I am not quite convinced bandwidth is not a problem, so it would be interesting to see some test data.

After finding several bugs in the Gcode checksum implementation, I came to the conclusion that the current checksum method is very poor and not fit for purpose. For example, if the "*" is corrupted, the firmware doesn't do any check and goes ahead and uses the data! What it really needs is a reliable framing protocol, like HDLC, before the error correction can even work properly.

The reason I found the problems was that a firmware bug randomly corrupted bytes, which turned out to be a good test of the error correction! 20% of the time the correction failed and the comms hangs.

I think the only reason people don't see a problem is that the comms is generally 100% reliable, and the checksum is not needed anyway. In the cases where people have comms problems they "fix" the problem by printing from SD.

There is another factor to be remembered here, which is that the Arduino is not trying to execute a whole operating system, with possibly a mountain of other (spurious to Reprap) operations that can require significant resources and time to execute, and some of the reprap functions are time critical. If the operation is downloaded to a dedicated processor, then that processor has no reason to care or respect any other process, so it can be devoted to doing what is needed,

I note the comments about errors in the check sum area with interest, that may be something that needs to be looked into by the developers of the core code, and hopefully fixed. I'm just getting into this whole scenario at the moment, so things like this that catch my attention tend to get noted in a scratchpad file as a possible gotcha that might come back to haunt me at a later date.

What is clear is that this is a fast moving and somewhat volatile area to be involved with, which could become challenging. I discovered a long time ago in relation to flight simulation that the only difference between the cutting edge and the bleeding edge is the amount of pressure that is being applied.

What is incredibly frustrating on occasions is that the innovators are often not good documentors, so a really good upgrade or enhancement is either passed by or sidelined because only the developer understands how it works, and others coming behind have a mountain to climb if they want to get to grips with the code, and depending on the complexity, sometimes, it's quicker to create a new version than it is to work out how the existing one is really working, or possibly, not working fully,

A long time ago, there was one piece of code in (I think) C that worked, but none of the programmers could actually explain in long hand what one small snippet of the code was actually doing, or why it was where it was, but if it wasn't in the source code, the module didn't provide the correct output. Very frustrating indeed when that happens, especially if changes are subsequently needed, but for some reason the modified version does not produce the expected output. Occasionally, it comes down to something as obscure as the "snippet" is forcing a piece of data to be aligned on a word boundary, or similar, and if changes are made, that assumption may no longer be true, and some hardware cannot then correctly process the unaligned data, and the snippet is forcing the compiler to execute an extra check or validation, and without it, there will be problems. What I can recall was that there was a comment in the code that went something like

We don't understand what the next 2 lines of code do, or why it is here at this point, but if you move it or change it,
the module will no longer work in the way that it has been for the last 5 years, so mess with this bit at your peril,
and if it no longer works, come back here first and make sure that this bit is still in here


the code then was in the source, followed by

You didn't mess with the last 2 lines of code, did you.
If you did, make sure that you document what was here and what is now here, because we can almost guarantee that at
some point before very long, you will be back here changing the code above to what it was before you started

YOU HAVE BEEN WARNED

I am about to start building a Delta, but I have still to find a good document that explains what all the offsets in the Marlin firmware really relate to, and how those measurements are determined. Part of the problem is getting to grips with understanding what the developers mean by some of their constants, and what changing them does, and it's become clear that the configuration of a Delta is a lot more parameter driven than an cartesian system, due to the interaction between the 3 towers to produce the Z movement. I just hope I don't end up regretting the Delta decision, pure maths never was my strong point, and making sense of complex formulae with layers of different bracket types tends to be head wrecking.

That said, I still reckon that a dedicated micro processor is a better option than trying to drive the printer from a PC based platform, especially if it is using Windows.

I still have painful memories of Windows deciding to defragment memory at a time when intense and time critical graphics updates were being done, so pausing or slowing down by a significant percentage, those sorts of issues caused us huge pain when we had to migrate some sections from the pure DOS environment that they had worked perfectly in for years.

The other aspect is that the micro code is probably open source, so can be debugged with far less pain than a closed source product like Windows, even finding out which area to look at in 4 or 5 Gb of executable code can be a nightmare, let alone then trying to work out what it is actually trying (or failing) to do. At least with open source, the designers original intention is usually reasonably clear, unless it's been written by a dyslexic spider .

.

---

Shore, if twas easy, we'd all be doin it

Irish Steve

The main reason I am wondering why GCode is interpreted on an Arduino is because I am trying to build a 3D printing platform on a small metal working lathe and I also want to use it as a CNC lathe. I have the Sprinter code but I don't think it is suitable for lathe work (and I haven't found any off-the-shelf alternative either) and it seems to me it would be a major task to learn it well enough to modify it confidently. As others have said the real downside of OpenSource is the absence of documentation.

I agree completely that for many / most users an off-the-shelf solution is ideal and essential - but, in another way, that's not really the spirit of Open Source.

I am dabbling with the idea of converting a GCode instruction into a few numbers that the Arduino will use as the time to execute the instruction and the intervals within that time at which it will make each of the motors step. It seems to work fine in about 200 lines of Arduino code. I realize, as someone else said, that producing an Arc requires a large number of short straight lines but my back-of-the-envelope calculations suggest that they can be sent to the Arduino more quickly than the Arduino can implement them. Of course it would be nice if there was documentation describing the design considerations of existing Open Source products so I don't fall into the same pits.

It's not necessary for the PC to process the GCode in real time - it would be sufficient to save the SCode (to borrow BOBC's name) in a file on the PC and just transfer the file line-by-line in real time.

I can't help feeling that if the Arduino code was being produced with commercial constraints the simplest solution would be chosen because it must be cheaper to develop and update PC software. It is a myth to think that Open Source software is free. The programmer always has a choice about how to use his/her time and consciously or unconsciously decides not to do other tasks that might be more useful to society. By producing unnecessarily complex Open Source products less time is available for those other things.

...R

Robin2 Wrote:
-------------------------------------------------------
> The main reason I am wondering why GCode is
> interpreted on an Arduino is because I am trying
> to build a 3D printing platform on a small metal
> working lathe and I also want to use it as a CNC
> lathe. I have the Sprinter code but I don't think
> it is suitable for lathe work (and I haven't found
> any off-the-shelf alternative either) and it seems
> to me it would be a major task to learn it well
> enough to modify it confidently. As others have
> said the real downside of OpenSource is the
> absence of documentation.

Have you looked at GRBL? It is designed for CNC work. [github.com]

From reading this introduction to Grbl I figure it would be very hard work to figure out what the code is doing. And I don't think there is a comprehensive user manual.

...R

Quote
From the Grbl website
The controller is written in highly optimized C utilizing every clever feature of the AVR-chips to achieve precise timing and asynchronous operation. It is able to m aintain more than 30kHz of stable, jitter free control pulses.

Robin2 Wrote:
-------------------------------------------------------
> From reading this introduction to Grbl I figure it
> would be very hard work to figure out what the
> code is doing. And I don't think there is a
> comprehensive user manual.

Ok, I think Open Source has nothing to offer you then.

You will need to develop your own solution or buy something ready made to your requirements.

Quote
Irish Steve
What is incredibly frustrating on occasions is that the innovators are often not good documentors

The incredible frustrating thing for developers is, people jump on everything fancy, regardless how faulty or sloppily it's coded. To date, the "major" firmwares still tend to produce step losses. Documenting, coding carefully and making sure algorithms follow physics are the most reliable ways to end up in the ignored zone.

Quote

From reading this introduction to Grbl I figure it would be very hard work to figure out what the code is doing.

Have a look at Teacup. Does linear acceleration (GRBL doesn't) and can even outperform GRBL with it's new ACCELERATION_CLOCK strategy. "Gen7" is Teacups experimental branch.

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

Traumflug Wrote:
-------------------------------------------------------
>

Quote
Irish Steve
What is incredibly
> frustrating on occasions is that the innovators
> are often not good documentors

>
> The incredible frustrating thing for developers
> is, people jump on everything fancy, regardless
> how faulty or sloppily it's coded. To date, the
> "major" firmwares still tend to produce step
> losses. Documenting, coding carefully and making
> sure algorithms follow physics are the most
> reliable ways to end up in the ignored zone.
>

Yes, that's another side of that same coin, and unfortunately, companies like Microsoft have led the charge towards eye candy over substance for so long, it's become pervasive in many areas, a little bit of eye candy will often get more response from reviewers than page after page of accurate stable and well performing code, which as you say is very frustrating.

I've been on both sides of that fence, having to get software to check between mechanical and electronic switch synchronisation was a nightmare on one task, and when we finally made it graphical, with colours indicating the switches that were wrong, the user was happy.

Equally, when a coder puts variables into a string, but doesn't give downstream users the information about the format, and possible range of that variable, working out what a strange hex string really means when it comes out of a "black box" network driver can be incredibly frustrating, and just a few lines of comments in a text file that came with the executable would have saved nearly a week of serious head scratching on one project, but that's the way these things go, and it's not just the open source community that is guilty, it happens with closed source code, which can be even worse, as there's no information in terms of source files to reveal what's going on.

The biggest nightmare is going back to a site to make a minor change a number of years down the line, and realising that the documentation is either so sparse, or so out of date, relying on it might be more dangerous that rewriting the section that needs to be changed. Even worse is when it goes missing.

Such is life,

---

Shore, if twas easy, we'd all be doin it

Irish Steve