I would put a volt meter on the output of the Adruino regulator. It's a long shot, but we may be pulling more current than they would like us to pull.....

I've pushed a major board layout change to bobc via github.

We've gone back to driving the FETs directly out of the ESTOP drivers using 5V, (using HCT/ACT chips) for the moment. Also increased the board size by 6mm to 95mm x 100mm. Moved the '125 closer to the 4 FETs it drives (need the space in the middle of the board and it reduces drive impedance to the FETs), and added an eeprom (same one as on RADDS, and on the same pins).

Regarding current, we've already mentioned we need more current so that would be my guess too, though admittedly with the Mega, drawing current from IOREF is probably a bad idea.

Note: We're nowhere near anything stable yet. There's a bunch of other changes to come and bugs to address before we're ready to consider making another run of boards.

I've made some progress in debugging my ADC errors. The symptom is easy to reproduce: whenever a heater is on, the ADC readings go up (temp readings go down), permanently, until the heater is turned off. More heaters, and more heater current, produces a higher error.



I've been measuring voltage between the ground connection of the thermistors and Due's GND pins next to Vin (presumably these have the best connection to GNDANA on the SAM3X8E). I find that the errors correspond to an increased and positive voltage here. In the worst case, with all heaters on, the voltage is about 0.03V. This must be due to some currents related to heaters being on. So I added a wire connection to the bottom of the board.



The wire decreased the voltage to 0.005V worst case, and the temp errors have decreased with that. I suspected there's extra current coming into my wire from the on-board connection of thermistor ground, so I cut that trace (the one at the bottom ground pin on my picture). Surprisingly, it made things worse, voltage has risen to 0.01V.

So I'm not sure what the unwanted current is due to, but I believe the thermistors should have a more direct connection to the Due, both ground and 3.3V. And ground should probably go to the two GND pins next to Vin, since I think these have the best connection to GNDANA.

P.S. credit goes to Triffid_hunter on IRC for suggesting a voltage may be present between GNDANA and thermistor grounds.

Edited 1 time(s). Last edit at 12/22/2013 09:26AM by ambrop7.

Ok, so I've added another wire to my setup, from where I soldered the existing wire to GND trace (top-left end of the wire in the picture), to the GND pin on the Due board, the one next to Vin (I soldered the wire to the bottom side of the board). I've also fixed the trace I've previously broken on purpose. I now measure thermistor-ground-to-due-ground voltage no more than 0.002V, and temperature measurements have also improved. I think turning bed on and off only offsets the ADC by 1/1024 (still haven't gotten around to using it in 12-bit mode, but this issue suggests it may not be worth it...).

Quote
ambrop7
I've been measuring voltage between the ground connection of the thermistors and Due's GND pins next to Vin (presumably these have the best connection to GNDANA on the SAM3X8E). I find that the errors correspond to an increased and positive voltage here. In the worst case, with all heaters on, the voltage is about 0.03V. This must be due to some currents related to heaters being on. So I added a wire connection to the bottom of the board.

Great catch.

Quote
ambrop7
The wire decreased the voltage to 0.005V worst case, and the temp errors have decreased with that. I suspected there's extra current coming into my wire from the on-board connection of thermistor ground, so I cut that trace (the one at the bottom ground pin on my picture). Surprisingly, it made things worse, voltage has risen to 0.01V.

The whole ground area in that section is also Ground to the Endstops. Can you perhaps disconnect your Endstops (and that wire) and see if there is any difference? Also, are you using Micro-switches or Hall-0/Optos? Would make some sense if you're using Hall-0/Optos, as they draw a continuous current.

FWIW: We're using 10K pullups for Micro-switches, but the Atmega series internally has 20K pullups, while the Due internally has 100k pullups. We could change RP801/802 to something higher (or change all the resistor packs to reduce current all-round).

Quote
ambrop7
So I'm not sure what the unwanted current is due to, but I believe the thermistors should have a more direct connection to the Due, both ground and 3.3V. And ground should probably go to the two GND pins next to Vin, since I think these have the best connection to GNDANA.

P.S. credit goes to Triffid_hunter on IRC for suggesting a voltage may be present between GNDANA and thermistor grounds.

I'll see what I can do about the Ground tracks, and where possible the thermistor circuitry. That area is a bit of a bottleneck in the board (particularly around the hole near Due pin 21), so not sure how much improvement I can give to the thermistor circuitry.

The tracks around the 2 grounds next to VIN aren't as big as they could be, so IMO there is room for improvement. The area is a bit cramped due to the Pololu sockets and the Arduino pins being so close together.

I am wondering if I can rearrange/swap the ESTOP and Thermistor connectors, as this should give a better ground connection. Making the board wider than 100mm may also help, as the ground plane can then run the edge of the board next to Aux-4, providing another path. That does increase the price of the board a bit at some places (100mm x 100mm seems to be a sweet spot for some board fab companies). As it is, we've increased the board size to 95mm x 100mm, which improves the ground in that corner a lot.

Can you try putting a wire from the Thermistor ground to the ground on I2C-0 and measuring the voltage drop? This should be reasonably equivalent to the changes I've made so far in that area.

As it is, I've just improved the ground a lot, so that may help, but IMO we should be able to do better.

See Github Bug #23 for RAMPS-FD for status/notes/details.

Edited 1 time(s). Last edit at 12/22/2013 01:51PM by Cefiar.

Giving the thermistors a dedicated ground and returning it via a dedicated pin to the Due probably is the only real way to get good resolution on the thermistors. A work around would be to send the local thermistor ground back to the Due as a signal, read it, and do the math. That (of course) only works if it's a positive voltage ....

The converse of that is indeed to have a solid ground on the FD, so you don't put a lot of offset onto the Due ground.

@Cefiar

I can't try the wire connection at the moment since I'm lacking a soldering iron, but I don't think it will do much. Let me emphasize that the parasitic voltage only appears when heaters are active, and in particular I believe it's due to a fraction of the current coming out of mosfet S terminals taking an unintended route through the ground traces. I've done some more measurements with the following results:

- Turning on the bed heating will cause significant current to flow across the two GND pins next to Vin, in the direction from the RAMPS-FD *into* the Due. I measure 5mV across the header connection.
- Similarly, current will also flow into the Due on the GND pin next to AREF (got 5mV here too).
- There is no measurable voltage/current across the two GND pins next to pins 52 and 53, as well as on the SPI GND pin.

That left a mystery, namely, where is this current going out of the Due. Having exhausted all other explanations, the truth must then be that it's not coming out. At least not back into the RAMPS-FD.

And, behold, disconnected the USB and the mysterious currents are gone. The thermistor-ground-to-due-GND voltage is also gone.

Just to be sure I'm not interpreting the results wrong, I tried just touching the USB plug and the socket, without pushing it in. And things went crazy again, currents and voltages all over the grounds.

EDIT: On my PSU, I connect DC negative to mains earth, as per instructions for my printer (RepRapPro). Presumably on the PC, USB ground (shield) also ends up on mains earth. Therefore the USB ground is just one possible path from the board to the PSU's negative terminal, and some current over the USB is actually expected. I just have no idea these problems are properly solved.

Edited 5 time(s). Last edit at 12/22/2013 08:13PM by ambrop7.

Here's a simple way to reduce these sorts of issues:

Your power supply needs a ground reference. Running it without a ground is not safe. (note to self ... ground the supply on the printer ...do as I say not as I do). A hard ground is the best thing to do. As you have found that can create issues. A compromise is to tie the supply to ground through a resistor. Something like 1K ohms will take care of random leakage issues and ESD. You won't be as safe as with a hard ground. I'd bet your strange USB issues go away. If not check the USB ground with the voltmeter. Your computer power supply may have grounding issues. You might also have a circuit ground (wall plug) problem, I've certainly seen a lot of them over the years.

USB ground loops like this explain all the issues, but this also adds to the fact that the ground connections were way too thin in many areas. That said, eliminating the ground loop issues is not really the job of the shield, as they're an issue with your PSU/PC/Due config.

Just submitted to bobc a major overhaul that provides a lot of thicker ground traces (with multiple redundant paths), plus extra thickness to almost all the Ground, +V_LOGIC and +5V traces (there are a few exceptions that I'm hoping to eliminate at some stage soon).

My PC and printer PSU are plugged into the same extension cord, they certainly both have a ground connection. And this is actually the issue, because the resistance of the ground connections is low enough to have a significant fraction of the heater current coming back into the PSU negative terminal. And you can't really prevent that, it's just Ohm's law. Better ground (lower resistance) will make my problems worse, as more of the returning current will take the ground path instead of the direct path back to the PSU.



I'll try the resistor, thanks for the suggestion.

This is a difficult one. Clearly, earth bonding of equipment is a useful safety feature (it's a mandatory requirement in safety standards). Unfortunately it also creates problems in other ways, so simply connecting 0V to ground may not be the best way to do it.

USB was designed for peripherals like mice and keyboards, it wasn't designed as a general purpose comms connection, although that is what it is now used for. What to do with 0V and shield on the USB connector does not appear to be specified in the USB standards, so some people leave the shield unconnected, a lot of people tie it to their 0V, as the Due designers did. Other people connect USB shield to 0V via some protection components, which seems to be best practice.

You can get USB isolators, which are magnetically coupled. I have found these may not always work reliably. Really we would want to use a comms connection with designed-in isolation like ethernet.

We have heard of many USB problems people have when things go wrong on these forums, e.g. with systems hanging, hot USB cables, components burning on the Arduino etc. When designing a shield though, most of these issues are out of our hands, since we can't dictate the design of the Arduino or how people wire up their systems. We can do whatever we can to alleviate the problem, but ultimately I think all we can do is to recommend some best practice.

Personally I use isolated PSUs and don't connect anything to ground, so I wouldn't see these type of problems.

I did wonder if we should try to improve the analog performance, e.g. by using a proper voltage reference, but the analog signals pick up so much noise it didn't seem worth it. Since we are only measuring temperature, which is relatively slow to change, just adding a lot of filtering reduces that problem. For some reason 4k7 is used in the voltage divider, but that is not the best value as it leads to a high slope in the temperature range of interest. It would be much better to have values suited to extruder temperatures, and a different value for heatbed.

---

What is Open Source?
What is Open Source Hardware?
Open Source in a nutshell: the Four Freedoms
CC BY-NC is not an Open Source license

If you do make a change to that 4.7K resistor - *please* go to a value that's only available in 1% resistors. Using a part that one can get in 20% is just asking for trouble. Yes it's the standard and it's way to late to change a standard. These days, the cost of 1% parts is a fraction of a cent. Having a 1% thermistor that costs $3 and a 20% resistor that saved you $0.002 just isn't right. Yes most parts are 1 or 2% no matter what you spec. Every once and a while you loose that bet.

--------------------

If you do use an isolated supply (which is what I do), put a bleed resistor to ground. There's always the strange case of leakage from this or that. You *could* get the ground up to something really silly. Yes I need to do that on the power supply sitting by my feet .... I should read my own posts ...

I've been looking at FET switching performance on the current board. I am switching the D8 output with a 50% duty cycle and a 1kHz frequency. My heatbed is 5 aluminium clad resistors, 6.8 ohm in parallel. Nominal current draw at 12V is 8.8A, but I haven't measured it. That's a bit less than a Mk2 PCB heatbed. The MOSFET has no heatsink, and gets slightly warm.

I'm measuring the voltage at the gate and drain of Q201, D8 mosfet. Ch1 is the gate (orange), and Ch2 is the drain (blue).

Here are some scope traces:






The turn on time seem to take about 12us, which doesn't seem too bad.
Switch off is fast of course due to the small FET, but there is a big spike and ringing on the switch off, which might be expected for wire wound resistors.

The results seem better than I was expecting, although I am not entirely sure I am measuring this the right way.

---

What is Open Source?
What is Open Source Hardware?
Open Source in a nutshell: the Four Freedoms
CC BY-NC is not an Open Source license

Changing the resistor on the drive FET to something smaller would most likely improve the switch-on time, though since we've changed the design currently to running direct off the buffers, that seems a moot point.

Would be interesting to see how this changes with the new circuit we've got using the 74HCT/ACT series buffers. If it turns out the performance is very poor, then perhaps we could investigate the TC4426A/TC4427A chips mentioned earlier in the thread.

Quote
Cefiar
Changing the resistor on the drive FET to something smaller would most likely improve the switch-on time, though since we've changed the design currently to running direct off the buffers, that seems a moot point.

Would be interesting to see how this changes with the new circuit we've got using the 74HCT/ACT series buffers. If it turns out the performance is very poor, then perhaps we could investigate the TC4426A/TC4427A chips mentioned earlier in the thread.

I hacked up a circuit on a spare board, using a 74HC244 running at 5V driving the IRLB8743 directly, with a square wave generated by an Arduino Mega.





Switch on and switch off are now around 300-400 ns.

Edited 1 time(s). Last edit at 12/24/2013 10:39PM by bobc.

---

What is Open Source?
What is Open Source Hardware?
Open Source in a nutshell: the Four Freedoms
CC BY-NC is not an Open Source license

As mentioned earlier ... you probably don't want to get the FET's switching a whole lot faster than half a microsecond or so. There will not be any real gain in power dissipation. The ringing issues will keep power sloshing around for quite a while with fast edges. I don't think we want to get into high current ferrite beads on the leads.

Quote
uncle_bob
As mentioned earlier ... you probably don't want to get the FET's switching a whole lot faster than half a microsecond or so. There will not be any real gain in power dissipation. The ringing issues will keep power sloshing around for quite a while with fast edges. I don't think we want to get into high current ferrite beads on the leads.

Might be useful to scope all the supplies (12V, 5V, 3.3V) to see how much they drop when you turn on FETs. I did it when I was debugging my ADC issue, and IIRC, 3.3V dropped to about 2.8V when I turned on an extruder heater, taking about 1us. It must be even bigger for bed switching. Presumably this is due to the disturbance on the supply traveling down the regulators.

Edited 2 time(s). Last edit at 12/25/2013 05:18AM by ambrop7.

I hate to say this, but at some point in circuit density, a multi layer board is indeed the way to go. Stuffing an unbroken ground plane under everything helps a number of issues. It's not going to improve the cost side of things any. Everything I do these days at work is multi layer....

The ARM's operate at higher speeds internally than most of the older MCU designs. That means faster edges coming out. Faster edges always will mean more "stuff" crawling all over everything. Bypass caps and decoupling resistors tend to multiply because of this. I'm not real sure that the Due boys have come completely to grips with all of this. That puts an even bigger burden on a shield board like this one.

Quote
uncle_bob
I hate to say this, but at some point in circuit density, a multi layer board is indeed the way to go.

+1, although I don't think it's necessary to go to a multi-layer board for the RAMPS just yet. The main things to control are edge rates and current loops. Note the ringing in the traces above with the 1-2 uS turn-off time and the much worse ringing with the well under 1 uS turn-off. Faster is NOT always better! Exactly the same thing happens with the signal traces having high edge rates. If signal edge rate is a problem on the RAMPS, a few well placed series termination resistors will probably fix things up (nothing on the shield needs to have nS scale timing accuracy anyway). For current loops, there are the obvious high-current paths with the heater FETs, and also the "phantom" high-speed current return paths for the signal traces. It should be possible to arrange connectors such that the heater currents don't loop through the Due grounds but instead flow out the main power connector. A ground plane would help most with the "phantom" return paths for the high-speed signals, but IMHO series termination should generally reduce the current spikes enough that a more "meandering" ground return on a 2-layer board would work OK and avoid serious cross-talk to the ADC lines.

The problem with the decoupling resistors is that they really *need* to be at the source end of the lines. That would put them on the Due board by the ARM. They have already run a long way before they get to the Ramps. You have "stuff" all over the place as a result of the fast rise / fast fall edges.

FWIW: We've improved the size of the existing ground plane (the back), and provided a number of thick top level paths to alleviate thin bottom/layer connections in various places. I've also added a few specific via's to provide further connection between the top and bottom level areas (eg: around filter caps under the Pololu's, etc).

I also re-routed a lot of signals and moved certain components (specifically U1, the buffer driving the FETs, which was right next to the thermistor circuitry), which should hopefully decrease any induced noise in the thermistor area, and also reduce any trace inductance between the buffer and the FETs (which are now MUCH shorter).

If you're not actually looking at the KiCad project and basing all your comments on the old board, then please actually download a copy of KiCad (you need Git 4022 or later) and have a look. We've still got some further stuff to do, like we've still not added in the thermistor protection circuitry, some connectors, components and text needs updating, etc. But it's a big change from the last board.

Once we get those issues down, then (and only then) will we look at doing another run of boards.

PS: If you're wondering, the most annoying thing has been trying to keep the whole board within 100mm x 100mm (it's a cost sweet spot with many manufacturers). The board is currently 95mm x 100mm, and it's the width that is at 100mm that is by far the most annoying dimension of the whole project. The next biggest issue (which affects the first) is the Arduino header layout.

Quote
uncle_bob
The problem with the decoupling resistors is that they really *need* to be at the source end of the lines. That would put them on the Due board by the ARM. They have already run a long way before they get to the Ramps. You have "stuff" all over the place as a result of the fast rise / fast fall edges.

Also, with the layout we have (due to the Arduino header), that area is already very full, so getting components into that area would be difficult and require a lot of rearrangement.

Some of it I have already been considering, but unless it's really needed, I'd like to avoid it. for an idea on the level of work, it'd probably be a lot easier to completely wipe the board, move all the components about and lay it all out again.

Edited 1 time(s). Last edit at 12/26/2013 06:25PM by Cefiar.

Quote
Cefiar
If you're not actually looking at the KiCad project and basing all your comments on the old board, then please actually download a copy of KiCad (you need Git 4022 or later) and have a look.

My comments are mostly based on general design, and not anything specific to the current in-progress or previous version(s) of the board layout. I've been working with mixed-signal design for about 25 years, and my gut feel is you don't need series damping resistors on the signal lines for this design. I'd still keep the FET gate resistors and try to limit the FET turn-off time to the uS scale, but those are totally different issues. As long as you're not seeing multiple transitions on the output of the '244/'125 drivers due to ground bounce or similar transient effects, I wouldn't worry about series termination.

I do have KiCAD available and intend to port the RAMPS-FD to a BeagleBone Cape when the design stabilizes a bit, so just holler if you want a review of the layout/routing. As I said before, the main thing is to keep the current loops small, especially the high-current loops with the heater FETs.

The problem with the edges is a bit strange. The clock rates aren't very fast, but the semiconductor process used for the ARM is indeed quite fast. The result is that the edge rates coming out of the MCU are a lot faster than you would guess they would be. Spray is all edge rate related, so indeed you can get "stuff" all over the place with an ARM. For short traces - not an issue. Longer traces over a good ground plane - not an issue. Traces all over the place and no solid ground plane ... you will see noise here and there. The objective is to get the thermistors at least to the 12 bit level and (hopefully) get them better than that. Since there are ARM products on the market with 13 to 14 bit ENOB A//D's you would like to be compatible with them as well. You never know when Atmel might play catch up ....

We don't need 12 bit resolution on thermistors though. We can also afford to have a lot of filtering on the inputs, because temperature varies relatively slowly.

---

What is Open Source?
What is Open Source Hardware?
Open Source in a nutshell: the Four Freedoms
CC BY-NC is not an Open Source license

I've been looking at the thermistor inputs, in particular the fixed resistor in the voltage divider. I notice that the Duet hardware uses a 1k resistor here for both heat bed and extruder, not sure how this was chosen. 4.7k is ok for a heat bed with best response around 100 deg. According to my calculations 220R is about the best value for the range 160 - 300 degrees suitable for extruders. The E96 (1%) value is 221R.

Therefore I suggest we specify 4.7k for the heatbed and 221R for the extruders.

I'd also like to update the schematic with the new thermistor protection circuit, do we have a good idea what this should be?

---

What is Open Source?
What is Open Source Hardware?
Open Source in a nutshell: the Four Freedoms
CC BY-NC is not an Open Source license

Quote
bobc
We don't need 12 bit resolution on thermistors though. We can also afford to have a lot of filtering on the inputs, because temperature varies relatively slowly.

Well, we do and we don't. If you want to set up to print at 150C one day and 300C the next day having a couple more bits would be quite handy with the standard choices of 1K or 4.7K ohms. The temperature resolution gets pretty poor at the high end with a 4.7K resistor. Indeed a lower resistor would help this quite a bit. A much lower value would make the resolution around room temp a bit poor. You might start getting mintemp errors as a result. Having reasonable resolution at room is also a good reality check when first setting up the printer.

With all metal hot ends coming along and some of the more exotic filaments, 300C printing is indeed a very real possibility. Tables probably should go to 350C for safety's sake if you are running that hot. Who knows how happy the thermistors will really be up there.....

The problem is that the extra ADC resolution only helps if you also improve the electronics a lot. The analog performance on the printer electronics I have looked at is basically terrible. If you have 50mV of noise, you end up with +/- 60 counts at 12 bits instead of +/- 15 counts at 10 bits, but no better accuracy. You would need a properly laid out and calibrated analog section to get down into the mV scale where the extra bits would be useful.

It's a great idea to have a useful range 25 to 350 but I think that is just not really realistic to achieve with what we have got to work with.

There is really a limit what you can do with an NTC thermistor and a generic base board like Arduino which is simply not designed for high analog performance.

---

What is Open Source?
What is Open Source Hardware?
Open Source in a nutshell: the Four Freedoms
CC BY-NC is not an Open Source license

Which brings us back to the original departure point of reduction of noise on the Due board....

Is there a (practical) way to give the thermistor "empire" it's own dedicated ground going back to the Due? Have the thermistors floating unless the Due is plugged in. What ever performance you get that way, it's going to be the best you can get.

These MCU ADC's are not real quiet on a single sample basis. A sample rate of 1 mega sample and a box car decimator of 32:1 is not at all unusual. You still have a very respectable sample rate for control stuff after you do that. If you have white noise the down sample process gives you more bits. If you have something weird (like impulse noise) it may not be as effective.

Quote
bobc
I've been looking at the thermistor inputs, in particular the fixed resistor in the voltage divider. I notice that the Duet hardware uses a 1k resistor here for both heat bed and extruder, not sure how this was chosen. 4.7k is ok for a heat bed with best response around 100 deg. According to my calculations 220R is about the best value for the range 160 - 300 degrees suitable for extruders. The E96 (1%) value is 221R.

Therefore I suggest we specify 4.7k for the heatbed and 221R for the extruders.

I'd also like to update the schematic with the new thermistor protection circuit, do we have a good idea what this should be?

The reason 1K was chosen is that it gives a reasonable response around the critical area, but doesn't put too much current through the thermistor. They all have max power ratings, and putting too much current through them will kill them.

We could go lower, but this means that you also need to generate all your own thermistor tables. A lot of firmwares have added in thermistor tables generated for 1K resistor dividers, which means they're drop-in usable.

Regarding the circuitry for thermistor protection, we probably want to use the circuit used by the latest RAMBo board (which came out of our own discussions). Note that they don't have ANY filtering cap in the 10uF range, so we can either add that in, or possibly even make it smaller (say 3.3uF)? If we find it's not necessary or we can go smaller, we can simply not populate the footprint and remove it on the next revision.

Note: Use a zener for the voltage protection rail, don't tie it to the 3.3V rail of the Due. This provides a nice fixed cap that isn't related to the Due at all that could cause problems if say the 3.3V supply on the Due fails (eg: VIN stops powering the Due, then suddenly the Due gets >3.3V on the Analog inputs, frying them). The extra cost for a resistor and zener is minimal. We can probably run the zener circuit off the same rail that runs VIN (which on 24V should be regulated down to 12V).

Edited 1 time(s). Last edit at 12/28/2013 09:38PM by Cefiar.