Help wanted: PIC programmer

Feel like helping to develop a programmer module for the RepRap? It's a small and simple project that would be a good introduction to the architecture.

This could involve electronics, firmware (C) or the host application development (Java), or any combination thereof.

Just leave a message here or send me a private message.

Well I guess here might be as good a place to leave a message about what I've been doing with this idea that Simon got me interested in. Here's my status to date.

1. Investigated how pics such as 16f628A/48A are programmed. The nitty is real gritty but the basics are you send commands clocked bit at a time to the data and clock leads. To get the chip in program/verify mode you first set MCLR, VDD, Clock and Data all to ground potential. Then raise MCLR to +12 followed by vdd rising to +5. Then the clock and data lines can send bits lsb first representing commands followed potentially by data the commands might need.

2. I investigated how the stepper motor firmware worked and started throwing code in there to use routines similar to Adrian's that was recently added to flash an led for the new universal board you all are trying to build now. After getting the basic commands written, I realized what I really wanted to do was integrate my code into the IOBOX that isn't being used for much. A vastly better idea. Well I moved my code over and pondered how stuff was supposed to work and then finally started trying to compile what I had, leaving much of it stubbed out except for a couple basic commands. There is one that turns 4 leds on and another that turns them off.

3. I build the hardware, programmed the chip and lo and behold I lit 2 of the LEDS. I re-examined my wiring and found I had brought over 2 +5 volt lines to those LEDS and connected them both correctly, forgetting entirely about the ground. Well correcting that, the 4 LEDS then came on and then went out with my complementary command. And to paraphrase our friend Forrest, with lights going on and off, how far off can a Pic programmer be?

When the 6 or 7 basic read/write/verify/erase commands are complete, there remains the higher level interface through java to open a file and stream small chunks down to the programmer or back up as the case might be.

Well I've given my status, time for bed since it's late. Have to start using all the rest of that code I wrote to stream bits across those LEDS tomorrow.

My question to you then would be this: what sort of programming do you want?

When you boil it all down, you are going to be required to have either a preprogrammed controller, or unnecessary hardware bit-banged by the PC.

Either you have this hardware to directly program from start to finish the entire PIC program, or you can have a preprogrammed PIC with a bootloader kernel that will check for commands from the PC and if the PC commands it to accept program code, it will ru nthat routine and save that to progam memory.

Before you get down into the grittiness of firmware development, you absolutely must figure out what you want to do.
====================================================

Here is my idea:

The kernel is programmed in C or asm through a compiling script which will take in arguements for which controller it is reprogramming and write some of that information to the hex file and call it whatever.

Bobby wants to update his kernel code so he loads the latest kernel through whatever program we may choose to use (either a module of the curent software must be made or a new program entirely) and it parses the file taking in information about what hex code is going where.

It sends the command to the PIC to start its reprogramming routine which will receive the hex code and save it to program memory and pass it down the daisy chain if necessary until all PIC needing reprogramming are updated.

========================================================================

This requires that a parser and module/program be made in Java for this. I am not a Java programmer (hate the language with a passion to be completely honest), and a bootloader kernel that can parse basic USART I/O and program memory, which will be a snap.


I can program kernel stuff, but I dont have access to hardware to make sure it works. I am also not good at programming Java, so that will be up to someone else.

I cant seem to find the source for the PICs, so I cant speculate on the API. In fact, I cant really find source for the PIC code or the Java stuff. Maybe I'm blind, but I will need it.

PS: What sort of C compiler are you using?

Edited 1 time(s). Last edit at 06/05/2007 05:47PM by AP2k.

all the source for everything on reprap is in the subversion repository on sourceforge... do a bit of googling, you'll find it.

AP2k Wrote:
-------------------------------------------------------
> When you boil it all down, you are going to be
> required to have either a preprogrammed
> controller, or unnecessary hardware bit-banged by
> the PC.

You can't bit-bang over the SNAP network :-) So your PIC Programmer will need a little intelligence in it -- a PIC can provide that.

The concept (as I understand it) is for a new kind of module to add to the SNAP network (which could be the first and only module in your network to start with!), that accepts a set of SNAP commands from a (probably Java for portability) host program and reads PICs and programs raw 16F628/16F648 PICs. Then you can use those programmed PICs to stick into your Stepper Controller and Extruder modules.

There is a "catch-22" about how to create the PIC Programmer module if it has a PIC in it that stays in it and does the programming of other PICs... how to get that first once programmed :-) I suspect the answer there is to buy that one initial PIC from the RRRF store, pre-programmed, or ask a friend who is further along in the Reprap construction process to mail you one.

> I can program kernel stuff, but I dont have access
> to hardware to make sure it works. I am also not
> good at programming Java, so that will be up to
> someone else.

You could in necessary write an initial version of the host-side software in any freely available and portable language. Python or Perl or Ruby. Someone else could then recode that in Java later. All you need for hardware would be a PowerComm board ... building the PIC Programmer board would be part of the project itself, of course.

> I cant seem to find the source for the PICs, so I
> cant speculate on the API. In fact, I cant really
> find source for the PIC code or the Java stuff.

Please, please, read the Reprap Wiki [reprap.org] . See [reprap.org] for how to install the full Reprap software development environment, including the firmware development environment (SDCC and its friends gputils, basically).

I have created a "do it all" shell script to install the entire software dev environment once you have a working JDK on your Linux box (it might work on OS X or even under Windows/Cygwin, or it might not), but it is very untested and not really ready for general use yet. So for now, please follow the instructions on the RepRapLinuxSoftware page.

Failing that, just browse [reprap.svn.sourceforge.net] .

> PS: What sort of C compiler are you using?

SDCC. The firmware/Makefile grabs SDCC sources from CVS and builds it for you (and then compiles the Reprap firmware sources into .hex files, ready to program into your 16F628s).

Jonathan

Jon, you've pretty much got it. The first one is again a chicken and egg problem. Except I built a JDM programmer and got it working and then programmed my own pics. The idea is to provide a system that does the programming without having to unplug your serial port from your reprap ( you say your new computer doesn't have a serial port? ahh.. ) anyays I started on some code, got some commands working that blink lights in what I think should be what is necessary to program a pic which is hooked up to the one I talk to. So its a pic programmer running in a pic. I have been thinking about moving up to a pic18, but that means writing the comm program for it and getting it to work. But if that could be done, the neat thing is as was mentioned, you program the pic once and then you flash load via the usb port. I have some 18f2550s that I got as samples, waiting for 20Mhz crystals to come in. Being able to flash load your target would sure speed up the development cycle if you know what you are doing. Ahh.. well its a path and I'm enjoying the journey. And every time I get a spare power supply, seems the one in my computer craps out and I don't have a spare again. Ce' la vie..

Hi guys,
I'm new to this project -- been following the project for a while, but I just bit the bullet and decided to try making one last week. So far I've got two boards made. Anyway, I've been thinking about the PIC bootstraping problem too.

Best I can figure it, the only viable option for programming everything from scratch is to build a JDM-like programmer into the power/comms board. I've had mixed success with JDM programmers, so I'm not thrilled with that option. To be fair, I suspect most of the problems with those programmers are because it's trying to generate Vpp from 5V. Since we've got 12V on board already, we could probably find or make an equivalent circuit that does away with the charge pumps and is much more reliable. The nice thing about this approach is there's tons of software to program with JDM programmers, so we wouldn't need to write our own.

Every other solution I can think of has the chicken & egg problem of needing a programmed PIC to do the programming, but that's probably not so bad.

An idea I've been toying with along those lines is switching the power/commm board to USB by putting an 18F2450 on it. Add a socket & transistor and I think you've got the makings of a programmer. Going from RS-232 to USB increases the cost by about $3, the complexity of the board is about the same. You could do the same by tossing another 16F628 on the power/comm board if you want to stick with RS-232 at about the same cost.

Another way of dealing with this problem is to just make it less relevant. If every chip had a serial bootloader on it, most builders wouldn't need a programmer. They'd just order five PICs with bootloaders from RRRF. Anyone who can make the rest of the hardware can make a basic PIC programmer if they need it. Maybe it's not so bad to leave the programmer as a separate tool.
(regardless, I think all the firmware should be upgradable in-place)

I like the "JDM-on-PowerComms" idea, if it can be made to work:

(1) Low cost but (probably) effective -- very much in line with Reprap overall approach in that regard :-).

(2) No pre-programmed bootstrap chip requirement (which might eventually limit exponential reproduction).

(3) No serial lead swapping needed for those with only one serial port (BTW, some of us still have PCI multiport serial boards, left over from days gone by when dialup external modems were used for your primary Internet connection -- but that's probably not very common!).

So... is anyone out there with appropriate electronics skills willing and able to attempt a "PowerCommsJDM" circuit design??

I also think the idea of adding ICSP headers on the UCB, so we can program the PICS in-circuit, would be rather nice to have. Was the current UCB circuit designed with ICSP in mind? If so, the extra cost might just be board layout and a header??! You'd probably also need a way to kill power to the UCB you're about to reprogram, though -- power switches on each UCB maybe??

I'm thinking (aloud and in public!) in an area in which I have very little real knowledge or experience here... hopefully it is useful :-)

Jonathan (who knows how to program a PIC, but not how to design a PIC Programmer)

This is the in-circuit programming specification for PICs: [ww1.microchip.com]

I think that Jonathan's idea for an onboard programmer is a good idea. I think that having a simple socket that fits the PIC and bit-banging the commands over. Of course this must be a separate socket since Vpp is a weird voltage (12V IIRC) and will need its own control logic to program each PIC, and programming will interfere with the normal control logic output of the PIC, Not a big deal, but the boards will need a different layout (I know, that sucks) if this thing is going to be done in an automated fashion.

You still are required to remove each PIC and program it separately, though. For a one-time programming of each PIC while building it, I dont see an issue, but it can get rather bothersome to have to program a PIC individually every time a firmware update comes out, at least to me.

I also forgot completely about the SVN repo. A proof of concept program written in C I can do. Thanks for that heads-up.

I always get a headache when I try to understand what's going on in a JDM programmer. I keep forgetting how much magic goes on in them. On second thought, a COM84 programmer looks like a simpler design to emulate. I'm not sure if it's compatible with JDM software, but it looks like it's pretty close, at least.

Looks like the UCB is close to being ICSP-able. The PGC/PGD lines aren't used (brought out to the expansion header, even). The only potential problem I see is there's no diode on MCLR, so while programming you'd be feeding the high programming voltage into the output of the 7805 through 10k. I'm not sure if that would do any damage or not. It might be able to take it for the few seconds it takes to program a chip. It wouldn't take much to make the expansion header double as an ICSP header, just add a pin for MCLR and maybe shuffle the pins around.

Looking over the serial code from the SVN, I would simply recommend that the kernel module be put at the very end of program memory for each PIC, along with its support EEPROM writing code. That way its out of the way of everything that will need to be written. The routine would consist of basic EEPROM I/O and USART I/O. After successful programming, the device would obviously need to be power cycled.

Here is my preliminary thinking:

PC sends the first PIC some command to tell it that it is ready to start programming. Main PIC ACKs the PC and tells the rest of the PICs that its time to reprogram. All PICs will proceed to the reprogramming functions.

The PC will then send some command packet to the first PIC which will contain the addresses of the PICs needing reprogramming. Each PIC will then write a GOTO $xxxx with the address of its function to the first word space in the program memory so if something REALLY bad happens, you simply power cycle and resend the program.

So, the program words will go through the chain until each PIC needing programming will get it. After its all done, the old program word at 0x0001 will be added back in.
========================================================================

On the programming side of it, the address of the kernel reprogramming routine can be hardcoded at compile time since it should never get rewritten after its first write.

Edited 2 time(s). Last edit at 06/06/2007 01:16PM by AP2k.

cool!

i can follow whats going on, but i have no advice, other than that i like the idea of in circuit programming, intergrating powercomms and the programmer, and making this easier in general.

if you're really interested in it, the designs for the powercomms and universal boards are in subversion, keep in mind the most current ones are under 'electronics/production'.

the first thing would be to design it and then make a home etched board. if it works, its not much of a stretch to imagine us doing a test PCB run of the board design, and shipping them out to people to test.

then of course theres the software side of things. also, keep in mind that alot of people are going to be using a usb => serial converter cable, so it has to be compatible with that.

rad!

emf Wrote:
-------------------------------------------------------
> On second thought, a COM84 programmer looks like a
> simpler design to emulate. I'm not sure if it's
> compatible with JDM software, but it looks like
> it's pretty close, at least.

I'm pretty sure a JDM *is* in essence a COM84 that has been enhanced to use the charge pump weirdness, so you can have a higher programming voltage and so program later chips than the 16F84 (when the magic actually works!). The control lines used appear to be identical. This means the two should be PIC-programming-software compatible.

> Looks like the UCB is close to being ICSP-able. ...

Nice. I think, regardless of exactly how we go about creating a programmer, getting ICSP capability into a revised UCB layout is very much worth having. A diode to protect the 78L05 from the programming voltage would (I think!) be inexpensive to add -- or would that have negative effects on the circuit in normal operation? This would be a worthwhile enhancement for people with existing semi-expensive commercial PIC programmers, too -- no more lifting and replacing PICs to reprogram them :-)

Jonathan

That self-programming stuff sounds good. Sounds pretty similar to what I was going to try. Since I've never done any self-programming stuff myself, I was going to try to leverage a bootloader someone else has written (I like TinyBld, but I know we'd need something with an open source programmer). I figured we could install the bootloader on all the chips, then add a SNAP command like "if your ID is [x axis stepper], reset yourself, everyone else just act as a dumb repeater until you get some magic cookie (or timeout)". So you send this command, then run the bootloader's code on the host as if only one device were connected, then send the cookie to wake all the other devices up and repeat for the other PICs.

Your approach sounds a lot more elegant. My only fear with more complicated schemes is that we shouldn't (easily) be able to put buggy firmware on it that would make it unresponsive, but I'm sure it can be designed prevent that kind of thing.

That is an interesting point.... and not one that I can answer easilly. I suppose I can add in a checking for its address in the bootloader and a repeater function.

My proposed bootloader will be a fully matured function and will not be able to be automatically rewritten after its first written by a programmer. Therefore, it will always have a specific address at the end of the memory. Thus, if something really bad happens like botched firmware, a button can be pushed when its power cycled which will bring a pin high on all PICs. The PIC will poll this at the beginning of the main function and if its high, go straight to the bootloader space. Its rather difficulkt to botch a simple pin poll, lol.

Botched firmware sounds like a really scary thing. Although I dont suppose it need be if the host program can verify data integrity as developers should never be exclusively using this bootloader method anyway. Thus it works for them, and it will work for Bobby just the same. Bobby has no need for worry because the firmware is known to work,

An MD5 sum of the program .hex at the host's end from the known good .hex should suffice for the trip between developers and hosts.

Edited 1 time(s). Last edit at 06/06/2007 03:18PM by AP2k.

A comparison, based on my understanding to this point:

COM84-style on PowerComms:

+ Simple, relatively easily understandable.
+ Uses existing PIC programming software such as picprog or whatever.
+ No "first PIC catch 22", can bootstrap using blank PICs.
- Needs a real serial port, because we'd be bit-banging the serial lines.

Self-programming over SNAP on every UCB:

+ Works with USB-to-serial converters. This is a big one.
- Relatively complex design.
- Needs Reprap-specific PIC programming software on PC host.
- Has the "catch-22" bootstrap issue.

ICSP Support on UCBs: Is independent of the PIC programming approaches above, but is more useful with the COM84-on-PowerComms approach, or for people using other commercial programmer hardware, than for use with the self-programming model.

Jonathan

jmarsden Wrote:
-------------------------------------------------------
> The control
> lines used appear to be identical. This means the
> two should be PIC-programming-software
> compatible.

Good. I noticed WinPic treated the two differently, but I don't know why. My guess is it just alters the timing for the JDM to allow the capacitors to charge.

> Nice. I think, regardless of exactly how we go
> about creating a programmer, getting ICSP
> capability into a revised UCB layout is very much
> worth having.

Agreed. Those sockets just asking for bent IC leads. Unless reprap can build IC extractors...

> A diode to protect the 78L05 from
> the programming voltage would (I think!) be
> inexpensive to add -- or would that have negative
> effects on the circuit in normal operation?

I don't think there's any real downside. It drops MCLR's voltage by a diode drop, but that's still safely above Vih. Without the diode, I think we're also driving the L298's Vss too high.

I've still got three boards to build, I'll toss an ICSP header on the next one (if I can figure out Kicad). Is there any reason the 1.2 board can't be manufactured at home as-is? Does it need plated vias under some components?

you could definitely make the pcbs at home. the gerbers are there for printing, and i'm working on getting pdf creation automated to make it even easier.

it just turned out that it was easier and more economical to have them commercially manufactured. it really lowers the barrier to entry alot.

how about this... we do both!

the obvious move is to modify the powercomms board to be a normal programmer. if its an easy fix, then theres no reason not to do it. an icsp header is also a great idea. it greatly reduces the chance of breaking a chip leg.

the self programming idea is probably the best one long term. it can also be developed in parallel. since you already have the manual programmer, there are no worries about mangled firmware. if you mess up, you take the chip out, reprogram it and try again!

my guess is these modifications are something that wont 'officially' be supported until v2.0 (mendel), simply because the designs we have now work, and are the 'frozen' boards for darwin. having said that, i encourage anyone interested to work on these. modify the v1.0 powercomms board, make a new layout, and see if it works! if it does, then thats one less thing we'll have to do in the future.

Zach,

Yes. Doing both is very much fine, if we have the manpower available to do so. I was reluctant to suggest it since I'm not really the right kind of manpower for that work, myself.

> because the designs we have now work

Well, if you don't mind drilling out the "not quite holes" into mounting holes, and inserting a couple of connectors backwards from the silkscreen info :-)

They work, sure. But that's probably not the same as their being well tested, much less their having no known significant defects. Is there really no way to officially support intermediate releases of some subsystems between v1.0 and v2.0? The whole "evolutionary" nature of the project surely suggests that there could usefully be incrementally improved versions coming out relatively frequently, doesn't it?

There is already a Darwin v1.1 listed as a "future plan" on the Wiki at [www.reprap.org] . Is there a reason why a set of improved and tested boards could not be officially released as part of, for example, a Darwin v1.2 release? If the only changes were (for example) adding ICSP headers, a diode, and the PowerComms integrated programmer, there should be no firmware changes required, and no host software changes needed either, so the additional support burden appears limited to the actual electronics themselves. Not too bad, IMO.

Forcing an improved subsystem to wait until all of the Reprap subsystems reach v2.0 status before any of them can be released ... just seems unnecessarily restrictive to me.

Jonathan

Doing both sounds reasonable. I think tweaking the universal board for an ICSP header is probably a small enough change that it could be worked into the next run of boards. It's useful for almost anyone with an external PIC programmer, and if you don't plan to use it you can just put a jumper wire in the diode's place.

The power board changes are more drastic. I'm pretty sure you'd still be able to omit the programmer-specific parts (most people wouldn't need it if we already had serial programming) and have something identical to the current board, just a little larger. Still, it's not a trivial change.

If it's not going to be part of the 1.x series, it sounds like this work would never see the light of day. Whatever we do to make an RS-232 programmer doesn't seem like it would have much relevance on a USB system (unless Mendel is going to have both?)

BTW, It looks like there are a few bits of this board that would be challenging to make by hand -- top-only traces under jumpers, LEDs. Most look like they'll be pretty easy to relocate.

okay, you got me!

what i was really trying to get across is that depending on the severity of the change, its something we may want to hold off on making it the 'official' version for a bit. it most certainly is going to see the light of day!!! i was not trying to imply that at all. infact, we'll absolutely need these things for mendel. i'm actually wondering if we even need to switch it directly to USB. with a usb -> serial converter it works great, and if we get these fixes integrated, then we can still use the db9 connector and do direct bit-banging programming. (but thats a discussion for later...)

one of the things that sucks about making component changes to the board is that all the pictures on the documentation need to be redone. i guess this wont be a big deal once we have them all up on the docs. then it becomes a matter of replicating the shots already taken and live on the site.

plus, it will probably take anywhere from a month or 2 to take this from a schematic to prototypes to production, so i guess thats enough time for the current version of the boards to live.

heres what it boils down to: i'd love to see these get made (all the way to production), and it would make sense to have these be v1.3 boards, not v2.0 boards (you're right!)

any objections/ideas for this plan?

ZachHoeken Wrote:
> what i was really trying to get across is that
> depending on the severity of the change, its
> something we may want to hold off on making it the
> 'official' version for a bit.

Agreed. What I'm thinking of is essentially a new (but compatible) programmer design, and it'll need a fair amount of testing. It looks like self-replication is still a ways down the road, I don't see much harm in a few more changes along the way.

> one of the things that sucks about making
> component changes to the board is that all the
> pictures on the documentation need to be redone.

True. Whoever does the documentation on building boards has done a great job, we don't need to make their job any harder than necessary.

>
> Agreed. What I'm thinking of is essentially a new
> (but compatible) programmer design, and it'll need
> a fair amount of testing. It looks like
> self-replication is still a ways down the road, I
> don't see much harm in a few more changes along
> the way.
>

cool, the compatibility part is definitely a crucial aspect.

>
> True. Whoever does the documentation on building
> boards has done a great job, we don't need to make
> their job any harder than necessary.

thanks... there are a few people involved in that. team effort!

Personally, I'd like to see the programmer not integrated into something else until we are ready for a major redesign. The iobox is simple, (I've infact in my code, made a 1-off naming it programmer with its own makefile and directory and all that), therefore simple to breadboard and you need room for code which a current pic won't have I don't think. And you want to keep the code complexity down. My circuit, to explain it, since you don't necessarily know is a card that speaks snap and that has a couple transistors to control the vdd and vpp voltages to the chip to be programmed and the data and clock lines (and blinky lights so you can tell whats happening). If we can put the snap code (o.w.e) comes next into a USB boot loader, that would eliminate the need for serial, however we might want to retain a serial versions for those that want the serial. It initially would keep you from having to unplug your one serial cable if that was all you had handy to switch from reprap to programmer and back. Just my $0.02.

bartlee45,

This sounds good (real hardware trumps pure ideas!), but from your description has the "chicken and egg" issue for the first PIC (the one in your iobox-based-programmer). Right? Whereas a bitbanging serial-based programmer on a modified PowerComms card would avoid that, because the host PC is doing the low level bit-banging of the programmer, so there is no PIC neeed in order to program a PIC. Also it could (probably) use existing host software. The two approaches have different benefits and different limitations.

As Zach suggested, doing both (especially if it is different people working on each approach!) is fine. Healthy competition leading to a better end product? :-)

We might end up with two options for PowerComms, the current one and the proposed one with inbuilt PIC programmer. Or if the board changes are simple enough, and the programmer parts can be optional... then we can make the new one "the standard" PowerComm PCB at some point, no harm done.

Have you designed the set of SNAP commands your programmer will use? Written firmware code for the PIC on your programmer module to implement those commands? Created some sort of test host software to talk to it? What is the next step in its evolution?

Jonathan

I've written a program that blinks some lights and gets confused. I think I have problems with interrupts and that's crucial to the pics doing anything timing related. I can add commands with ease, it's getting it to work right, thats the trick. I've messaged nophead to see if he would like me to collaborate with him, he seems engaged and knowlegeable.

For those of you who dont know about the other thread for the software updating, I have hit a snag and it will be impossible to make it work as long as we use the 628/648 chips. I have recommended the 16f88 as it has the same hardware and pinout and has its program memory available to the firmware, so we shall see what the devs would like to do with it.

Keep up the good work.

I guess I don't understand this. We have the Universal Boards or precursors being powered by the 16F628A's running several prototype RepRap machines as we speak.

IIRC, most of the memory on the PIC's is not RAM but read-only. What is the snag you are referring to that has suddenly made using the 628A's impossible?

forrest, please read posts thoroughly. he ran into the snag of not being able to program the circuit from within the firmware, not to actually run the chip normally.