Simon McAuliffe wrote:
> I'd just like to make a suggestion regarding the use of the subversion
> repository...
>
> I would personally like to see it only really used for source, not as a
> general repository for all sorts of binary files. That is because it
> tracks all versions of everything forever and if we keep putting in
> large binary files that essentially completely change on any revision,
> then the repository gets really large really quickly. That is not
> really the intended use of a system like subversion. Subversion stores
> only the changes between revisions but for certain file types,
> particularly compressed files or various compiled files, there is very
> little relationship between successive versions so you get full copies
> every time.
>
> So for example, PIC firmware binaries, STL files, photos or video clips
> are not so appropriate.
>
> Perhaps we can put some of those things just on the sourceforge
> downloads instead? Or somewhere else? They can still always be
> regenerated from what is in subversion, and they can be archived
> somewhere else too.
>
> What do others think?
We are all agreed that the repository is absolutely the best way to
handle the software development. And (almost implicitly, and without
much decision making) we use the same model for mechanical and
electronic development. This it is a huge success (and as far as I know
- which is not very far - it is also unique in hardware engineering).
The problem comes about because we have compilers for both Java and C,
and so the way that the repository tools work allows us to store small
amounts of source data for them, and even smaller changes, and then to
generate the big files we need from them locally using make (or equivalent).
But the CAD tools that we use have no similar process. You have to load
files in and save them in a different format interactively. This means
that we tend to save the output in order not to have to repeat that
interactive process all the time.
But those CAD tools (KiCad and AoI) are both open-source, so it ought to
be comparatively easy to add a batch compile option to them. I should
emphasize that I'm not suggesting that we devote our effort to this; I'm
just pointing out that it's both technically possible and desirable.
If this were to be done the problem that Simon correctly identified
would go away.
So. Are we actually causing a problem at the repository, in the sense
that we're taking so much space that we're getting complaints, or will
get complaints in the near future? It seems to me that disc capacity
and bandwidth are so fat these days (and getting fatter) that we're
probably not; well - not short-term, anyway. But if I'm wrong then we
should do something now.
But if I'm right, then I think we should make the compile-option
suggestion to the developers of AoI and KiCad, keep doing what we're
doing temporarily (because it's such a good engineering discipline), and
immediately switch to that option when it becomes available. (Note that
KiCad source files are text, incidentally.)
As a final thought, video compression looks for coherence not just in a
frame, but between frames in the sequence. It would be really neat to
use a similar lossless compression tool for binaries in a
version-control repository that treated each version as a frame in such
a sequence.
--
Best wishes
Adrian
Dr Adrian Bowyer
[
staff.bath.ac.uk]
[
reprap.org]
_______________________________________________
Developers mailing list
Developers@reprap.org
[
reprap.org]