release processes

i've started documenting the release processes i've been doing to put files up on sourceforge. its a rough start, but here is the electronics release process:

[reprap.org]

They are all linked from the 'Documentation' page. I'll try and get the rest up soon.

This looks fine as far as it goes, but there seems to be no test process involved at all??

Surely after making a package intended for release, one would expect to test it.

Something like: move it to a different PC, open it, check all expected files are in there, do something to test as many of them as reasonable possible, read the README.txt (or similar) file and make sure it is up to date, with correct version numbers and dates etc. in it... open PDFs in a couple of different PDF viewers, check the version number and latest board fixes are in there... maybe upload some files to an online board producing place and make sure each file passes their checks for production-readiness? Only *then* would you upload to SourceForge? Right?

The test process above is ad hoc and probably incomplete/deficient, but the point is, there should *be* one, and we should be following it :-)

Some tests may belong in SubversionCommitGuidelines [www.reprap.org] as well as or instead of being in the Release process.

Jonathan

yes, good point.

although i think that the testing should go before the release instructions, on a separate page to make things clearer. the release instructions were intended to be instructions on *how* to make a release... assuming that you've already tested and made sure everything is correct. the testing process should be separate, especially since part of the testing process is *making* a -pre release. having a tested package then just becomes a prerequisite to putting it out on sourceforge.

if you'd like to come up with some pre-release testing guidelines, we could definitely link them to that page. something like this:

* make a x.x-pre release
* check to make sure files are there, etc.
* post on the forums and/or blog with a link to the pre release
* if its electronics, order a test-batch as a reality check, distribute to testers
* if no bugs are brought up 24 hours after posting on forums/blogs (and after real, live testing) then post a real release to sourceforge

IMO you want to test the *actual* release package, not (or not only) an earlier pre-release. It's too easy for someone else to slip in a "minor" change between your generation and testing of the pre-release package and your generation of the "real" package, otherwise!

You could (I suppose) have the release process include "make sure every single file in the release package is bit-for-bit identical to each file in the pre-release package you tested earlier" -- but there's no gain in doing two packages and then having to remember to check their content is the same, is there?? And when compilers/linkers embed date and time stamps into executables or libraries, then some files won't actually be identical... it gets messy.

Also, if the tests are an integral part of the release process, it is perhaps psychologically harder to decide "well, I'm in a hurry, I'll skip them" than if they are on a separate page somewhere else.

Ideally you should follow the same set of tests for a release candidate as for the real release; it's just that you may publicise them differently.

Jonathan

those are good ideas in theory, but in reality for at least the next year or so it will be one, maybe two people making releases. i dont have the time, nor do i think its worth the effort to take it that extra step. when we are a project the size of linux, then yes... those absolutely make sense.

if you'd like to write scripts to do testing, i'd be willing to integrate them. i think we'll be able to function just fine with an informal 'hey, heres the new release candidate... everyone look at it for problems' process. also, for some areas of the project (electronics, objects) theres not much we can do in the way of testing. firmware and software are two areas that could use unit/regression testing... but it seems that nobody is working on those areas much...

also, there is an easy way to make sure they are identical, which is to use the file you posted on the forums/blog as the actual release file. you dont need to make a new release, you simply rename the reprap-universal-controller-1.3-rc1.zip to reprap-universal-controller-1.3.zip. bingo! same files, no hassle.