[Biopython-dev] Continuous integration

[Tiago Antão]

Hi Peter,

2010/10/11 Peter <biopython at maubp.freeserve.co.uk>:
> (1) Several of our tests go online to the NCBI or UniProt etc.
> These tests can and do fail sometimes due to network issues.
> Also, having some/many buildbot slaves running on a regular
> basis (once a week? once a day?) would add up and this
> load may be unwelcome. Perhaps we need to add an -offline
> flag to run_tests.py which can skip any online tests?

That might be a good idea (to have an --offline flag, I mean). A very
good idea, indeed.

I would like to put the infrastructure in place (if people are
interested in going ahead with this...), but after that we need to
stabilize a test policy and that will mean answering questions like
that.

As far as I see we will have many builders (tests under different
conditions). Say 5 different Python versions (Jython included), at
least 3 OSes. This is already 15 builders. This can easily creep up.
Though the numbers are high, it is quite easy to maintain all this
stuff: 3 volunteer machines (one for each OS) are enough. The cool
thing about buildbot is that it is designed for volunteer machines to
be added, so you can start your buildbot slave on your laptop when you
are idle. It does not need an array of servers on demand to produce
the tests.

NCBI and Uniprot might not like to see 30 daily connections for tests
:( . So we might need to have, say, one weekly test for each OS doing
the network stuff (just a single Python version per OS, maybe) and
dailies not doing network loads.

> (2) You mention buildbot doesn't have built in support for
> spotting changes in a git repository - but can it do this for
> SVN? Since github.com also allow access to the git repo
> via svn that might be a more elegant workaround.

There are 2 different things to consider:
1. Spotting the git repository. There is no builtin support, but this
is TRIVIAL nonetheless with the general adaptor of buildbot. It works
like this:
     a. a developer does a push
     b. github has a hook system which allows for reporting a change
to the repository to a certain URL/CGI. Fully automated, transparent
to the developer.
     c. We supply a CGI that receives the event and informs buildbot.
There are CGIs for github. We just have to stuff one in a webserver.

2. The slaves/builders have to download github code. In this case,
buildbot HAS NATIVE SUPPORT.


> (3) Does the buildbot master require the buildbot slaves
> be online most/all of the time? Would a desktop machine
> which is typically only on during office hours on week
> days still be useful? I could probably answer this myself
> with a bit more background reading ;)

That is one of the wonders of buildbot. Just the server needs to be online.
You can indeed have a desktop machine: Whenever it suits you better
you start your buildbot slave, it connects to the server to see if
there is work to do and the server supplies work to be done. The
server can be instructed to only allow the slave to do a single task
at a time (to avoid overloading the slave).

I am now at a stage were I really need a server to test (with a public
address). I would volunteer to do the installation myself, but I would
need shell access to a machine where I could run a server process. No
root access is needed, but a web server is not enough as buildbot is
twisted based. Maybe we can convince the OBF to help. Again, I can
volunteer to do the installation. No root access is needed, just the
ability to run a server process and a couple of open ports.

Tiago