[BioRuby] [Open-bio Board] Buildbot on testing.open-bio.org

[Peter Cock]

On Thu, Nov 3, 2011 at 10:37 AM, Pjotr Prins <pjotr.public14 at thebird.nl> wrote:
> On Thu, Nov 03, 2011 at 09:23:17AM +0000, Peter Cock wrote:
>> On Thu, Nov 3, 2011 at 6:18 AM, Pjotr Prins <pjotr.public14 at thebird.nl> wrote:
>> > On Wed, Nov 02, 2011 at 09:01:40PM +0000, Peter Cock wrote:
>> >> Have you decided which versions of Ruby should be tested?
>> >> For example, do you just need two targets, 1.8.x and 1.9.x?
>> >
>> > My feeling is that 1.9 is relevant for default testing. In the wild
>> > there may be versions running 1.8 - and if someone feels like it,
>> > they can set up the buildbot too. Right?
>>
>> Yes, but we need to know the planned matrix to setup the server.
>> So both Ruby 1.8.x and Ruby 1.9.x as two targets then, potentially
>> on the usual Windows 32 bit, Linux 32bit, Linux 64bit and Mac OS X?
>> So that could be 8 targets.
>
> OK, sounds good. It makes sense, also for other languages, to have
> two versions of compiler or interpreter for testing.

OK (pending possible changes for plugins later, see below).

>> > It is more important to test plugins. Raoul and I will set that up,
>> > as soon as we grasp the infrastructure.
>>
>> Right, we hear you. Essentially that will require knowing the
>> commands to call. Right now we have (roughly - Tiago can
>> correct me) the following to do an in-situ test (no installation):
>
>> (1) Get BioRuby from github (usually the latest, but you can
>> ask for a specific revision via the buildbot web interface)
>> (2) Run: ruby test/runner.rb (or similar)
>>
>> If it possible to have Ruby 1.8 and 1.9 installed at the same
>> time, is there a convention for calling them? I'd guess ruby1.8
>> and ruby1.9 if the same style as Python is used. Then we'd
>> have two versions of the above:
>>
>> (1) Get BioRuby from github
>> (2) Run: ruby1.8 test/runner.rb (or similar)
>>
>> or,
>>
>> (1) Get BioRuby from github
>> (2) Run: ruby1.9 test/runner.rb (or similar)
>>
>> The server configuration tells it which "recipe" to call on
>> which slaves, so if one machine only has Ruby 1.9, it will
>> not be asked to test on Ruby 1.8.
>>
>> We may also need slight variants for Windows vs Linux
>> vs Mac.

Does such a convention exist in Ruby for explicitly calling
Ruby 1.8 or 1.9? We can just require the aliases ruby1.8
and ruby1.9 as part of setting up a buildslave.

>> So what we'll need is an extended recipe which also gets the
>> appropriate gem versions for that revision, and tests them.
>> This could be a shell script, but if given as a list of commands
>> you can see the progress, output and success/failure of each
>> on the buildbot web interface.
>
> We don't have a protocol just yet. The important containers are bio-core,
> bio-core-ext and bio-biolinux. Each of these installs all important plugins.
>
> For testing you'd want a test of each *released* plugin on
> rubygems.org. The plugins to test are listed inside above containers
> - for example in
>
>  https://github.com/helios/bioruby-core/blob/master/Rakefile
>
> and can also be queried on rubygems (REST interface). I do that to
> generate http://biogems.info/.
>
> The install of a released plugin is simple:
>
>  gem install plugin
>
> installs the gem in
>
>  cd ~/.gems/gems/plugin-0.7
>
> and the tests should run with
>
>  rake test
>
> Some dependencies may be needed at test time. These can (in principle)
> be run with
>
>  cd ~/.gems/gems/plugin-0.7
>  bundle
>  rake test
>
> Note1: all plugins should support 'rake test' - though now they don't.

Is it possible to test the plugins *without* installing them? i.e. an
in-situ test? Or, can we control where they are installed (i.e. a temp
folder) and test them from there?

The point is the buildslave may be running several tests at once,
and while ideally it will be configured as a separate user account,
we don't want it using anything outside the BuildSlave's folder
(other than /tmp provided the usual precautions about name
clashes are taken),

>
> Note2: plugins require ruby 1.9.x - on Debian install ruby-1.9.1-dev
>

Maybe we'd want three targets (per OS) then?:

BioRuby core only on Ruby 1.8.x
BioRuby core only on Ruby 1.9.x
BioRuby with plugins on Ruby 1.9.x

Peter