[BioRuby] GSoC: Testing framework for biogem plugins

[Pjotr Prins]

Hi Jakub,

On Mon, Mar 19, 2012 at 05:29:24PM +0100, Jakub Naliwajek wrote:
> Hi,
> 
> My name's Jakub. I'm a CS student and a back-end RoR developer from Poland.
> 
> I just wanted to check if my reasoning of whole process of adding a new bio
> gem is right:
> 
> 1. New bio gem is created by someone on GitHub;
> 2. Then gem is added by him to RubyGems;
> 3. Then gem is added (by him and *accepted* by OBF?) to BioGems;

No, it is automatic. If you gem starts with bio- it will
automatically show up on http://biogems.info/

> 4. Then all three services are automatically linked together by a framework;
> 5. Tests (all tests across gem and its dependencies) run for a first time
> (initial release);

At the moment the tests are handled by travis-ci. You need to
register your project with them (two small steps!), and create a
.travis.yml file in your tree.

I like travis-ci, even when the service would disappear, it would be
reasonably easy to recreate ourselves (the source code is online).

> 6. After that tests run only:
>    *** when manually triggered (BioGems link action somewhere) OR;
>    *** labeled on GitHub OR;
>    *** new release (check if pushed to BioGems [or RubyGems if they have
> such event handler]) is out;

The default for travis is to run a test, every time you push to
github. There is also a manual override.

> 7. (Eventually) with person responsible about restyle of bioruby.org we
> decide how (if any) those results are going to look at main
> bioruby.orgsite (not dev-only BioGems.info)

Yes, both sites could be one site, but I agree biogems.info mostly
targets experienced developers. Design decisions are part of the
discussion on the ML. We created the BioRuby panel to ascertain we
find the best solution for the community.

> (For already existing gems let's say we begin from point 5 treating current
> release as an initial one)

That would be great.

> Of course, besides testing each gem with its dependencies individually, we
> include option to test different BioGems together (different setups) and
> display results at BioGems.info too.

Agree. 

Gems posted on rubygems.org have tagged releases on github. We can
test each release, with its dependencies. Travis-ci, btw, would also
allow us to run the test in dependent gems in one go(!)

> Test results are displayed not only with "failed/passed" statement but also
> with error log so developer can see straight from BioGems site what is
> wrong, right?
> 
> Is it okay or did I forget something?

Sounds right to me. I am glad you appreciate the importance of a
proper testing framework.

> Also, I lack CoffeeScript knowledge but I guess it's not that hard to learn.

Yes.

Pj.