[Open-bio-l] Automated testing server; was [BioRuby] tutorial

[Peter Cock]

Hi all,

I've retitled this thread to reflect the current topic, and CC'd the open-bio-l
mailing list as Raoul suggested. Can we continue the discussion there?
http://lists.open-bio.org/mailman/listinfo/open-bio-l

On Thu, Mar 3, 2011 at 8:52 PM, Chris Fields <cjfields at illinois.edu> wrote:
> On Mar 3, 2011, at 2:29 PM, Raoul Bonnal wrote:
>
>> Dear All,
>> a) this discussion is a good starting point for a workgroup during Codefest
>> 2011 and/or BOSC (can't post to bopen-bio list cause I don't have the addr
>> here, sorry) a.idea) why not create a common "machine" on the cloud for
>> testing our projects ?! It would be fun to see all the bio* with stats about
>> testing etc...
>
> I think the buildbot instance covers some of that, but they are mainly
> automated builds:
>
> http://testing.open-bio.org/
>
> biopython is the only one set up currently.

Note that this is a small Amazon cloud machine, and doesn't actually run
the tests itself (that would require a more expensive machine due to the
higher load). Instead we need client machines (covering a range of OS,
Python version etc) whose run-time is donated (currently by Biopython
developers or their employers).

When setting up http://testing.open-bio.org/ the intention was to have
it host other Bio* projects too - and they don't have to be using the
buildbot software (but they could if they wanted to).

>> b) testing is very important and new contributes should be accepted
>> only with tests, which implies a problem how to check fake tests? :-)
>
> Code review and checking the test results.  Otherwise, you have to trust
> your developers.

This is something we're pushing much more with Biopython in recent
years (a depressing amount of legacy code had no tests when I first
got involved, things are a lot better now). For any new code contributions
we do ask for tests and minimal documentation (and try and work
with the contributor to help this happen).

>> c) usually ruby's community relies a lot on doc==tests that's
>> completely wrong, because the newbies can't read fluently the
>> code, also because there are different tools and sometimes a lot
>> of mocking.

With Biopython we're starting to make more use of testable docs
(Python's doctest system) which is really good I think. But you are
right, you need proper unit tests AS WELL. With our doctests,
the primary aim is to be useful documentation (which can double
as a basic functional test).

>> c.a) recently i had an idea, but I have no time to do that: establish
>> a relationship between a mock and a real object and validate the
>> mock or enable it only if the real object has been tested before.
>> That would make mocking more sens to me, but i think this is
>> another story.
>
> Yes, I've thought about that as well re: Moose-based work, haven't
> wrapped my head around it either.

I'm not quite following - it may be a terminology difference.

>> d) I agree with Toshiaki, after this very useful brain storming
>> start working (Impressed to see all this traffic :-) )
>>
>> Cheers.
>
> +1.  Will be interesting to see how things progress.
>
> chris

Same here :)

Peter