[Open-bio-l] Automated testing server; was [BioRuby] tutorial

[Tiago Antão]

On mobile phone, fast mail. Buidbot CAN be trigerred by git commits.

2011/3/4, Peter Cock <p.j.a.cock at googlemail.com>:
> On Fri, Mar 4, 2011 at 2:14 PM, Chris Fields <cjfields at illinois.edu> wrote:
>> On Mar 4, 2011, at 5:45 AM, Peter Cock wrote:
>>> On Thu, Mar 3, 2011 at 8:52 PM, Chris Fields <cjfields at illinois.edu>
>>> wrote:
>>>> On Mar 3, 2011, at 2:29 PM, Raoul Bonnal wrote:
>>>>
>>>>> Dear All,
>>>>> a) this discussion is a good starting point for a workgroup during
>>>>> Codefest
>>>>> 2011 and/or BOSC (can't post to bopen-bio list cause I don't have the
>>>>> addr
>>>>> here, sorry) a.idea) why not create a common "machine" on the cloud for
>>>>> testing our projects ?! It would be fun to see all the bio* with stats
>>>>> about
>>>>> testing etc...
>>>>
>>>> I think the buildbot instance covers some of that, but they are mainly
>>>> automated builds:
>>>>
>>>> http://testing.open-bio.org/
>>>>
>>>> biopython is the only one set up currently.
>>>
>>> Note that this is a small Amazon cloud machine, and doesn't actually run
>>> the tests itself (that would require a more expensive machine due to the
>>> higher load). Instead we need client machines (covering a range of OS,
>>> Python version etc) whose run-time is donated (currently by Biopython
>>> developers or their employers).
>>>
>>> When setting up http://testing.open-bio.org/ the intention was to have
>>> it host other Bio* projects too - and they don't have to be using the
>>> buildbot software (but they could if they wanted to).
>>
>> I think it would be nice to have a smoke server there as well (I don't
>> think buildbot can do that, AFAIK).  Will ask Chris D about it.
>
> Can you clarify what you mean by smoke server? I'd assumed it was
> a product name but now it sounds like you mean some methodology.
>
> We have buildbot setup to do a nightly test of the latest code on all
> the slaves. Additionally, via the web interface an admin can force a
> rebuild, and force a build of the current latest code.
>
> In theory one can also hook up buildbot to run tests triggered by
> commits - I understand this isn't available for git repositories yet
> though (at least, that's my recollection from dicussion with Tiago,
> BCC'd in case he isn't on open-bio-l yet).
>
>> I like the fact that we now are diversifying beyond perl-based
>> solutions with buildbot and redmine (I would include mediawiki,
>> but PHP gives me the hives :).  Any easily-installable Java-based
>> ones?
>
> As long as the OBF has enough people familiar with the tool
> to support it, I don't care what language it is written in.
>
>>>>> b) testing is very important and new contributes should be accepted
>>>>> only with tests, which implies a problem how to check fake tests? :-)
>>>>
>>>> Code review and checking the test results.  Otherwise, you have to trust
>>>> your developers.
>>>
>>> This is something we're pushing much more with Biopython in recent
>>> years (a depressing amount of legacy code had no tests when I first
>>> got involved, things are a lot better now). For any new code
>>> contributions
>>> we do ask for tests and minimal documentation (and try and work
>>> with the contributor to help this happen).
>>
>> Same here.  This is a common cycle with OS projects.
>>
>>>>> c) usually ruby's community relies a lot on doc==tests that's
>>>>> completely wrong, because the newbies can't read fluently the
>>>>> code, also because there are different tools and sometimes a lot
>>>>> of mocking.
>>>
>>> With Biopython we're starting to make more use of testable docs
>>> (Python's doctest system) which is really good I think. But you are
>>> right, you need proper unit tests AS WELL. With our doctests,
>>> the primary aim is to be useful documentation (which can double
>>> as a basic functional test).
>>
>> The problem with bioperl is that much of the testing framework
>> for in-line or testable docs wasn't present during the initial
>> development process.  We also have a LOT of cruft modules
>> that lack true author support.
>
> Also true for Biopython to some extent.
>
>> What we are planning on for future releases is to modularize
>> bioperl installations, with an installer to put together the dists
>> one wants.  The current release has well over 1000 classes
>> and modules; modularizing will help remove some of the
>> maintenance headaches we had been facing during
>> development.  We are also pushing new code to go into
>> separate CPAN installations, as these can be added to an
>> installer as needed (as someone else long ago put it, we're
>> essentially creating a mini-BioPerlPAN within CPAN).
>
> I'm happy enough with the current Biopython monolith install -
> although we are slowly gaining extra runtime dependencies,
> doing the install/build is pretty easy. As the Python packaging
> infrastructure matures, we too may want to consider
> modularising - but for now I feel that is premature.
>
> There is also an issue here for testing - if you can have a
> diverse set of build slaves, then hopefully some will have
> and some will lack each soft dependency. This is useful to
> check graceful failure when things are missing.
>
> Peter
>

-- 
Enviada a partir do meu dispositivo móvel

"If you want to get laid, go to college.  If you want an education, go
to the library." - Frank Zappa