[Biopython-dev] Rethinking Biopython's testing framework

[Giovanni Marco Dall'Olio]

On Mon, Dec 29, 2008 at 1:18 AM, Peter <biopython at maubp.freeserve.co.uk> wrote:
> Giovanni wrote:
>>> nose is a testing framework, so it is a dependency
>>> only for developers.
>
> Requiring another external dependency does count against using nose -
> it is much nicer if anyone installing Biopython from source can run
> our test suite without having to install anything further.

As I was saying before, it will be not a dependency. It's an external
tool that you can use or not to execute the tests automatically.
Also, it is not a replacement for unittest. It is comparable to using
epydoc for the documentation.

> Giovanni wrote:
>> If you want to reorganize the biopython's testing infrastructure, then
>> you should think about adopting a serious testing environment, whether
>> it is nose or something else. You can't continue on relying on wrapper
>> scripts, they are too difficult to mantain and they are not really
>> scientifically valid.
>
> I'm not sure I understand your point here (especially re difficult to
> maintain and not scientifically valid).
>

The wrapper script itself is a program. Therefore, if you want to be
paranoid, you will have to test it too :)
It will be difficult to mantain because everytime you will have to
modify it to adapt to the new tests etc.
Many big opensource python project make use of this framework, and it
has already been proven to work correctly; so the quality of biopython
would be comparable with those existing projects.
Another projecty that make use of nose is pytables (hdf5 format
wrapper for python). They say they have some billions of tests :).

> I'm failry happy with the current test framework - I would rather see
> any effort be spent on writing more tests under the current framework
> than switching the framework itself.
>
> Giovanni wrote:
>> In brief, I think it doesn't make sense to write a new testingg
>> framework just for biopython, when there are many already existing
>> tool available and free to use.
>
> We haven't been talking about writing a new test frame work (which I
> agree isn't a good idea).  Rather we're talking about a modification
> to the existing Biopython test framework (part of which uses the built
> in python unittest library).  Michiel's proposal on 24th Dec seems
> like it will simplify working with unittest based tests (especially
> not having to track their trivial output in CVS/SVN).

Then you will have to develop a way to execute only some of the tests
(e.g. only those who doesn't make use of internet connection, or only
those who make use of a database).
You will need to write some methods for running some setUp and
tearDown methods globally.
You will have to verify your wrapper script works.
In short, you will end up with writing a tool which will be really
similar to nose. So, since this tool already exists now, you will save
a lot of time by using it.
Michel's proposal is good, but I am saying that there are already
tools that do the same thing automatically.

>
> Peter
>



-- 

My blog on bioinformatics (now in English): http://bioinfoblog.it