[Biopython-dev] Rethinking Biopython's testing framework

[Giovanni Marco Dall'Olio]

On Sun, Dec 28, 2008 at 5:04 PM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
> --- On Sat, 12/27/08, Giovanni Marco Dall'Olio <dalloliogm at gmail.com> wrote:
>> >> > If we use nose, does that mean adding another
>> >> > dependency to Biopython? If so, I don't think
>> >> > it's worth it. If not, how does this work?
>> >>
>> >> nose is a testing framework, so it is a dependency
>> >> only for developers.
>> >
>> > If we use nose, can our users still run the Biopython
>> tests (without having to install nose first)?
>>
>> Yes, but they will have to do it manually, or with a
>> wrapper script (as it is now).


> If we use a wrapper script, then what is the difference between using nose and using Python's unittest framework?

The wrapper script won't be as efficient as using nose.
Writing a separated wrapper script will take much time and it will be
very difficult to mantain updated; moreover, you will have to test the
wrapper script itself, to prove that it works and doesn't alter the
results of the tests.

Nose is not a replacement for unittests: it is a tool that searches
for every unittest and script that look like a test, and execute it.
It has a few advantages more, for example it enables global methods
for setUp and tearDown, but it is not necessary to use them.


If you want to reorganize the biopython's testing infrastructure, then
you should think about adopting a serious testing environment, whether
it is nose or something else. You can't continue on relying on wrapper
scripts, they are too difficult to mantain and they are not really
scientifically valid.

The pygr project (another bioinformatics library in python) make use
of nose, and they explain how in their documentation:
- http://bioinformatics.ucla.edu/pygr_0_7_b3/testing-doc.html

Please have a look at the pages I have posted before.


> By manually, do you mean running each test separately by hand?

I mean they will have to be run in the same way as it is now.

Maybe, there is a way to use nose itself to create a wrapper script
automatically.
In fact, what nose does is to find all the functions that look like
tests, and then execute them. It should be possible to just save the
statements that are executed in a log file, that can be used as a
wrapper script.
If this option doesn't exists yet, we can just propose it to nose's developers.

In brief, I think it doesn't make sense to write a new testingg
framework just for biopython, when there are many already existing
tool available and free to use.



> --Michiel.
>
>
>
>



-- 

My blog on bioinformatics (now in English): http://bioinfoblog.it