[Biopython-dev] run_tests.py rewrite

[Giovanni Marco Dall'Olio]

On Mon, Feb 2, 2009 at 11:29 AM, Peter <biopython at maubp.freeserve.co.uk> wrote:
> On Mon, Feb 2, 2009 at 10:03 AM, Giovanni Marco Dall'Olio
> <dalloliogm at gmail.com> wrote:
>> On Sun, Feb 1, 2009 at 9:38 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
>>> Hi everybody,
>>>
>>> I just uploaded to CVS a rewritten version of Tests/run_tests.py. This new
>>> version automatically detects whether a test is a unittest-style test or a
>>> print-and-compare test. By doing so, the unittest-style tests no longer
>>> need to have a file containing the test output in Tests/output. For users,
>>> run_tests.py works essentially the same as before.
>>
>> ok:
>> - it seems it doesn't support doctest yet.
>
> I think Michiel has only switched over test_Cluster.py thus far.  The
> doctests are currently run via test_docstrings.py which is still a
> print-and-compare test for now.

ah! I see.
However, this way, test_docstring will be difficult to mantain in the future.
A better solution would be to have run_test.py go throught all
biopython's modules, and then execute every doctest it encounters.
You can do this with doctest.DocTestFinder (have a look at nose's
code, which does it already:
- http://code.google.com/p/python-nose/source/browse/trunk/nose/plugins/doctests.py)


> Could you show us the error with test_CAPS.py please, with details of
> your setup.  This test is working for me.

sorry.. it works fine if I run it from within the Tests dir.

>> - some tests have some docstring associated. It would be more useful
>> if, along with the name, you print these docs.
>> For example, instead of:
>> test_ACE ... ok
>> It would be nice to see:
>> test_ACE (tests the ACE module for .... which does ...) .... ok
>> again, nose does this already.
>
> I think it would be unnecessary text, of little interest to the typical user.

It would be useful to make sure that every test is documented.
Most of the tests in biopython are not: for example, can you tell
which is the difference between test_Fasta.py and test_Fasta2.py?

Moreover, why the typical user should be running biopython's tests?


>> - while you are at it, it would be nice to be able to define some
>> global fixtures for all tests.
>> Something like setup_BioSQL ran only once and with a warning that it
>> has been created.
>> nose already does that by using the @classmethod syntax - it's not
>> very intuitive at first but it works.

What about having support to global fixtures?
For example, many test scripts begin in the same way: they 'import
numpy', check for python's version, etc.. All of this could be moved
to a global fixture and then executed only once for all the tests.
All the Bio.Seq files could open the sequence files only once,
therefore it will be easier to write more complex tests.
The Bio.BioSQL modules could create a database only once, reducing memory usage.


> Peter
>



-- 

My blog on bioinformatics (now in English): http://bioinfoblog.it