[Biopython-dev] run_tests.py rewrite

[Peter]

On Mon, Feb 2, 2009 at 10:03 AM, Giovanni Marco Dall'Olio
<dalloliogm at gmail.com> wrote:
> On Sun, Feb 1, 2009 at 9:38 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
>> Hi everybody,
>>
>> I just uploaded to CVS a rewritten version of Tests/run_tests.py. This new
>> version automatically detects whether a test is a unittest-style test or a
>> print-and-compare test. By doing so, the unittest-style tests no longer
>> need to have a file containing the test output in Tests/output. For users,
>> run_tests.py works essentially the same as before.
>
> ok:
> - it seems it doesn't support doctest yet.

I think Michiel has only switched over test_Cluster.py thus far.  The
doctests are currently run via test_docstrings.py which is still a
print-and-compare test for now.

> - how this run_tests script is supposed to be called? Can you add this
> information in run_tests's docstring?

I guess it could be more explicit.

> If I run it from the biopython main directory (python
> Tests/run_tests.py) it gives me an error on test_AlignAce, but if I
> run it from within the Tests directory, it retunrs me an import error
> on test_CAPS.

Michiel hasn't changed this.  From the Tests directory do:
python run_tests.py

Or, from the parent directory (typically between doing build and install):
python setup.py test

Trying to call run_tests.py from outside the Tests directory is not
expected to work.

As explained in the docstring for run_tests.py (read the start of the
file), if you want to run just some of the tests, you can list them
like this:

python run_tests.py test_CAPS test_docstrings

You can include the py extension here optionally.

Could you show us the error with test_CAPS.py please, with details of
your setup.  This test is working for me.

> - some tests have some docstring associated. It would be more useful
> if, along with the name, you print these docs.
> For example, instead of:
> test_ACE ... ok
> It would be nice to see:
> test_ACE (tests the ACE module for .... which does ...) .... ok
> again, nose does this already.

I think it would be unnecessary text, of little interest to the typical user.

> - while you are at it, it would be nice to be able to define some
> global fixtures for all tests.
> Something like setup_BioSQL ran only once and with a warning that it
> has been created.
> nose already does that by using the @classmethod syntax - it's not
> very intuitive at first but it works.
>
> There is something that has never been clear to me about biopython's doctest.
> Are they supposed to be ran by the developers only, or by the users
> who install biopython manually?

Both - developers, and optionally/ideally anyone installing from source.

With CVS, they should also work for Windows users who used the
installation setup exe, but this requires them to download the source
code separately to get the unit tests.

> Some of the test seems to be written to check whether biopython can
> run on the user's computer correctly, others are tests on the code.

In a sense they are all tests on the code - some of the code by its
nature is a wrapper for a command line tool, so this may or not be
present on the user's machine.

Peter