[Biopython-dev] run_tests.py rewrite

[Michiel de Hoon]

Just for clarification:
The only purpose of the run_tests.py rewrite is to remove the requirement of an output file for unittest-based tests. While personally I am in favor of unittest-based tests, it is not my intention to remove support for the print-and-compare tests. I expect that for the most part, the test scripts themselves won't need to be changed. A few test scripts will need to be adjusted; test_Cluster.py was one of them. The main visible result of the new run_tests.py is that we will be able to remove the output files in Tests/output/test_* for the unittest-based tests.
As Peter wrote, the doctests are being run via test_docstrings.py, which is picked up by run_tests.py

--Michiel.


--- On Mon, 2/2/09, Peter <biopython at maubp.freeserve.co.uk> wrote:

> From: Peter <biopython at maubp.freeserve.co.uk>
> Subject: Re: [Biopython-dev] run_tests.py rewrite
> To: dalloliogm at gmail.com
> Cc: mjldehoon at yahoo.com, biopython-dev at biopython.org
> Date: Monday, February 2, 2009, 5:29 AM
> On Mon, Feb 2, 2009 at 10:03 AM, Giovanni Marco
> Dall'Olio
> <dalloliogm at gmail.com> wrote:
> > On Sun, Feb 1, 2009 at 9:38 AM, Michiel de Hoon
> <mjldehoon at yahoo.com> wrote:
> >> Hi everybody,
> >>
> >> I just uploaded to CVS a rewritten version of
> Tests/run_tests.py. This new
> >> version automatically detects whether a test is a
> unittest-style test or a
> >> print-and-compare test. By doing so, the
> unittest-style tests no longer
> >> need to have a file containing the test output in
> Tests/output. For users,
> >> run_tests.py works essentially the same as before.
> >
> > ok:
> > - it seems it doesn't support doctest yet.
> 
> I think Michiel has only switched over test_Cluster.py thus
> far.  The
> doctests are currently run via test_docstrings.py which is
> still a
> print-and-compare test for now.
> 
> > - how this run_tests script is supposed to be called?
> Can you add this
> > information in run_tests's docstring?
> 
> I guess it could be more explicit.
> 
> > If I run it from the biopython main directory (python
> > Tests/run_tests.py) it gives me an error on
> test_AlignAce, but if I
> > run it from within the Tests directory, it retunrs me
> an import error
> > on test_CAPS.
> 
> Michiel hasn't changed this.  From the Tests directory
> do:
> python run_tests.py
> 
> Or, from the parent directory (typically between doing
> build and install):
> python setup.py test
> 
> Trying to call run_tests.py from outside the Tests
> directory is not
> expected to work.
> 
> As explained in the docstring for run_tests.py (read the
> start of the
> file), if you want to run just some of the tests, you can
> list them
> like this:
> 
> python run_tests.py test_CAPS test_docstrings
> 
> You can include the py extension here optionally.
> 
> Could you show us the error with test_CAPS.py please, with
> details of
> your setup.  This test is working for me.
> 
> > - some tests have some docstring associated. It would
> be more useful
> > if, along with the name, you print these docs.
> > For example, instead of:
> > test_ACE ... ok
> > It would be nice to see:
> > test_ACE (tests the ACE module for .... which does
> ...) .... ok
> > again, nose does this already.
> 
> I think it would be unnecessary text, of little interest to
> the typical user.
> 
> > - while you are at it, it would be nice to be able to
> define some
> > global fixtures for all tests.
> > Something like setup_BioSQL ran only once and with a
> warning that it
> > has been created.
> > nose already does that by using the @classmethod
> syntax - it's not
> > very intuitive at first but it works.
> >
> > There is something that has never been clear to me
> about biopython's doctest.
> > Are they supposed to be ran by the developers only, or
> by the users
> > who install biopython manually?
> 
> Both - developers, and optionally/ideally anyone installing
> from source.
> 
> With CVS, they should also work for Windows users who used
> the
> installation setup exe, but this requires them to download
> the source
> code separately to get the unit tests.
> 
> > Some of the test seems to be written to check whether
> biopython can
> > run on the user's computer correctly, others are
> tests on the code.
> 
> In a sense they are all tests on the code - some of the
> code by its
> nature is a wrapper for a command line tool, so this may or
> not be
> present on the user's machine.
> 
> Peter