[Biopython-dev] Testing examples in the Tutorial

[Peter Cock]

Hi all,

In part prompted by recent discussions on the BioPerl and BioRuby
mailing lists about documentation and testing, I was inspired to try
and code up something I'd been thinking about doing for some time:
treat the embedded examples in our tutorial as doctests.

This required adding a little extra markup to the LaTeX file since
I wanted the tests to be "opt in" (some examples will not make
good tests, for example random outputs or command line tool
dependencies). I've gone for a special comment line, %doctest
(which can include an optional name for the test) to mark the
the following verbatim block is a doctest. Likewise, %cont-doctest
indicates the following verbatim block is a continuation of the
previous doctest - important when there is text in the middle
so variable definitions or import statements would otherwise
need to be repeated.

Then I wrote test_Tutorial.py which parses Tutorial.tex to find
these examples, and turn them into doctests. It is a bit hacky
but appears to do the job. Running it like this:

python run_tests.py test_Tutorial.py

you can very minimal output from failures. For more details,
use this:

python test_Tutorial.py

I've already marked up some basic Seq examples (and in the
process fixed a couple of trivial errors and one typo) so you
can see how this works:

https://github.com/biopython/biopython/commit/dd3f5f7cf9a4bd120e4b8634f530f6de0b873622

Yes, maybe I should have asked before committing this,
but I wanted to run it on all our platforms via the buildbot.
That was a good idea as the is a tuple issue on Jython
and Python 2.4 that I've seen before:

https://github.com/biopython/biopython/commit/45cb3e05aa86c96820a3d866d766d0fc62cb830a

So, any comments or feedback? If people have ideas for
improvement that's great - otherwise I think we should begin
to mark up the rest of Tutorial.tex for maximum test coverage...

Peter