[Biopython-dev] Debugging the warning filters in the unit tests

[Peter Cock]

Hi all,

I've suspected for a some time that there is some subtle bug
in run_tests.py or a limitation in the Python warnings module -
although our code tries to reset the filters between tests this
doesn't seem to be working all the time.

The obvious effect of this is getting loads of ResourceWarning
messages under Python 3.3, e.g.
http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.3/builds/130/steps/shell/logs/stdio

Sometimes theses warnings can manifest as real bugs, in
particular on Windows where you cannot delete a file when
there is still an open handle to it which will not be closed
until the garbage collection runs. i.e. Bugs which tend to
only show under Windows with PyPy or Jython. e.g.
https://github.com/biopython/biopython/commit/943fffe2835dca4a996a6a171f026f6374ecaaa9

After fixing most of the handle leaks (which is a good thing
to do anyway), the remainder of being shown are in the
documentation tests (where fixing them can make the
examples harder to follow):
http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.3/builds/140/steps/shell/logs/stdio

I'm hoping someone might have some insight into the
current (not entirely successful) attempts to handle
warning filters in run_tests.py

Note this is complicated because many of the
individual test_XXX.py files manipulate the (global)
warnings filters, either to silence harmless expected
warnings, or to verify that desired warnings appear.

Thanks,

Peter

P.S.

One of the first things I want to do once we drop support
for Python 2.5 (currently we're planning just one more
release support it) is update the doctests to use context
managers for file handles (i.e. use the with statement).
Currently we can't easily do this in plain doctests without
including this line explicitly:

from __future__ import with_statement

We could do that automatically in test_Tutorial.py though
as part of the code which extracts doctest examples form
the LaTeX and runs them.