[Biopython-dev] Debugging the warning filters in the unit tests

[Wibowo Arindrarto]

Hi Peter,

Could we apply a simple warning filter prior to the doctest run? Something
like:

with warnings.catch_warnings():
    warnings.simplefilter('ignore', ResourceWarning)
    suite.run(result)

I see that the run_tests.py script already can differentiate between
doctests and non-doctests. The filter can then be applied before the 'else'
clause from L375 ends (consequently, the previous 'if' should do its own
unfiltered `suite.run(result)` invocation. I tried this locally, and it
does work for the ResourceWarning from the doctests. It does not silence
all ResourceWarnings, though (test_Tutorial isn't really a doctest as seen
by `run_tests.py`, and I saw a separate warning raised from L431 during GC
cleanup).

However, maybe a related issue is whether we want to silence this warnings
at all?

After digging a bit into the tests and code, the warnings do seem to be
generated by real leaky handles (so not a Python bug). For example, the
SearchIO and SeqIO warnings here:
http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.3/builds/140/steps/shell/logs/stdio
seems
to be caused by the `index` or `index_db` function not closing its open
file handles. When the doctest suite resets the global environment for each
test (doctest.py:L1439), Python discovers that these handles are still open
and raises a warning.

I haven't checked if the same issue is the root cause of other
ResourceWarning occurences. Perhaps we need to look deeper into this?

Cheers,
Bow


On Wed, Apr 10, 2013 at 1:17 PM, Peter Cock <p.j.a.cock at googlemail.com>wrote:

> Hi all,
>
> I've suspected for a some time that there is some subtle bug
> in run_tests.py or a limitation in the Python warnings module -
> although our code tries to reset the filters between tests this
> doesn't seem to be working all the time.
>
> The obvious effect of this is getting loads of ResourceWarning
> messages under Python 3.3, e.g.
>
> http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.3/builds/130/steps/shell/logs/stdio
>
> Sometimes theses warnings can manifest as real bugs, in
> particular on Windows where you cannot delete a file when
> there is still an open handle to it which will not be closed
> until the garbage collection runs. i.e. Bugs which tend to
> only show under Windows with PyPy or Jython. e.g.
>
> https://github.com/biopython/biopython/commit/943fffe2835dca4a996a6a171f026f6374ecaaa9
>
> After fixing most of the handle leaks (which is a good thing
> to do anyway), the remainder of being shown are in the
> documentation tests (where fixing them can make the
> examples harder to follow):
>
> http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.3/builds/140/steps/shell/logs/stdio
>
> I'm hoping someone might have some insight into the
> current (not entirely successful) attempts to handle
> warning filters in run_tests.py
>
> Note this is complicated because many of the
> individual test_XXX.py files manipulate the (global)
> warnings filters, either to silence harmless expected
> warnings, or to verify that desired warnings appear.
>
> Thanks,
>
> Peter
>
> P.S.
>
> One of the first things I want to do once we drop support
> for Python 2.5 (currently we're planning just one more
> release support it) is update the doctests to use context
> managers for file handles (i.e. use the with statement).
> Currently we can't easily do this in plain doctests without
> including this line explicitly:
>
> from __future__ import with_statement
>
> We could do that automatically in test_Tutorial.py though
> as part of the code which extracts doctest examples form
> the LaTeX and runs them.
> _______________________________________________
> Biopython-dev mailing list
> Biopython-dev at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/biopython-dev
>