[Biopython-dev] buildbot issue on Python 3.1 - stdout?

[Peter Cock]

On Thu, Dec 6, 2012 at 10:07 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>
> OK, the buildbot looks happy now from the SearchIO work.
>
> There is one issue under Python 3.1.5 on a 64 bit Linux server,
> which I suspect is down to the Python version (this buildslave
> used to run an older version - Python 3.1.3 (separate email
> to follow).

There are 18 test failures like this - all to do with handles and stdout,
which have been happening for a while now but I've not found time
to look into it. Example:

======================================================================
ERROR: test_needle_piped (test_Emboss.PairwiseAlignmentTests)
needle with asis trick, output piped to stdout.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/EmbossIO.py",
line 74, in __next__
    line = self._header
AttributeError: 'EmbossIterator' object has no attribute '_header'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/Tests/test_Emboss.py",
line 571, in test_needle_piped
    align = AlignIO.read(child.stdout, "emboss")
  File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/__init__.py",
line 418, in read
    first = next(iterator)
  File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/__init__.py",
line 366, in parse
    for a in i:
  File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/EmbossIO.py",
line 77, in __next__
    line = handle.readline()
AttributeError: '_io.FileIO' object has no attribute 'read1'

Lasting working build, Python 3.1.3,
http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.1/builds/710/steps/shell/logs/stdio
https://github.com/biopython/biopython/commit/e90db11f4a1d983bc2bfe12bec30edbdbb200634

Next build (after a couple of weeks offline while this server was
being rebuilt), Python 3.1.5,
http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.1/builds/722/steps/shell/logs/stdio
https://github.com/biopython/biopython/commit/3ea4ea58ed80d6e517699bcab8810398f9ce5957

The timing does suggest an issue introduced in the rebuild, and
the obvious difference is the version of Python jumped from
3.1.3 to 3.1.5 (likely things like NumPy etc also changed).

There were some security fixes only in Python 3.1.5, none of
which sound relevant here:
http://www.python.org/download/releases/3.1.5/

The change log for Python 3.1.4 is longer, and does mention
stdout/stderr issues so this is perhaps the cause:
hg.python.org/cpython/raw-file/feae9f9e9f30/Misc/NEWS

See also http://bugs.python.org/issue4996 as possibly
related. The whole Python 3 text vs binary handle issue
is important with stdout/stderr.

What I am doing now is testing those two commits (with
Python 3.1.5) to confirm they both fail, and thus rule out
a Biopython code change in those two weeks being to
blame.

Peter