[Biopython-dev] buildbot issue on Python 3.1 - stdout?

[Peter Cock]

On Thu, Dec 6, 2012 at 10:24 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Thu, Dec 6, 2012 at 10:07 AM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>
>> OK, the buildbot looks happy now from the SearchIO work.
>>
>> There is one issue under Python 3.1.5 on a 64 bit Linux server,
>> which I suspect is down to the Python version (this buildslave
>> used to run an older version - Python 3.1.3 (separate email
>> to follow).
>
> There are 18 test failures like this - all to do with handles and stdout,
> which have been happening for a while now but I've not found time
> to look into it. Example:
>
> ======================================================================
> ERROR: test_needle_piped (test_Emboss.PairwiseAlignmentTests)
> needle with asis trick, output piped to stdout.
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/EmbossIO.py",
> line 74, in __next__
>     line = self._header
> AttributeError: 'EmbossIterator' object has no attribute '_header'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>   File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/Tests/test_Emboss.py",
> line 571, in test_needle_piped
>     align = AlignIO.read(child.stdout, "emboss")
>   File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/__init__.py",
> line 418, in read
>     first = next(iterator)
>   File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/__init__.py",
> line 366, in parse
>     for a in i:
>   File "/home_local/buildslave/BuildBot_Biopython/lin3164/build/build/py3.1/build/lib.linux-x86_64-3.1/Bio/AlignIO/EmbossIO.py",
> line 77, in __next__
>     line = handle.readline()
> AttributeError: '_io.FileIO' object has no attribute 'read1'
>
> Lasting working build, Python 3.1.3,
> http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.1/builds/710/steps/shell/logs/stdio
> https://github.com/biopython/biopython/commit/e90db11f4a1d983bc2bfe12bec30edbdbb200634
>
> Next build (after a couple of weeks offline while this server was
> being rebuilt), Python 3.1.5,
> http://testing.open-bio.org/biopython/builders/Linux%2064%20-%20Python%203.1/builds/722/steps/shell/logs/stdio
> https://github.com/biopython/biopython/commit/3ea4ea58ed80d6e517699bcab8810398f9ce5957
>
> The timing does suggest an issue introduced in the rebuild, and
> the obvious difference is the version of Python jumped from
> 3.1.3 to 3.1.5 (likely things like NumPy etc also changed).
>
> There were some security fixes only in Python 3.1.5, none of
> which sound relevant here:
> http://www.python.org/download/releases/3.1.5/
>
> The change log for Python 3.1.4 is longer, and does mention
> stdout/stderr issues so this is perhaps the cause:
> hg.python.org/cpython/raw-file/feae9f9e9f30/Misc/NEWS
>
> See also http://bugs.python.org/issue4996 as possibly
> related. The whole Python 3 text vs binary handle issue
> is important with stdout/stderr.
>
> What I am doing now is testing those two commits (with
> Python 3.1.5) to confirm they both fail, and thus rule out
> a Biopython code change in those two weeks being to
> blame.
>
> Peter

Confirmed, using test_Emboss.py and Python 3.1.5 on
this machine (running as the buildslave user using the
same Python 3.1.5 installation), using the current tip
5092e0e9f2326da582158fd22090f31547679160 and
the two commits mentioned above, that is
e90db11f4a1d983bc2bfe12bec30edbdbb200634 and
3ea4ea58ed80d6e517699bcab8810398f9ce5957 -
all three builds show the same failure.

i.e. The failure is not due to a change in Biopython
between those commits, but is in some way caused
by a change to the buildslave environment. My first
suggestion that this is due to Python 3.1.3 -> 3.1.5
remains my prime suspect.

I could try downgrading Python 3.1 on this machine
to confirm that I suppose... or updating Python 3.1 on
another machine?

The other recent Python 3.1 buildbot runs were both
using Python 3.1.2 (Windows XP 32bit and Linux 32 bit).

Can anyone else reproduce this, or have an idea what
the fix might be?

Regards,

Peter