[Biopython-dev] Biopython under PyPy

[Eric Talevich]

On Fri, Sep 16, 2011 at 6:56 PM, Peter Cock <p.j.a.cock at googlemail.com>wrote:

> On Fri, Sep 16, 2011 at 9:14 PM, Eric Talevich <eric.talevich at gmail.com>
> wrote:
> > On Fri, Sep 16, 2011 at 1:07 PM, Peter Cock <p.j.a.cock at googlemail.com>
> > wrote:
> >>
> >> Hi all,
> >>
> >> I've been trying Biopython under PyPy 1.6, and the unit tests for
> >> a lot of things work fine. In the short term I'm skipping all the C
> >> extensions (not clear how easy they will be under PyPy):
> >>
> >>
> https://github.com/biopython/biopython/commit/2a26ceebed01508a69aefd6a3a6437245347a5a2
> >>
> >
> > Neato! Here's the relevant bug in Redmine:
> > https://redmine.open-bio.org/issues/3236
>
> Oh yeah - what I did to setup.py is almost the same.
>
> >>
> >> PyPy ships with a minimal numpy implementation, but it seems
> >> to be very minimal - e.g. there is no dot function. This is actually
> >> a bit annoying as "import numpy" works but you don't get everything!
> >> Anyway, there are some easy checks we can add to individual
> >> unit tests to skip them under pypy.
> >
> > Presumably this will get better in future releases of numpy,
> > but yeah, it will be awkward to have to check that the numpy
> > module not only exists, but is in fact the 'real' numpy.
>
> I'm hoping we just need to check if it is good enough,
> i.e. has the bits of numpy required for that module.
> That's my aim with the test_*.py changes.
>
> >> What is interesting is running the full test suite reports some
> >> false positives (tests which when run on their own, or as part
> >> of a smaller group pass), and the test suite itself never finishes:
> >> error: Too many open files
> >>
> >> I'm not sure what this is from... I fixed an obvious handle leak:
> >>
> >>
> https://github.com/biopython/biopython/commit/f7ce81b3751745970c32cc813836507e93da3c30
> >>
> >> I suspect the problem is some of the individual tests are
> >> leaking handles - which we know already from warnings
> >> under Python 3 etc.
> >
> > Now that we've ditched Py2.4, we can start using context managers
> ('with')
> > instead of explicit open/close. This should help ensure handles are
> closed
> > when exceptions are raised.
>
> Yeah - in the example above can you put the with statement
> inside an if?
>


In runTest it's a little strange because the open() call depends on the
Python version. So we could do:

if sys.version_info[0] >= 3:
    #Python 3 problem: Can't use utf8 on output/test_geo
    #due to micro (\xb5) and degrees (\xb0) symbols
    open_kwargs = {'encoding': 'latin'}
else:
    open_kwargs = {'mode': 'rU'}
with open(outputfile, **kwargs) as expected:
    # Everything else...


But then we lose the try/except block that catches the missing-file error.
The cleanest solution would be a separate context handler:

@contextlib.contextmanager
def open_outputfile(fname):
    try:
        if sys.version_info[0] >= 3:
            #Python 3 problem: Can't use utf8 on output/test_geo
            #due to micro (\xb5) and degrees (\xb0) symbols
            expected = open(outputfile, encoding="latin")
        else:
            expected = open(outputfile, 'rU')
        yield expected
    except IOError:
        self.fail("Warning: Can't open %s for test %s" % (outputfile,
self.name))
    finally:
        expected.close()


I think that would do everything we want.


-Eric