[Biopython-dev] Bug#751277: python-biopython: FTBFS on mips* powerpc s390x

[Peter Cock]

Hi Andreas, Jakub,

On Fri, Jun 13, 2014 at 1:55 PM, Andreas Tille <andreas at an3as.eu> wrote:
> Hi BioPython developers,
>
> I have updated the Debian package to version 1.64 (BTW, it is fine to
> ping debian-med at lists.debian.org about new upstream versions - we might
> become more quick in packaging new versions).

Good point. I used to email the Phillipe & Charles when there was
a change I anticipated might cause trouble.

Should we just email debian-med at lists.debian.org after each new
Biopython release? If so we can add that to our build process:
http://biopython.org/wiki/Building_a_release

> Thanks for adopting the patches we sended to you.
>

No problem - while some of your tweaks are Debian specific, where
it makes sense to incorporate fixes into Biopython we try to do that.

> Since the Debian package is built on different hardware architectures,
> we were facing different problems in the test suite.  Here you have an
> overview about all build logs:
>
>    https://buildd.debian.org/status/package.php?p=python-biopython&suite=unstable
>
> Jakub Wilk was kind enough to point to the real problems of the tests
> which can be read below (since the end of the build log does not say
> a lot).
>
> It would be great if you give some advise how to deal with these
> problems.

Most of these are tests where we call an external command line
tool (i.e. muscle, dnal, dialign2-2). The purpose of the tests is
in part to check our command line wrappers are current (and
catch any API changes), but also in many cases to check we
can parse the current output (to catch any format changes).

I would *guess* that some of these platforms have problems
in these underlying tools - or a very different version is being
tested? i.e. Older than the mainstream platforms.

--

The final category of failures was from test_Cluster.py under
powerpc and s390x, under Python 3.4, which suggests it
could be something in the C code for Bio.Cluster - probably
Python 3 specific.

>From line 138-141,

     clusterid, error, nfound = kcluster(data, nclusters=nclusters,
                                            mask=mask, weight=weight,
                                            transpose=0, npass=100,
                                            method='a', dist='e')

Line 210-212,

    distance = clusterdistance(data, mask=mask, weight=weight,
                                   index1=c1, index2=c2, dist='e',
                                   method='a', transpose=0)

Line 289-290,

        tree = treecluster(data=data1, mask=mask1, weight=weight1,
                           transpose=0, method='a', dist='e')

Line 555-557,

        clusterid, celldata = somcluster(data=data, mask=mask, weight=weight,
                                         transpose=0, nxgrid=10, nygrid=10,
                                         inittau=0.02, niter=100, dist='e')

This all give the following error via C function distance_converter
in Bio/Cluster/clustermodule.c

ValueError: distance should be a single character

Yet in all those examples, dist='e' which is a single character...

The good news is I can reproduce a related problem on Mac OS X
under Python 3.3 and 3.4 where this error is not raised:

Test branch:
https://github.com/peterjc/biopython/tree/cluster_single_char

This commit makes the error messages more explicit:
https://github.com/peterjc/biopython/commit/fa597040cfb7e5f18d55257367397e88274563b8

This commit verifies the errors are thrown (and they are not
under Python 3 on the Mac):
https://github.com/peterjc/biopython/commit/5b99854a82f08321ad78feaf0b362002d2d1fd2b

I'm going to have to pass this one to Michiel to look at... but it
looks like a glitch in the bytes vs unicode handling, which for
some reason mostly works - but breaks under some unusual
platforms?

Thanks,

Peter