[Biopython-dev] Including the online tests under TravisCI

[Peter Cock]

On Mon, Jun 20, 2016 at 5:57 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Mon, Jun 20, 2016 at 5:31 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> On Mon, Jun 20, 2016 at 5:20 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>>
>>> Minus points: This will put some load on the assorted online
>>> servers Biopython tests against (e.g. NCBI BLAST, Entrez,
>>> TogoWS, etc). This will occasionally fail due to temporary
>>> network issues and remote server problems. We may want
>>> to mark the online test as an allowed failure under TravisCI.
>>>
>>> I propose to continue the online tests for at least a week, in the
>>> believe they will be useful, but am willing to revert to only doing
>>> the offline tests with TravisCI.
>>
>> Here's an example (not yet part of a pull request) from a branch
>> I'm currently working on which just failed in an online test:
>>
>> https://travis-ci.org/peterjc/biopython/jobs/138942395
>>
>> This test run timed out during test_NCBI_qblast.py - completely
>> unrelated to the changes on this branch, so a false positive.
>
> Marking the online test environment as allowed to fail on Travis:
>
> https://github.com/biopython/biopython/commit/8cc8a7fb5d3cc266892599605a40663ad7be0d8d
> https://travis-ci.org/biopython/biopython/builds/138958258
>
> The reliability of the online tests will probably depend in part
> on the time (and day), but the NCBI online tests were completing
> much more quickly in my earlier TravisCI online testing.
>
> Peter

We have (temporarily) lost the online test as a side effect
from applying https://github.com/biopython/biopython/pull/877
which changed TravisCI to use Tox in order to add additional
checks for pep8 and building biopython.

Prior to that we were often getting online test failures, most
recently in test_motifs_online.py and test_SCOP_online
(timeout exceptions) or sometimes the entire test suite
timing out as the whole thing had taken too long.

Peter