[Biopython-dev] Including the online tests under TravisCI

[Peter Cock]

On Mon, Jun 20, 2016 at 5:20 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> Hello all,
>
> This is potentially contentious: With this commit I have
> included the online tests on one of the TravisCI test targets:
>
> https://github.com/biopython/biopython/commit/7b5ab8bd8c09cb725ec798afd2087d25189e2e5b
>
> Plus points: This will catch real mistakes introduced to the master
> branch or in pull requests much earlier. It will give more accurate
> test coverage reports on codecov.io
>
> https://codecov.io/gh/biopython/biopython

Long term this might encourage us to put more effort into
the reliability of the online tests (e.g. removing redundant
tests, making the examples as small and fast as possible),
which will reduce the end user queries we get from people
running the tests while installing Biopython from source.

> Minus points: This will put some load on the assorted online
> servers Biopython tests against (e.g. NCBI BLAST, Entrez,
> TogoWS, etc). This will occasionally fail due to temporary
> network issues and remote server problems. We may want
> to mark the online test as an allowed failure under TravisCI.
>
> I propose to continue the online tests for at least a week, in the
> believe they will be useful, but am willing to revert to only doing
> the offline tests with TravisCI.

Here's an example (not yet part of a pull request) from a branch
I'm currently working on which just failed in an online test:

https://travis-ci.org/peterjc/biopython/jobs/138942395

This test run timed out during test_NCBI_qblast.py - completely
unrelated to the changes on this branch, so a false positive.

Peter