[Biopython-dev] Online tools to track our test coverage & code quality

[Peter Cock]

On Thu, Nov 26, 2015 at 5:06 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
> On Thu, Nov 26, 2015 at 4:01 PM, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> Hi all,
>>
>> I've just been looking at https://codecov.io/ for logging unit test coverage,
>> which can be used an add on on top of the GitHub and TravisCI with just
>> a few small changes to the .travis.yml file ...
>
> I'd forgotten about another option in this area, https://coveralls.io
>

Tiago raised this on the list back in May,
http://lists.open-bio.org/pipermail/biopython-dev/2015-May/020934.html

We had a good discussion about what to do with the online tests and
those will many external dependencies - which otherwise get poor
test coverage scores in our current TravisCI configuration. On a
related point, I've been meaning to try the Mac OS X support under
TravisCI out.

I started comparing coveralls.io and codecov.io yesterday, both using
the same coverage data as input. It took a little tweaking to get them
to report about the same overall coverage score:

e.g. using my test branch and personal repository:

https://codecov.io/github/peterjc/biopython?branch=codecov
https://coveralls.io/github/peterjc/biopython?branch=codecov

It seems that for now codecov.io does a much better job of combining
all the TravisCI build targets to give overall coverage - while coveralls.io
seems to list each file once per build target.

My overall impression so far is a preference for the CodeCov.io
interface - it also seems to have a more responsive website.
We could of course setup both.

Peter