[BioPython] Re: UnitTests

[Andrew Dalke]

After rethinking Cayte's comments, I metaphorically slapped
myself cross-side my head.  I think I understand now her
points.

I had forgotten that when I've had QA support before, the
QA tests were different than the developer tests.  Our QA people
weren't coders, so weren't writing directly against the
libraries, but instead did functional tests against what the
user saw.  I had assumed that QA code testing directly against
APIs would be the same as the way I write against, but that's
a likely wrong assumption.

And as she said,
> We'll need all the testing we can get, however it's done.  

Combined with
> I like the [test] coding decisions to be as automatic as possible

> PyUnit only claims to be a unit testing tool.
and
> I have no problems with others using a different approach.

means (to me) it should be easy for both QA and coders to add test
codes, and if that isn't the same framework, so be it.  Working
test code is better than less test code and a single framework.
As I pointed out, it's even possible to wrap one unit test viewpoint
inside the other, so how about I help work on a way to get the
code I've got easily accessible to your UnitTest code?

At the least, I would like get a "run all tests" to have some sort
of judge on if anything broke.

Commenting on the rest of Cayte's email:
> Its something like having an HTML editor instead of having to
> insert html codes yourself, although both are functionally
> equivalent.  Even people who know html prefer editors.

That's probably my problem - I've never used an HTML editor.
Then again, my web pages suffer from blandness.  :(  :)

> On the XTreme web pages, a lot of posters feel you should be
> leery of comments.  They don't get maintained, ( still less
> in test code :) ) and the need for them may pinpoint a weakness
> in the code.  Also, I like having the output reporting automated.

I tend, as you can tell from the code, to have minimal comments.
They mostly describe what the code is about, and not details of
what it does.  I think that prevents some of the comment rot.

The comments I have in debug code are similar, and the output
comments are used as place markers as a way to trace back to where
the tests failed.

As to the syncronization of the two, well, I think my emails show
I tend to be one of these pedantic people who likes crossing T's
and dotting I's and J's.  (ABCD... And crossing X's too.  :)

> Also, if you have a suite of 40 tests, you don'twhich ones will
> break, even if you are a skilled coder.  Why run all 40.

Was that ".... you don't *know* which ones will break...." or
*care* or *need to know*?

If I know there are relevant tests, I'll run them for testing.
But I like running the full suite too, since I'm a defensive coder
who likes the computer telling me I didn't do something bad.  And
I mean defensive in the good sense.

There are, of course, tests which are only rarely run.  One of
my regression tests took almost three days to test - but it did
manage to uncover a rare threading bug in Python 1.5.1. :)  (The
test compared an NP-time subgraph comparison algorithm to a faster
but more imprecise genetic algorithm and verified that all solutions
were at least as good as the GA.  Because of the complexity of the
comparison, it was hard to come up with a minimal test set which
could be executed in only a couple of seconds.  One of the subtests
took 12 hours.)

Doesn't XP also suggest that tests point out which behaviour is
needed and which isn't, since if there are no tests for a behaviour,
it likely isn't needed?

Ach, I really should get that XP Explained book.  Trying to
read all the links from extremeprogramming.org/rules.html over my
modem gets to be too laborious, since I keep having to reconnect.
Or maybe I should mirror the sites - better wait 'til I can
leech off someone's T1.

Or maybe I shouldn't, since http://c2.com/cgi/wiki?UnitTestsDefined
says unit tests are:
> written by the same coders, in the same language, on the same
> platform, as the bugs, with support from the objects to be tested,
> and tuned to run automatically and with dependencies correctly
> built
which I've corrupted to say there can be two different styles
of unit testing, one of which does not include this definition.

Okay, I'll admit I know nearly squat about XP, then.

                    Andrew
                    dalke@acm.org