[Biopython-dev] Rethinking Biopython's testing framework

[Giovanni Marco Dall'Olio]

On Mon, Nov 24, 2008 at 7:44 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
> Hi everybody,
>
> Biopython's testing framework is built on top of Python's unit testing framewerk. Python's unit testing framework makes use of assertion statements to compare the result of a command to the expected result.

Hi,
I was also proposing to use the doctest framework for some of the
modules, and for enhancing documentation.

- http://bugzilla.open-bio.org/show_bug.cgi?id=2640


> Biopython uses test scripts that print output to stdout, together with an output file that contains the
> correct output. After running each test script, it compares the generated output with the correct
> output to see if the test was successful.
>
> This approach can be useful for modules that deal with different file formats. For example, you can read in a file in one format, write it out in a different format, and compare it with the expected result.
>
> However, more than half of Biopython's tests do not actually make use of this testing framework:
>

Do you need help in re-organizing all of these modules?

> test_BioSQL
> test_CAPS
> test_Cluster
> test_CodonTable
> test_Compass
> test_Crystal
> test_DocSQL
> test_EmbossPrimer
> test_Entrez
> test_Fasta
> test_GACrossover
> test_GAMutation
> test_GAOrganism
> test_GAQueens
> test_GARepair
> test_GASelection
> test_GFF
> test_GFF2
> test_GraphicsChromosome
> test_GraphicsDistribution
> test_GraphicsGeneral
> test_HMMCasino
> test_HMMGeneral
> test_HotRand
> test_KDTree
> test_KeyWList
> test_LogisticRegression
> test_Medline
> test_NNExclusiveOr
> test_NNGene
> test_NNGeneral
> test_Pathway
> test_PopGen_FDist
> test_PopGen_FDist_nodepend
> test_PopGen_SimCoal
> test_PopGen_SimCoal_nodepend
> test_Registry
> test_Restriction
> test_SCOP_Astral
> test_SCOP_Cla
> test_SCOP_Des
> test_SCOP_Dom
> test_SCOP_Hie
> test_SCOP_Raf
> test_SCOP_Residues
> test_SCOP_Scop
> test_Wise
> test_docstrings
> test_kNN
> test_lowess
> test_psw
>
> These tests have trivial output, for example test_Cluster:
>
> test_Cluster
> test_clusterdistance (test_Cluster.TestCluster) ... ok
> test_distancematrix_kmedoids (test_Cluster.TestCluster) ... ok
> test_kcluster (test_Cluster.TestCluster) ... ok
> test_matrix_parse (test_Cluster.TestCluster) ... ok
> test_median_mean (test_Cluster.TestCluster) ... ok
> test_somcluster (test_Cluster.TestCluster) ... ok
> test_treecluster (test_Cluster.TestCluster) ... ok
>
> ----------------------------------------------------------------------
> Ran 7 tests in 0.015s
>
> OK
>
> I suspect that for many of the remaining tests Biopython's unit testing framework doesn't bring any real advantage, but is used anyway solely because it currently is the standard in Biopython.
>
> Personally, I find Python's unit testing framework easier to understand than Biopython's testing framework. It doesn't need a separate output file, and it is easier to match each line of code with the correct behavior.
>
> I would therefore like to suggest to move from Biopython's testing framework to Python's testing framework. This also relieves us of the task of explaining Biopython's testing framework to contributors, and allows us to make better use of what Python already provides. Comparing output line-by-line, as Biopython's testing framework currently does, can still be used by test scripts that need this functionality.
>
> Comments, suggestions, anybody?
>
> --Michiel.
>
>
>
>
>
>
> _______________________________________________
> Biopython-dev mailing list
> Biopython-dev at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/biopython-dev
>



-- 
-----------------------------------------------------------

My Blog on Bioinformatics (italian): http://bioinfoblog.it