[Biopython-dev] Need guidance.

Adam Kurkiewicz adam at kurkiewicz.pl
Thu Jul 6 20:25:03 UTC 2017 

I've understand now why running the test directly from Scripts didn't
work.

I can get the test cases to pass OK 90% of the time inside Tests by
running

`python test_testseq.py`

However, these test cases are still non-deterministic, in particular
test_codon_tables fails about 10% of the time. Is this intended
behaviour?

That's the stack trace:

picrin at lamport:~/programming/biopython/adil/Tests$ python
test_testseq.py 
test_alphabet (__main__.TestTestseq)
Testing 'alphabet' argument... ... ok
test_codon_tables (__main__.TestTestseq)
Testing codon sets... ... FAIL
test_gc_target (__main__.TestTestseq)
Testing 'gc_target' argument... ... ok
test_messenger (__main__.TestTestseq)
Testing 'messenger' argument... ... ok
test_seeding (__main__.TestTestseq)
Testing 'rand_seed' argument... ... ok
test_size (__main__.TestTestseq)
Testing 'size' and 'truncate' arguments... ... ok

======================================================================
FAIL: test_codon_tables (__main__.TestTestseq)
Testing codon sets...
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_testseq.py", line 87, in test_codon_tables
    self.assertFalse(seq[0] == "M")
AssertionError: True is not false

----------------------------------------------------------------------
Ran 6 tests in 0.008s

FAILED (failures=1)

Adam

On Thu, Jul 6, 2017, at 09:04 PM, Adam Kurkiewicz wrote:
> Another issue, perhaps more likely to be a real issue is that the test
> in line 127 fails about 50% of the time, and I've seen occasional
> failure in line 107.
> 
> Are the results of the tests non-deterministic? If so, would it be
> advisable to make them fail much less often, e.g. once per billion runs
> (this can usually be achieved by doing a probability calculation)?
> 
> That's the stack trace of the failing tests:
> 
> **********************************************************************
> File "testseq.py", line 107, in __main__.testseq
> Failed example:
>     my_seq2.alphabet
> Expected:
>     IUPACProtein()
> Got:
>     HasStopCodon(IUPACProtein(), '*')
> **********************************************************************
> File "testseq.py", line 127, in __main__.testseq
> Failed example:
>     -5 < error < 5
> Expected:
>     False
> Got:
>     True
> **********************************************************************
> 
> 
> Adam
> 
> On Thu, Jul 6, 2017, at 08:46 PM, Adam Kurkiewicz wrote:
> > Hi,
> > 
> > I've been just having a look at the pull request.
> > 
> > Please correct me if I'm wrong.
> > 
> > 1. At the moment one of the ways to run tests locally is to execute
> > `python Tests/run_tests.py` from the project's main directory.
> > 2. Adil's code will not be tested in this way.
> > 3. In order to test Adil's code one has to `run python
> > Scripts/testseq.py` from the main directory.
> > 
> > If that's the case, then it doesn't work on my machine (but I'm most
> > likely missing something):
> > 
> > picrin at lamport:~/programming/biopython/adil$ which python
> > /home/picrin/anaconda3/bin/python
> > picrin at lamport:~/programming/biopython/adil$ python --version
> > Python 3.6.0 :: Anaconda custom (64-bit)
> > picrin at lamport:~/programming/biopython/adil$ python Scripts/testseq.py 
> > Running doctests...
> > **********************************************************************
> > File "Scripts/testseq.py", line 65, in __main__.testseq
> > Failed example:
> >     from Scripts.testseq import testseq
> > Exception raised:
> >     Traceback (most recent call last):
> >       File "/home/picrin/anaconda3/lib/python3.6/doctest.py", line 1330,
> >       in __run
> >         compileflags, 1), test.globs)
> >       File "<doctest __main__.testseq[0]>", line 1, in <module>
> >         from Scripts.testseq import testseq
> >     ModuleNotFoundError: No module named 'Scripts.testseq'
> > **********************************************************************
> > 1 items had failures:
> >    1 of  41 in __main__.testseq
> > ***Test Failed*** 1 failures.
> > Done
> > 
> > 
> > Adam
> > 
> > On Tue, Jul 4, 2017, at 04:41 PM, Peter Cock wrote:
> > > Could someone else please take a look at Adil's pull request,
> > > particularly his doctest tweaks and import magic in order to
> > > test the code under Scripts/ rather than under Bio/ as usual:
> > > 
> > > https://github.com/biopython/biopython/pull/1306
> > > 
> > > (I'd just like the reassurance of a second opinion here, as
> > > the import stuff has a lot of subtleties.)
> > > 
> > > Many thanks,
> > > 
> > > Peter
> > > 
> > > 
> > > On Thu, Jun 15, 2017 at 10:33 AM, Peter Cock <p.j.a.cock at googlemail.com>
> > > wrote:
> > > > Hi Adil,
> > > >
> > > > You've certainly learnt lots about imports and doctests - and probably
> > > > know more than me now.
> > > >
> > > > I think you've made a sensible choice in restricting the special cases
> > > > to your new test_testseq.py while leaving run_tests.py and the
> > > > existing doctests as they are.
> > > >
> > > > Biopython is likely to keep supporting Python 2.7 until the year 2020, see:
> > > >
> > > > http://mailman.open-bio.org/pipermail/biopython-dev/2016-December/021613.html
> > > >
> > > > Peter
> > > >
> > > > On Thu, Jun 15, 2017 at 5:43 AM, Adil Iqbal <aiqbal85 at gmail.com> wrote:
> > > >> Hey, though I managed to solve the importing issue, in my conversation with
> > > >> Peter, I realized I made a rather large decision without consulting the
> > > >> community.
> > > >>
> > > >> In my current build, I am omitting all modules that are not part of the Bio
> > > >> package from executing doctests in version 2.7.
> > > >>
> > > >> My reason for doing that is copy-pasted below this message. Please read it
> > > >> for context.
> > > >>
> > > >> I realize that 'testseq' is the only module this change effects but is it
> > > >> something that we should extend to future modules?
> > > >>
> > > >> Best,
> > > >> Adil
> > > >>
> > > >> That method will certainly succeed in importing the ['testseq'] module. The
> > > >> issue is that once the importing is complete, the doctests are run. And the
> > > >> doctest also contain an import statement. The doctests cause an error in 2.7
> > > >> because they are subject to the same importing limitations as
> > > >> "run_tests.py". I will now describe these limitations below:
> > > >>
> > > >> Prior to 3.3, hierarchical importing (with "dot" notation) required a
> > > >> directory to be initialized with a "__init__.py" file. With the introduction
> > > >> of 3.3, hierarchical importing was generalized to all directories,
> > > >> regardless of initialization. This is why my commits were passing all tests
> > > >> except 2.7.
> > > >>
> > > >> Since we can't initialize folders carelessly, its best to remove the non-Bio
> > > >> modules from the "run_tests.py" file just prior to version 3.3. All versions
> > > >> of python after-and-including 3.3 don't require an extra "__init__.py" file,
> > > >> so the doctests will run fine on 3.3, 3.4, 3.5, and 3.6. That's why the most
> > > >> recent build passed the Travis CI test.
> > > >>
> > > >> I agree that my import methods were becoming overly-complicated. In the most
> > > >> recent build, I've eliminated the complicated code in favor of just deleting
> > > >> non-Bio files.The only complicated code left is limited to just the unit
> > > >> test ("test_testseq.py").
> > > >>
> > > >> Thankfully, the unit test for 'testseq' runs all of the same tests that are
> > > >> in the doctests, so we can be confident that 'testseq' is compatible with
> > > >> all versions of python. In the future, if "biopython" decides to stop
> > > >> supporting python 2.7, I would be happy to remove the last bit of
> > > >> complicated code.
> > > >>
> > > >> TL:DR - In 2.7 only: Modules outside 'Bio' package can be imported from
> > > >> "run_tests.py", however, their doctests will likely fail because they must
> > > >> also contain import statements. It's best to remove them from doctesting in
> > > >> earlier versions of python and allow the unit tests to confirm
> > > >> compatibility. (Only for non-Bio files in only 2.7).
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> _______________________________________________
> > > >> Biopython-dev mailing list
> > > >> Biopython-dev at mailman.open-bio.org
> > > >> http://mailman.open-bio.org/mailman/listinfo/biopython-dev
> > > _______________________________________________
> > > Biopython-dev mailing list
> > > Biopython-dev at mailman.open-bio.org
> > > http://mailman.open-bio.org/mailman/listinfo/biopython-dev

More information about the Biopython-dev mailing list