[Bioperl-l] Problems with Bioperl-ext package on WinVista?
Yee Man Chan
ymc at yahoo.com
Sun Aug 16 09:36:59 UTC 2009
Hi Chris
Thanks for your suggestions. I think it is indeed better to check sum to 1.0 using sprintf. I fixed this in the newly committed HMM.pm
I also fixed codes that will lead to warnings with use warnings.
So now the only problem left is that "monotonic increasing" error. For that part of the code, I was trying to perform an expectation maximization step. Theoretically, the expectation should monotonically increase in every step. But I suppose this is not necessarily true when double precision floating point numbers are involved. I don't know why I used a 1e-100 tolerance for this. Therefore I "fixed" it by using the same tolerance to terminate the maximization step (ie .000001). I suppose this "fix" will make it much more unlikely to throw exception with your 5.10.0 perl.
Can you give that a try again and see if it works now.
Thank you
Yee Man
--- On Sat, 8/15/09, Chris Fields <cjfields at illinois.edu> wrote:
> From: Chris Fields <cjfields at illinois.edu>
> Subject: Re: [Bioperl-l] Problems with Bioperl-ext package on WinVista?
> To: "Yee Man Chan" <ymc at yahoo.com>
> Cc: "Robert Buels" <rmb32 at cornell.edu>, "BioPerl List" <bioperl-l at lists.open-bio.org>
> Date: Saturday, August 15, 2009, 10:38 PM
> Yee,
>
> I took the liberty of making a few simple changes to
> Bio::Tools::HMM in svn to point out the problem and possible
> solutions. Feel free to revert these as needed.
>
> I'm seeing two errors, which appear randomly when running
> 'make test'. The first is easily fixable, the second,
> I'm not so sure. I'll let you make the decisions on
> both.
>
> 1) There is an assumption in the module that, when
> adding floating points, you will always get 1.0. You
> may run into problems: see 'perldoc -q long decimals'.
> Lines like this (two places in the module):
> ...
> if ($sum != 1.0) {
> $self->throw("Sum of
> probabilities for each state must be 1.0; got $sum\n");
> }
> ...
>
> won't work as expected (note I added a simple diagnostic,
> just print out the 'bad' sum). With perl 5.8.8, this
> appears to work fine, but this is what I get with perl 5.10
> (64-bit):
>
> pyrimidine1:HMM cjfields$ make test
> PERL_DL_NONLAZY=1 /opt/perl/bin/perl "-Iblib/lib"
> "-Iblib/arch" test.pl
> Baum-Welch Training
> ===================
> Initial Probability Array:
> 0.499978 0.500022
> Transition Probability Matrix:
> 0.499978 0.500022
> 0.499978 0.500022
> Emission Probability Matrix:
> 0.133333 0.143333
> 0.163333 0.123333
> 0.143333 0.293333
> 0.133333 0.143333
> 0.163333 0.123333
> 0.143333 0.293333
>
> Log Probability of sequence 1: -521.808
> Log Probability of sequence 2: -426.057
>
> Statistical Training
> ====================
> Initial Probability Array:
> 1 0
> Transition Probability Matrix:
>
> ------------- EXCEPTION -------------
> MSG: Sum of probabilities for each from-state must be 1.0;
> got 0.999999999999999976
>
> STACK Bio::Tools::HMM::transition_prob
> /Users/cjfields/bioperl/bioperl-live/Bio/Tools/HMM.pm:499
> STACK toplevel test.pl:82
> -------------------------------------
>
> make: *** [test_dynamic] Error 255
>
> I'm assuming this needs to simply be rounded up to
> 1.0. That could be accomplished with something like
> 'if (sprintf("%.2f", $sum) != 1.0) {...}'
>
> 2) The second error is a little stranger. I have been
> randomly getting this:
>
> pyrimidine1:HMM cjfields$ make test
> PERL_DL_NONLAZY=1 /opt/perl/bin/perl "-Iblib/lib"
> "-Iblib/arch" test.pl
> Baum-Welch Training
> ===================
> S should be monotonic increasing!
> make: *** [test_dynamic] Error 255
>
> When I add strict and warnings pragmas to Bio::Tools::HMM
> (with a little additional cleanup to get things running), I
> get an additional warning (arrow):
>
> pyrimidine1:HMM cjfields$ make test
> PERL_DL_NONLAZY=1 /opt/perl/bin/perl "-Iblib/lib"
> "-Iblib/arch" test.pl
> Argument "FL" isn't numeric in numeric lt (<) at
> /Users/cjfields/bioperl/bioperl-live/Bio/Tools/HMM.pm line
> 188. <----
> Baum-Welch Training
> ===================
> S should be monotonic increasing!
> make: *** [test_dynamic] Error 255
>
> So something is not being converted as expected.
>
> chris
>
> On Aug 15, 2009, at 11:32 PM, Yee Man Chan wrote:
>
> > When are you going to release 1.6? Maybe let me work
> on it before it releases. If it doesn't resolve the problem,
> then we can think about other alternatives.
> >
> > Also, please show me the latest errors you have for
> 5.10.0.
> >
> > Thanks
> > Yee Man
> >
> > --- On Sat, 8/15/09, Chris Fields <cjfields at illinois.edu>
> wrote:
> >
> >> From: Chris Fields <cjfields at illinois.edu>
> >> Subject: Re: [Bioperl-l] Problems with Bioperl-ext
> package on WinVista?
> >> To: "Yee Man Chan" <ymc at yahoo.com>
> >> Cc: "Robert Buels" <rmb32 at cornell.edu>,
> "BioPerl List" <bioperl-l at lists.open-bio.org>
> >> Date: Saturday, August 15, 2009, 7:05 PM
> >> I'm still seeing the same errors on
> >> Mac OS X for 64-bit perl 5.10.0. Mac OS X,
> native perl
> >> (v5.8.8) passes fine now (as well as perl 5.8.8
> on
> >> dev.open-bio.org).
> >>
> >> I'm wondering if this is a problem with my local
> perl
> >> build. I'm very tempted to push the
> HMM-related code
> >> into a separate distribution (bioperl-hmm) and
> make a CPAN
> >> release out of it so it gets wider testing via
> CPAN testers;
> >> it would just require a minimum bioperl 1.6
> installation for
> >> Bio::Tools::HMM and any related modules.
> Yee, would
> >> that be okay with you?
> >>
> >> chris
> >>
> >> On Aug 15, 2009, at 8:23 PM, Yee Man Chan wrote:
> >>
> >>>
> >>> I just committed HMM.xs and typemap to SVN.
> Can you
> >> test it to confirm it works in 64-bit machines?
> >>>
> >>> Thanks
> >>> Yee Man
> >>>
> >>> --- On Sat, 8/15/09, Chris Fields <cjfields at illinois.edu>
> >> wrote:
> >>>
> >>>> From: Chris Fields <cjfields at illinois.edu>
> >>>> Subject: Re: [Bioperl-l] Problems with
> Bioperl-ext
> >> package on WinVista?
> >>>> To: "Robert Buels" <rmb32 at cornell.edu>
> >>>> Cc: "Yee Man Chan" <ymc at yahoo.com>,
> >> "BioPerl List" <bioperl-l at lists.open-bio.org>
> >>>> Date: Saturday, August 15, 2009, 12:11 PM
> >>>> I'm not sure, but it makes more sense
> >>>> to commit these changes directly.
> Yee, need
> >> us to set
> >>>> you up with a commit bit? If so,
> fill out
> >> the
> >>>> information on this page:
> >>>>
> >>>> http://www.bioperl.org/wiki/SVN_Account_Request
> >>>>
> >>>> and forward it to support at open-bio.org.
> >>>> I'll sponsor you.
> >>>>
> >>>> chris
> >>>>
> >>>> On Aug 15, 2009, at 11:44 AM, Robert Buels
> wrote:
> >>>>
> >>>>> The usual procedure for developing
> code is to
> >> exchange
> >>>> code via commits to a version control
> >> system. Yee, do
> >>>> you know how to use Subversion? Does Yee
> need a
> >> commit bit?
> >>>>>
> >>>>> Rob
> >>>>>
> >>>>> Yee Man Chan wrote:
> >>>>>> Hi Chris
> >>>>>> I find
> that there is a
> >> memory
> >>>> access bug in my code. Attached is the
> fixed
> >> HMM.xs. This
> >>>> file together with the simpler typemap
> should fix
> >> all
> >>>> problems. (I hope..)
> >>>>>> Please let
> me know if it
> >> works
> >>>> for you.
> >>>>>> Sorry for the bug...
> >>>>>> Yee Man
> >>>>>> --- On Fri, 8/14/09, Chris Fields
> <cjfields at illinois.edu>
> >>>> wrote:
> >>>>>>> From: Chris Fields <cjfields at illinois.edu>
> >>>>>>> Subject: Re: [Bioperl-l]
> Problems
> >> with
> >>>> Bioperl-ext package on WinVista?
> >>>>>>> To: "Yee Man Chan" <ymc at yahoo.com>
> >>>>>>> Cc: "Robert Buels" <rmb32 at cornell.edu>,
> >>>> "Jonny Dalzell" <jdalzell03 at qub.ac.uk>,
> >>>> "BioPerl List" <Bioperl-l at lists.open-bio.org>
> >>>>>>> Date: Friday, August 14, 2009,
> 8:31
> >> AM
> >>>>>>> Yee Man,
> >>>>>>>
> >>>>>>> I tested this out locally
> (perl 5.8.8
> >> 32-bit,
> >>>> perl 5.10.0
> >>>>>>> 64-bit) and on
> dev.open-bio.org (which
> >> is perl
> >>>> 5.8.8,
> >>>>>>> appears to be 32-bit).
> The patch
> >> results
> >>>> in cleaning
> >>>>>>> up warnings for 5.10.0 but
> results in
> >> similar
> >>>> warnings for
> >>>>>>> 5.8.8 (linux or OS X).
> >>>>>>>
> >>>>>>> On OS X perl 5.8.8, this
> sometimes
> >> passes
> >>>> (note the first
> >>>>>>> attempt fails, the second
> succeeds),
> >> so it's
> >>>> not entirely a
> >>>>>>> 32-bit issue:
> >>>>>>>
> >>>>>>> http://gist.github.com/167860
> >>>>>>>
> >>>>>>> OS X and perl 5.10.0, this
> always
> >> fails as the
> >>>> previous
> >>>>>>> gist shows, but demonstrates
> similar
> >> behavior
> >>>> (multiple
> >>>>>>> attempts to test get
> different
> >> responses):
> >>>>>>>
> >>>>>>> http://gist.github.com/167542
> >>>>>>>
> >>>>>>> On linux, everything passes
> with or
> >> w/o the
> >>>> patched files
> >>>>>>> (patched files have warnings
> as
> >> indicated
> >>>> above):
> >>>>>>>
> >>>>>>> Specs for all three perl
> executables
> >> (they
> >>>> vary a bit):
> >>>>>>>
> >>>>>>> http://gist.github.com/167883
> >>>>>>>
> >>>>>>> chris
> >>>>>>>
> >>>>>>> On Aug 14, 2009, at 3:27 AM,
> Yee Man
> >> Chan
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Ah.. I find that the
> typemap can
> >> become as
> >>>> simple as
> >>>>>>> this
> >>>>>>>> =====================
> >>>>>>>> TYPEMAP
> >>>>>>>> HMM *
> T_PTROBJ
> >>>>>>>> =====================
> >>>>>>>>
> >>>>>>>> Then the generated HMM.c
> will have
> >> a
> >>>> function called
> >>>>>>> INT2PTR to do the pointer
> conversion.
> >> I
> >>>> believe this should
> >>>>>>> solve the warnings.
> >>>>>>>> Attached are the updated
> HMM.xs
> >> and
> >>>> typemap. Can
> >>>>>>> someone with a 64-bit machine
> give it
> >> a try?
> >>>>>>>> Thank you
> >>>>>>>> Yee Man
> >>>>>>>> --- On Thu, 8/13/09, Chris
> Fields
> >> <cjfields at illinois.edu>
> >>>>>>> wrote:
> >>>>>>>>> From: Chris Fields
> <cjfields at illinois.edu>
> >>>>>>>>> Subject: Re:
> [Bioperl-l]
> >> Problems with
> >>>> Bioperl-ext
> >>>>>>> package on WinVista?
> >>>>>>>>> To: "Yee Man Chan"
> <ymc at yahoo.com>
> >>>>>>>>> Cc: "Robert Buels"
> <rmb32 at cornell.edu>,
> >>>>>>> "Jonny Dalzell" <jdalzell03 at qub.ac.uk>,
> >>>>>>> "BioPerl List" <Bioperl-l at lists.open-bio.org>
> >>>>>>>>> Date: Thursday, August
> 13,
> >> 2009, 5:31
> >>>> PM
> >>>>>>>>> (just to point out to
> >> everyone, Yee
> >>>>>>>>> Man's contact
> information was
> >> in the
> >>>> POD)
> >>>>>>>>>
> >>>>>>>>> Yee Man,
> >>>>>>>>>
> >>>>>>>>> I have the output in
> the below
> >> link:
> >>>>>>>>>
> >>>>>>>>> http://gist.github.com/167542
> >>>>>>>>>
> >>>>>>>>> There are similar
> problems
> >> popping up
> >>>> on 32- and
> >>>>>>> 64-bit
> >>>>>>>>> perl 5.10.0, Mac OS X
> 10.5.
> >>>> Haven't had time
> >>>>>>> to debug
> >>>>>>>>> it unfortunately.
> >>>>>>>>>
> >>>>>>>>> I think we should
> seriously
> >> consider
> >>>> spinning this
> >>>>>>> code off
> >>>>>>>>> into it's own
> distribution
> >> for
> >>>> CPAN. It's
> >>>>>>>>> unfortunately
> bit-rotting away
> >> in
> >>>>>>> bioperl-ext. If you
> >>>>>>>>> want to continue
> supporting it
> >> I can
> >>>> help set that
> >>>>>>> up.
> >>>>>>>>> chris
> >>>>>>>>>
> >>>>>>>>> On Aug 13, 2009, at
> 6:58 PM,
> >> Yee Man
> >>>> Chan wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi
> >>>>>>>>>>
> >>>>>>>>>>
> So is
> >> this
> >>>> an HMM only
> >>>>>>> problem? Or does
> >>>>>>>>> it apply to other
> bioperl-ext
> >>>> modules?
> >>>>>>>>>>
> What
> >>>> exactly are the
> >>>>>>> compilation errors
> >>>>>>>>> for HMM? I believe my
> >> implementation
> >>>> is just a
> >>>>>>> simple one
> >>>>>>>>> based on Rabiner's
> paper.
> >>>>>>>>>> http://www.google.com/url?sa=t&source=web&ct=res&cd=1&url=http%3A%2F%2Fwww.cs.ubc.ca%2F~murphyk%2FBayes%2Frabiner.pdf&ei=QqiESvClHNCfkQXbh8mWBw&rct=j&q=rabiner+hmm&usg=AFQjCNHeXLhTHmuKUXKKCHYSs58TxVGfZg
> >>>>>>>>>>
> >>>>>>>>>>
> I
> >> don't
> >>>> think I did
> >>>>>>> anything fancy that
> >>>>>>>>> makes it machine
> dependent or
> >> non-ANSI
> >>>> C.
> >>>>>>>>>> Yee Man
> >>>>>>>>>>
> >>>>>>>>>> --- On Thu,
> 8/13/09, Chris
> >> Fields
> >>>> <cjfields at illinois.edu>
> >>>>>>>>> wrote:
> >>>>>>>>>>> From: Chris
> Fields
> >> <cjfields at illinois.edu>
> >>>>>>>>>>> Subject: Re:
> >> [Bioperl-l]
> >>>> Problems with
> >>>>>>> Bioperl-ext
> >>>>>>>>> package on WinVista?
> >>>>>>>>>>> To: "Robert
> Buels"
> >> <rmb32 at cornell.edu>
> >>>>>>>>>>> Cc: "Jonny
> Dalzell"
> >> <jdalzell03 at qub.ac.uk>,
> >>>>>>>>> "BioPerl List" <Bioperl-l at lists.open-bio.org>,
> >>>>>>>>> "Yee Man Chan" <ymc at yahoo.com>
> >>>>>>>>>>> Date:
> Thursday, August
> >> 13,
> >>>> 2009, 3:18 PM
> >>>>>>>>>>>
> >>>>>>>>>>> On Aug 13,
> 2009, at
> >> 4:37 PM,
> >>>> Robert Buels
> >>>>>>> wrote:
> >>>>>>>>>>>> Jonny
> Dalzell
> >> wrote:
> >>>>>>>>>>>>> Is it
> >> ridiculous of me
> >>>> to expect
> >>>>>>> ubuntu to
> >>>>>>>>> take
> >>>>>>>>>>> care of this
> for
> >> me? How
> >>>> do
> >>>>>>>>>>>>> I go
> about
> >> compiling
> >>>> the HMM?
> >>>>>>>>>>>> Yes.
> This is
> >> a very
> >>>> specialized
> >>>>>>> thing
> >>>>>>>>> that
> >>>>>>>>>>> you're doing,
> and
> >> Ubuntu does
> >>>> not have
> >>>>>>> the
> >>>>>>>>> resources to
> >>>>>>>>>>> package every
> single
> >> thing.
> >>>>>>>>>>>>
> Unfortunately, it
> >> looks
> >>>> like
> >>>>>>> bioperl-ext
> >>>>>>>>> package is
> >>>>>>>>>>> not
> installable under
> >> Ubuntu
> >>>> 9.04 anyway,
> >>>>>>> which is
> >>>>>>>>> what I'm
> >>>>>>>>>>> running.
> For
> >> others on
> >>>> this list,
> >>>>>>> if
> >>>>>>>>> somebody is
> >>>>>>>>>>> interested in
> doing
> >>>> maintaining it, I'd be
> >>>>>>> happy
> >>>>>>>>> to help out
> >>>>>>>>>>> by testing on
> >> Debian-based
> >>>> Linux
> >>>>>>> platforms.
> >>>>>>>>> We need to
> >>>>>>>>>>> clarify this
> >> package's
> >>>> maintenance status:
> >>>>>>> if
> >>>>>>>>> there is
> >>>>>>>>>>> nobody
> interested in
> >>>> maintaining it, I
> >>>>>>> would
> >>>>>>>>> recommend that
> >>>>>>>>>>> bioperl-ext be
> removed
> >> from
> >>>> distribution.
> >>>>>>>>> It's not in
> >>>>>>>>>>> anybody's
> interest to
> >> have
> >>>> unmaintained
> >>>>>>> software
> >>>>>>>>> out there
> >>>>>>>>>>> causing
> confusion.
> >>>>>>>>>>>
> >>>>>>>>>>> I have cc'd
> Yee Man
> >> Chan for
> >>>> this.
> >>>>>>> If there
> >>>>>>>>> isn't a
> >>>>>>>>>>> response or
> the
> >> message
> >>>> bounces, we do one
> >>>>>>> of two
> >>>>>>>>> things:
> >>>>>>>>>>> 1) consider
> it
> >> deprecated
> >>>> (probably
> >>>>>>> safest).
> >>>>>>>>>>> 2) spin it out
> into a
> >> separate
> >>>> module.
> >>>>>>>>>>>
> >>>>>>>>>>> Just tried to
> comile
> >> it myself
> >>>> and am
> >>>>>>> getting
> >>>>>>>>> errors (using
> >>>>>>>>>>> 64bit perl
> 5.10), so I
> >> think,
> >>>> unless
> >>>>>>> someone wants
> >>>>>>>>> to take
> >>>>>>>>>>> this on,
> option #1 is
> >> best.
> >>>>>>>>>>>
> >>>>>>>>>>>> So Jonny,
> in
> >> short, I
> >>>> would say "do
> >>>>>>> not use
> >>>>>>>>>>> bioperl-ext".
> >>>>>>>>>>>
> >>>>>>>>>>> In general,
> that's a
> >> safe
> >>>> bet. We're
> >>>>>>> moving
> >>>>>>>>> most of
> >>>>>>>>>>> our C/C++
> bindings to
> >> BioLib.
> >>>>>>>>>>>
> >>>>>>>>>>>> Step
> back.
> >> What are
> >>>> you trying
> >>>>>>> to
> >>>>>>>>>>> accomplish?
> >> Chris
> >>>> already
> >>>>>>> recommended some
> >>>>>>>>> alternative
> >>>>>>>>>>> methods in his
> email
> >> of 8/11
> >>>> on this
> >>>>>>>>> subject.
> Perhaps
> >>>>>>>>>>> we can guide
> you to
> >> some
> >>>> software that is
> >>>>>>>>> actively
> >>>>>>>>>>> maintained and
> will
> >> meet your
> >>>> needs.
> >>>>>>>>>>>> Rob
> >>>>>>>>>>> Exactly.
> Lots of
> >> other
> >>>> (better
> >>>>>>> supported!)
> >>>>>>>>> options
> >>>>>>>>>>> out there.
> >> HMMER, SeqAn,
> >>>> and
> >>>>>>> others.
> >>>>>>>>>>> chris
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >>
> __________________________________________________
> >>>>>>>> Do You Yahoo!?
> >>>>>>>> Tired of spam?
> Yahoo! Mail
> >> has the
> >>>> best spam
> >>>>>>> protection around
> >>>>>>>> http://mail.yahoo.com
> >>>>>>>
> >>>>
> >>
> <HMM.xs><typemap>_______________________________________________
> >>>>>>>> Bioperl-l mailing list
> >>>>>>>> Bioperl-l at lists.open-bio.org
> >>>>>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --Robert Buels
> >>>>> Bioinformatics Analyst, Sol Genomics
> Network
> >>>>> Boyce Thompson Institute for Plant
> Research
> >>>>> Tower Rd
> >>>>> Ithaca, NY 14853
> >>>>> Tel: 503-889-8539
> >>>>> rmb32 at cornell.edu
> >>>>> http://www.sgn.cornell.edu
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>
> >>
> >
> >
> >
>
>
More information about the Bioperl-l
mailing list