[Bioperl-l] Parsing FASTA files into PrimarySeq objects

Chris Fields cjfields at illinois.edu
Wed Feb 17 22:03:39 UTC 2010


Rob, 

Looks like Jason took care of it.  There are likely more tests still
failing on trunk, but hopefully much fewer of them than before ;>

chris

On Wed, 2010-02-17 at 09:52 -0800, Robert Buels wrote:
> OK, well are you guys going to roll them back and do a branch?  Broken 
> trunk is not good.
> 
> R
> 
> Jason Stajich wrote:
> > hmm yes. Let's roll back those Florent.
> > 
> > I guess we need to also discuss the concept of branches again here for 
> > major changes like this. The speed fix is helpful for short read but 
> > probably not for all systems so just having the ability to provide the 
> > right type when you use your scripts focused on short-reads might be the 
> > best way to go forward and preserve backwards compatibility.
> > 
> > -jason
> > 
> > Chris Fields wrote:
> >> Yes, except that tests are now failing on live.  Bio::PrimarySeq isn't 
> >> a Bio::SeqI, so it's a bit of an API change in a lot of places 
> >> expecting a Bio::SeqI.  We should make this optional for the time 
> >> being, until we discuss it more on list.
> >>
> >> chris
> >>
> >> On Feb 16, 2010, at 10:57 AM, Jason Stajich wrote:
> >>
> >>   
> >>> Great!  thanks for committing it.
> >>>
> >>> Florent Angly wrote:
> >>>     
> >>>> Thanks for your reply Jason. It's very valuable since you coded the 
> >>>> modules in question!
> >>>>
> >>>> Now that I understand the purpose of these modules, I did a little 
> >>>> bit of documentation clarification. I also figured out that I should 
> >>>> implement the 'type' method in Bio::Seq::SeqFastaSpeedFactory and 
> >>>> made it support creating Bio::PrimarySeq. This is all committed to SVN.
> >>>>
> >>>> When sequences are large, you are right that it does not make much 
> >>>> difference in time or memory resource to use one sequence type over 
> >>>> the other. However, that's different for short sequences. I took a 
> >>>> FASTA file weighting 39.3 MB and containing 360,000 sequences of 106 
> >>>> bp on average.
> >>>>
> >>>> CASE 1:
> >>>>   Requested factory Bio::Seq::SeqFactory
> >>>>   Requested sequence type is Bio::Seq
> >>>>   Memory used: 329 MB
> >>>>   Time spent: 1m8.522s
> >>>>
> >>>> CASE 2:
> >>>>   Requested factory Bio::Seq::SeqFactory
> >>>>   Requested sequence type is Bio::PrimarySeq
> >>>>   Memory used: 246 MB
> >>>>   Time spent: 0m52.663s
> >>>>
> >>>> CASE 3:
> >>>>   Requested factory Bio::Seq::SeqFastaSpeedFactory
> >>>>   Requested sequence type is Bio::Seq
> >>>>   Memory used: 298 MB
> >>>>   Time spent: 0m31.292s
> >>>>
> >>>> CASE 4:
> >>>>   Requested factory Bio::Seq::SeqFastaSpeedFactory
> >>>>   Requested sequence type is Bio::PrimarySeq
> >>>>   Memory used: 231 MB
> >>>>   Time spent: 0m28.500s
> >>>>
> >>>> Clearly, when only simple sequence attributes are needed or FASTA 
> >>>> files are used, it's fast and memory efficient to use the 
> >>>> SeqFastaSpeedFactory with PrimarySeq.
> >>>>
> >>>> Florent
> >>>>
> >>>>
> >>>> On 16/02/10 10:03, Jason Stajich wrote:
> >>>>       
> >>>>> I don't think that aspect of the documentation vs interface was 
> >>>>> ever implemented - the interface object doesn't specify a type 
> >>>>> method or init argument even though the documentation says so. Not 
> >>>>> really sure why not, this was ages ago unfortunately.
> >>>>>
> >>>>> This particular factory definitely assumes you are building 
> >>>>> Bio::Seq objects - you can try and subclass and build your own to 
> >>>>> see if it makes much of a difference in speed/memory - I would 
> >>>>> posit it won't make a significant difference but be interested to 
> >>>>> see what you find.
> >>>>>
> >>>>> Just make your own Bio::Seq::PrimarySeqFactory object based on 
> >>>>> Bio::Seq::FastaSpeedFactory and simplify that code so it doesn't 
> >>>>> build Bio::Seq object wrapper around the Bio::PrimarySeq and do 
> >>>>> some perf tests so we'll know if it makes any difference here.
> >>>>>
> >>>>> -jason
> >>>>> Florent Angly wrote:
> >>>>>         
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I am trying to reduce memory usage and speedup reading FASTA files 
> >>>>>> using the facilities provided by BioPerl.
> >>>>>>
> >>>>>> The first thing I noticed is that when using Bio::SeqIO::fasta, 
> >>>>>> the objects returned are Bio::Seq, not Bio::PrimarySeq objects. 
> >>>>>> Bio::PrimarySeq sequences are lighter than Bio::Seq sequences, so 
> >>>>>> it's what I need. See code below:
> >>>>>>           
> >>>>>>> use warnings;
> >>>>>>> use strict;
> >>>>>>> use Data::Dumper;
> >>>>>>> use Bio::SeqIO;
> >>>>>>> my $in = Bio::SeqIO->new(-fh=>\*DATA);
> >>>>>>> my $seqfactory = $in->sequence_factory; # 
> >>>>>>> Bio::Factory::ObjectBuilderI
> >>>>>>> print "The factory is a ".ref($seqfactory)."\n";
> >>>>>>> $seqfactory->type('Bio::PrimarySeq'); # gives an error
> >>>>>>> my $seq = $in->next_seq;
> >>>>>>> print Dumper($seq);
> >>>>>>> __END__
> >>>>>>>             
> >>>>>>>> seq1 a small test sequence q
> >>>>>>>>                
> >>>>>>> ACGTACGACTACGACTAGCGCCATCAGC
> >>>>>>>              
> >>>>>> It returns:
> >>>>>>           
> >>>>>>> $VAR1 = bless( {
> >>>>>>>                  'primary_id' =>  'seq1',
> >>>>>>>                  'primary_seq' =>  bless( {
> >>>>>>>                                            'display_id' =>  'seq1',
> >>>>>>>                                            'primary_id' =>  'seq1',
> >>>>>>>                                            'desc' =>  'a small 
> >>>>>>> test sequence',
> >>>>>>>                                            'seq' =>  
> >>>>>>> 'ACGTACGACTACGACTAGCGCCATCAGC',
> >>>>>>>                                            'alphabet' =>  'dna'
> >>>>>>>                                          }, 'Bio::PrimarySeq' )
> >>>>>>>                }, 'Bio::Seq' );
> >>>>>>>              
> >>>>>> Actually, we have a Bio::Seq containing a Bio::PrimarySeq. I 
> >>>>>> really only need the Bio::PrimarySeq. Looking at the documentation 
> >>>>>> for Bio::SeqIO I found that I could in theory adjust the sequence 
> >>>>>> factory and sequence builder to my liking... I tried this:
> >>>>>>           
> >>>>>>> use warnings;
> >>>>>>> use strict;
> >>>>>>> use Data::Dumper;
> >>>>>>> use Bio::SeqIO;
> >>>>>>> my $in = Bio::SeqIO->new(-fh=>\*DATA);
> >>>>>>> my $seqfactory = $in->sequence_factory; # 
> >>>>>>> Bio::Factory::ObjectBuilderI
> >>>>>>> print "The factory is a ".ref($seqfactory)."\n";
> >>>>>>> $seqfactory->type('Bio::PrimarySeq'); # gives an error
> >>>>>>> my $seq = $in->next_seq;
> >>>>>>> print Dumper($seq);
> >>>>>>> __END__
> >>>>>>>             
> >>>>>>>> seq1 a small test sequence
> >>>>>>>>                
> >>>>>>> ACGTACGACTACGACTAGCGCCATCAGC
> >>>>>>>              
> >>>>>> This returns:
> >>>>>>           
> >>>>>>> The factory is a Bio::Seq::SeqFastaSpeedFactory
> >>>>>>> Can't locate object method "type" via package 
> >>>>>>> "Bio::Seq::SeqFastaSpeedFactory" at ./seqbuilder_test_3.pl line 
> >>>>>>> 12,<DATA>  line 1.
> >>>>>>>              
> >>>>>> According to Bio::Seq::FastaSpeedFactory's documentation:
> >>>>>>           
> >>>>>>> If you want the factory to create Bio::Seq objects instead
> >>>>>>> of the default Bio::PrimarySeq objects, use the -type parameter
> >>>>>>>              
> >>>>>> So, PrimarySeq should be the default type, even though in my case 
> >>>>>> it seems not to be. Second, I can't seem to use the -type method 
> >>>>>> to change what the return type is... It errors.
> >>>>>>
> >>>>>> Any ideas??? Thanks,
> >>>>>>
> >>>>>> Florent
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Bioperl-l mailing list
> >>>>>> Bioperl-l at lists.open-bio.org
> >>>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
> >>>>>>            
> >>> _______________________________________________
> >>> Bioperl-l mailing list
> >>> Bioperl-l at lists.open-bio.org
> >>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
> >>>      
> >>
> >>    
> > _______________________________________________
> > Bioperl-l mailing list
> > Bioperl-l at lists.open-bio.org
> > http://lists.open-bio.org/mailman/listinfo/bioperl-l
> > 
> 
> 





More information about the Bioperl-l mailing list