[Bioperl-l] Too many links at LoadHelper.pm

Sat Feb 6 14:27:53 UTC 2010

Ok, so I looked at my code, and it's a totally unrelated issue that 
caused files/filehandles to not be removed/closed properly.
The BioPerl tempdir() system seems to be doing fine :)
Florent

On 06/02/10 15:11, Dan Kortschak wrote:
> When I was working through those issues (I was having problems with
> Bio::DB::Sam causing tempfiles to be released before they were finished
> with) I paid quite close attention to having a clean /tmp after running
> tests, so I pretty confident that my modules are working with
> Bio::Root::IO properly and that Bio::Root::IO is working properly with
> them.
>
> Sorry I can't be more helpful.
> Dan
>
> On Fri, 2010-02-05 at 23:49 -0500, Mark A. Jensen wrote:
>    
>> Dan has been partial to temp in Root::IO (bowtie); I scurried off to File::Temp
>> (sam)--
>> any issues with tempfile handles, Dan?
>> ----- Original Message -----
>> From: "Chris Fields"<cjfields at illinois.edu>
>> To: "Florent Angly"<florent.angly at gmail.com>
>> Cc:<bioperl-l at lists.open-bio.org>
>> Sent: Friday, February 05, 2010 11:41 PM
>> Subject: Re: [Bioperl-l] Too many links at LoadHelper.pm
>>
>>
>>      
>>> The only place I can think of tempfiles being generated offhand are
>>> Bio::SeqFeature::Collection (actually, tied handles), but as you mention this
>>> may not be related.  There are calls to tempfile from
>>> Bio::Assembly::IO::sam/bowtie (one using Root::IO, one using File::Temp
>>> directly).
>>>
>>> chris
>>>
>>> On Feb 5, 2010, at 9:24 PM, Florent Angly wrote:
>>>
>>>        
>>>> It can't be, Chris, because I use the workaround mentioned here:
>>>> http://bugzilla.open-bio.org/show_bug.cgi?id=2577#c5
>>>>
>>>> It seems like temporary files keep accumulating assembly after assembly.
>>>> Looking at the lsof command, I get a lot of files described as "(deleted)". I
>>>> am thinking that maybe some filehandle is not closed or something.
>>>>
>>>> I'll investigate...
>>>>
>>>> Florent
>>>>
>>>> On 06/02/10 13:13, Chris Fields wrote:
>>>>          
>>>>> Florent,
>>>>>
>>>>> The Bio::Assembly error is documented in Bugzilla:
>>>>>
>>>>> http://bugzilla.open-bio.org/show_bug.cgi?id=2577
>>>>>
>>>>> Switching to a Bio::SeqFeature::CollectionI capable of distinguishing
>>>>> features by seq_id (such as Bio::DB::SeqFeature::Store) may be the best way
>>>>> to go.
>>>>>
>>>>> chris
>>>>>
>>>>> On Feb 5, 2010, at 9:06 PM, Florent Angly wrote:
>>>>>
>>>>>
>>>>>            
>>>>>> I've run into this issue very recently, albeit working on something
>>>>>> unrelated to loading GFF files: generating many small independent
>>>>>> assemblies. Maybe there's a bug in the tempdir() code, or its documentation
>>>>>> is not clear. I will take a look at it when I have some time.
>>>>>> Florent
>>>>>>
>>>>>>
>>>>>> On 05/02/10 22:27, Mark A. Jensen wrote:
>>>>>>
>>>>>>              
>>>>>>> The tempfiles are not being unlinked before there are too many for the OS
>>>>>>> to handle.
>>>>>>> Sounds like this may be your job, fungazid, if you're not employing the
>>>>>>> script (but
>>>>>>> not sure about that)
>>>>>>> ----- Original Message ----- From: "Fungazid"<fungazid at yahoo.com>
>>>>>>> To:<Bioperl-l at lists.open-bio.org>
>>>>>>> Sent: Thursday, February 04, 2010 9:52 PM
>>>>>>> Subject: [Bioperl-l] Too many links at LoadHelper.pm
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>>>> Bioperl hi,
>>>>>>>>
>>>>>>>> I have multiple gff3 files that look like:
>>>>>>>>
>>>>>>>> --------------------------------------------------------
>>>>>>>> ##gff-version 3
>>>>>>>> # file: temp1.gff3
>>>>>>>> contig453278    hs    contig    1    699    .    +    .
>>>>>>>> Name=contig453278
>>>>>>>> contig453278    hs    alignment    1    194    .    +    .
>>>>>>>> Name=hs.ENST00000258455
>>>>>>>> contig453278    hs    alignment    252    699    .    +    .
>>>>>>>> Name=hs.ENST00000258455
>>>>>>>> contig453278    hs    mi2    194    252    .    +    .    Name=mi2
>>>>>>>> contig453278    hs    transcript    104    183    .    +    .
>>>>>>>> Name=hs.ENST00000258455;ID=transcript-hs.ENST00000258455;Note=CD,4
>>>>>>>> contig453278    hs    transcript    1    9    .    +    .
>>>>>>>> Name=hs.ENST00000258455;ID=transcript-hs.ENST00000258455;Note=CD,1
>>>>>>>> contig453278    hs    transcript    73    103    .    +    .
>>>>>>>> Name=hs.ENST00000258455;ID=transcript-hs.ENST00000258455;Note=CD,3
>>>>>>>> contig453278    hs    transcript    10    72    .    +    .
>>>>>>>> Name=hs.ENST00000258455;ID=transcript-hs.ENST00000258455;Note=CD,2
>>>>>>>> ---------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I uploaded gff3 files to mysql and view the tracks in gBrowse like this:
>>>>>>>>
>>>>>>>> --------------------------------------------
>>>>>>>> Bio::DB::SeqFeature::Store->new(...)
>>>>>>>> $loader=Bio::DB::SeqFeature::Store::GFF3Loader->new(...)
>>>>>>>> for($i=0; $i<$#files+1; $i++){
>>>>>>>> $loader->load($file[$i]);
>>>>>>>> }
>>>>>>>> --------------------------------------------
>>>>>>>>
>>>>>>>> Eventually after uploading ~15000 small gff3 files I got the error:
>>>>>>>>
>>>>>>>> Error in tempdir() using /tmp/XXXXXXXXXX: Could not create directory
>>>>>>>> /tmp/9bbpdE5ibf: Too many links at
>>>>>>>> /usr/local/share/perl/5.8.8/Bio/DB/SeqFeature/Store/LoadHelper.pm line 49
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I am trying to see what went wrong, maybe you can rescue me,
>>>>>>>> Avi
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Bioperl-l mailing list
>>>>>>>> Bioperl-l at lists.open-bio.org
>>>>>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                  
>>>>>>> _______________________________________________
>>>>>>> Bioperl-l mailing list
>>>>>>> Bioperl-l at lists.open-bio.org
>>>>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>>>>>>
>>>>>>>                
>>>>>> _______________________________________________
>>>>>> Bioperl-l mailing list
>>>>>> Bioperl-l at lists.open-bio.org
>>>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>>>>>
>>>>>>              
>>>>>            
>>>> _______________________________________________
>>>> Bioperl-l mailing list
>>>> Bioperl-l at lists.open-bio.org
>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>>>          
>>>
>>> _______________________________________________
>>> Bioperl-l mailing list
>>> Bioperl-l at lists.open-bio.org
>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>>
>>>
>>>        
>
>    