[Bioperl-l] Blast temporary open files not closed
ende
mmacho at gmail.com
Sun Sep 10 12:19:15 UTC 2006
El 10/09/2006, a las 5:59, Hilmar Lapp escribió:
> Your code snippet leaves plenty of opportunities for temp files
> accumulating, depending on what you do with the report object. So
> long as you use the report object the associated result file will
> remain present and open.
>
It sounds possible.. I have thought it previously... but I returns a
local copy of the result of $blastMachine->blastall(..); not the
$blastMachine at all. How it could be possible the original object
could get caught with that copy?
> You will have to provide the calling code as well if you want
> someone to look at the problem.
>
From the main object I call this DoBlast
########################
# #
# ANALIZE EACH SEQ #
# #
########################
#
for (my $i=0; $i < $self->{inSeqArrTotal}; ++$i) {
my $s = ${$self->{inSeqArr}}[$i];
my $ID = $s->id();
local *x = \$self->{SQ}{$ID}; # short for ref curr seq
$x->{seq} = $s;
# search for cloning vector seqs. in all the query seqs.
#
my $blastTemp = $self->DoBlastSeq($self->
{cloningVectorsFilename}, $s);
and then use this $blastTemp and save it in another place of the object:
$x->{blastcv} = $blastTemp;
...
> On Sep 9, 2006, at 7:50 PM, ende wrote:
>
>>
>> Thank all of you for your quick and precise answer!!!
>>
>> My bioperl version must be 1.20 :(
>>
>> I can read it in the file Bio::Perl.pm since
>>
>> perl -MCPAN -e shell
>> and then
>> i Bio::Perl
>>
>> does not inform me about any local version but says instead:
>>
>> trange distribution name [Bio:Perl]
>> Module id = Bio::Perl
>> CPAN_USERID BIRNEY (Ewan Birney <birney at sanger.ac.uk>)
>> CPAN_VERSION undef
>> CPAN_FILE B/BI/BIRNEY/bioperl-1.4.tar.gz
>> UPLOAD_DATE 2003-12-23
>> DSLIP_STATUS (,,,,)
>> MANPAGE Bio::Perl - Functional access to BioPerl for people
>> who don't know objects
>> INST_FILE /Library/Perl/5.8.6/Bio/Perl.pm
>> INST_VERSION undef
>>
>> I read 1.20 at top of /Library/5.8.6/Bio/Perl.pm file!
>>
>> After many (many) attempts of installing Bio::Perl via -MCPAN always
>> it ends with the same message (of course using force!!):
>>
>>
>> Failed 3/25 tests, 88.00% okay
>> t/WABA.......................ok
>> t/XEMBL_DB...................SOAP::Lite and/or XML::DOM not
>> installed. This means that Bio::DB::XEMBL module is not usable.
>> Skipping tests.
>> t/XEMBL_DB...................ok
>> Failed Test Stat Wstat Total Fail Failed List of Failed
>> ---------------------------------------------------------------------
>> ---
>> -------
>> t/BioFetch_DB.t 27 1 3.70% 8
>> t/EMBL_DB.t 15 3 20.00% 6 13-14
>> t/Ontology.t 255 65280 50 100 200.00% 1-50
>> t/TreeIO.t 41 1 2.44% 42
>> t/Variation_IO.t 25 3 12.00% 15 20 25
>> t/simpleGOparser.t 255 65280 98 196 200.00% 1-98
>> 121 subtests skipped.
>> Failed 6/179 test scripts, 96.65% okay. 154/8273 subtests failed,
>> 98.14% okay.
>> make: *** [test_dynamic] Error 2
>> /usr/bin/make test -- NOT OK
>> Running make install
>> make test had returned bad status, won't install without force
>> Failed during this command:
>> BIRNEY/bioperl-1.4.tar.gz : make_test NO
>>
>>
>> ..and I am using "force" (exactly:)
>>
>> cpan> force install B/BI/BIRNEY/bioperl-1.4.tar.gz
>>
>>
>> I have installed many other modules without no problems but this
>> seems to reject me. Finally I have dowloaded the bioperl
>> 1.5.1 .tar.gz and installed ignoring the many errors make test gave
>> me. sudo make install and wow!!! the bioperl version changed and
>>
>> lsof +p pidofperl | grep stat | wc -l
>>
>> again grows during the execution until reach 250 (!) and crashes.
>>
>> It was also imposible from CPAN shell to obtain the local bioperl
>> version (as was also impossible in a remote Linux installation, that
>> also said) INST_VERSION undef.
>>
>> But now the Bio/Perl.pm file heads:
>>
>> # Perl.pm,v 1.23.2.1 2005/10/09 15:16:18 jason Exp
>> #
>> # BioPerl module for Bio::Perl
>>
>>
>>
>>
>>
>> (I am using 10.4.7 on the MacOSX)
>>
>> the code launch blast all for each seq inside a perl object (use of
>> $blastMachine->io->_io_cleanup(); did not resolve the problem)
>>
>>> sub DoBlastSeq ($$$$) {
>>> my ($self, $db, $seq, $outFileName) = @_;
>>> my %params = (
>>> program => "blastn",
>>> outfile => $outFileName,
>>> database => "$self->{path_db}/$db",
>>> # q => "-5",
>>> G => 3, # si
>>> E => 3, # si
>>> F => "\"m D\"", # "mD", # si
>>> e => 700, # si
>>> Y => 1.75e12, # si
>>> best => 1,
>>> );
>>> if (!$params{outfile}) {
>>> delete $params{outfile};
>>> }
>>> my $outErrs = ($outFileName || "blastErrs.err"),
>>>
>>>
>>> my $blastMachine = Bio::Tools::Run::StandAloneBlast->new(%
>>> params);
>>>
>>> open(OLDSTDERR, ">&", \*STDERR) or die "Can't dup STDERR: $!";
>>> open(STDERR, ">", $outErrs ) or die "ERROR reopening STDERR:
>>> $!";
>>>
>>> print "Running Blast with id: ", $seq->id(), "\n" if $DEBUG;
>>>
>>> my $blastResult = $blastMachine->blastall($seq);
>>>
>>>
>>> close(STDERR);
>>> open(STDERR, ">&", \*OLDSTDERR) or die "Can't dup OLDSTDERR:
>>> $!";
>>> unlink ($outErrs) if (-z $outErrs);
>>>
>>> return $blastResult;
>>> }
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> El 09/09/2006, a las 15:45, Brian Osborne escribió:
>>
>>> Juan,
>>>
>>> I recall a bug like this was fixed a while back - what version
>>> Bioperl are
>>> you using? By the way, always give version numbers when reporting a
>>> bug, the
>>> answer "already fixed" is very common.
>>>
>>> Brian O.
>>>
>>>
>>> On 9/9/06 4:31 AM, "ende" <mmacho at gmail.com> wrote:
>>>
>>>>
>>>> Processing a fasta file with about 500 dna seqs.. my MacOSX (that
>>>> has the max number of opened files up to 512) crashes... You
>>>> need to
>>>> divide the problem in pieces or (in bash shell, with ulimit -n
>>>> 1024)
>>>> augment that max number of opened files.
>>>>
>>>> This has no sense for me since my perl program nor leave any open
>>>> file without its corresponding closing. On the other side, the
>>>> problem arises when the number of dnas grows _in one file_.
>>>>
>>>> In the code I run blast (StandAloneBlast... $blastMachine-
>>>> >blastall)
>>>> for each seq.
>>>>
>>>>
>>>> Then sniffing int the perl program stopped perl program I confirmed
>>>> my suspects. BioPerl (StandAloneBlast) does not closes temporary
>>>> opened files. Those files seems to be created to save seqs for
>>>> to be
>>>> then processed by blastall program... The output of lsof indicates
>>>> (as MacOSX System Monitor) that those files are left opened but not
>>>> there (!?)
>>>>
>>>> The output of lsof +p pidofperlprogram
>>>>
>>>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
>>>> perl 21977 juanfc cwd VDIR 14,7 238 6835993 /Users/
>>>> juanfc/Documents/programperl 21977 juanfc txt VREG 14,7
>>>> 19280 1589055 /usr/bin/perl
>>>> perl 21977 juanfc txt VREG 14,7 23476 1580272 /System/
>>>> Library/Perl/5.8.6/darwin-thread-multi-2level/auto/IO/IO.bundle
>>>> perl 21977 juanfc txt VREG 14,7 17772 1580263 /System/
>>>> Library/Perl/5.8.6/darwin-thread-multi-2level/auto/Fcntl/
>>>> Fcntl.bundle
>>>> perl 21977 juanfc txt VREG 14,7 114116 1580381 /System/
>>>> Library/Perl/5.8.6/darwin-thread-multi-2level/auto/POSIX/
>>>> POSIX.bundle
>>>> perl 21977 juanfc txt VREG 14,7 23684 1580265 /System/
>>>> Library/Perl/5.8.6/darwin-thread-multi-2level/auto/File/Glob/
>>>> Glob.bundle
>>>> perl 21977 juanfc txt VREG 14,7 1797788 6275687 /usr/lib/
>>>> dyld
>>>> perl 21977 juanfc txt VREG 14,7 4379472 6276030 /usr/lib/
>>>> libSystem.B.dylib
>>>> perl 21977 juanfc txt VREG 14,7 1086420 6276221 /System/
>>>> Library/Perl/5.8.6/darwin-thread-multi-2level/CORE/libperl.dylib
>>>> perl 21977 juanfc 0u VCHR 4,2 0t3748 63113092 /dev/
>>>> ttyp2
>>>> perl 21977 juanfc 1u VCHR 4,2 0t3748 63113092 /dev/
>>>> ttyp2
>>>> perl 21977 juanfc 2u VCHR 4,2 0t3748 63113092 /dev/
>>>> ttyp2
>>>> perl 21977 juanfc 3u VCHR 4,2 0t3748 63113092 /dev/
>>>> ttyp2
>>>> perl 21977 juanfc 4r VREG stat(/
>>>> ...
>>>> Please, help.
>>>>
>>>>
>>>> --
>>>> Juan Falgueras
>>>> Profesor del Depto. de Lenguajes y Ciencias de la Computación
>>>> Universidad de Málaga
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Bioperl-l mailing list
>>>> Bioperl-l at lists.open-bio.org
>>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>>
>>>
>>
>>
>>
>> ---- ende
>>
>>
>>
>>
>> _______________________________________________
>> Bioperl-l mailing list
>> Bioperl-l at lists.open-bio.org
>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>>
>
> --
> ===========================================================
> : Hilmar Lapp -:- Durham, NC -:- hlapp at gmx dot net :
> ===========================================================
>
>
>
>
>
---- ende
More information about the Bioperl-l
mailing list