[Bioperl-l] load_seqdatabase.pl memory requirements unusually large

[Florent Angly]

Out of the blue, I'm going to ask: could it be some memory leak due to a 
circular reference?
Florent

Hilmar Lapp wrote:
> Something seems to cause Perl to be crapping out. It it were a 
> programmatic exception you would see the message and the trace.
>
> Could you run these tests by themselves:
>
>     ./Build test --test-files t/11locuslink.t
>
> If that doesn't reveal the error, add a verbose=1 argument.
>
> Let us know what you find.
>
>     -hilmar
>
> On Feb 6, 2009, at 11:03 AM, Michael Muratet wrote:
>
>> Greetings
>>
>> I have use bioperl-db and load_seqdatabase.pl many times in the past 
>> and it's worked pretty much out of the box.
>>
>> I have been trying to load fasta files from hg18. For chr1, the 
>> virtual and resident memory quickly builds to 14 GB or so, then 
>> starts using up the 2GB swap until it's full and then the system 
>> hangs. The system is an 8-core Dell with 16GB of physical memory. 
>> chr1.fa is ~242MB. All of the disk storage is network mounted on an 
>> EMC system which (I am told) has a proprietary version of something 
>> that's NFS-like.
>>
>> I loaded chrM (~17kb) and load_seqdatabase grew to over 4 GB before 
>> it completed.
>>
>> I am using MySQL 5.0.51a-community, DBI 1.607, perl 5.85, bioperl 
>> 1.6.0, bioperl-db 1.006900.
>>
>> I have the innodb engine enabled in MySQL and the buffers and caches 
>> set for a 'large' system.
>>
>> I had some errors during the bioperl-db install:
>>
>> Test Summary Report
>> -------------------
>> t/01dbadaptor.t (Wstat: 256 Tests: 23 Failed: 1)
>>  Failed test:  23
>>  Non-zero exit status: 1
>> t/10ensembl.t   (Wstat: 65280 Tests: 5 Failed: 0)
>>  Non-zero exit status: 255
>>  Parse errors: Bad plan.  You planned 18 tests but ran 5.
>> t/11locuslink.t (Wstat: 65280 Tests: 7 Failed: 0)
>>  Non-zero exit status: 255
>>  Parse errors: Bad plan.  You planned 113 tests but ran 7.
>> t/15cluster.t   (Wstat: 65280 Tests: 7 Failed: 0)
>>  Non-zero exit status: 255
>>  Parse errors: Bad plan.  You planned 162 tests but ran 7.
>> Files=16, Tests=1205, 21 wallclock secs ( 0.51 usr  0.12 sys + 14.90 
>> cusr  2.69 csys = 18.22 CPU)
>> Result: FAIL
>> Failed 4/16 test programs. 1/1205 subtests failed.
>>
>> The most recent bioperl-db documentation says that a workable version 
>> may be possible after some errors and so I went ahead with the install.
>>
>> The mailing list archive has some discussion about throughput but 
>> nothing really about filling up memory.
>>
>> Can anyone offer any clues about what's going on or where to start 
>> looking?
>>
>> Thanks
>>
>> Mike
>> _______________________________________________
>> Bioperl-l mailing list
>> Bioperl-l at lists.open-bio.org
>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>