[Bioperl-l] Memory in Perl

[Keith James]

>>>>> "Maido" == Maido Remm <mremm@ebc.ee> writes:

    Maido> 1. Reading more than 1 000 000 blast scores into hashes
    Maido> takes > 100 Mb memory. Thus ca 100 bytes per hash entry. Is
    Maido> there a way to make the work with large hashes more
    Maido> efficient?  Numerical 2D-array is not a good solution
    Maido> either, because the size of the full array would be
    Maido> enormous (50000 genes x 50000 genes for example), and most
    Maido> of the slots in array will stay empty and unused.

I'm only just starting to look into it myself, but maybe 2d arrays are
an option using PDL (Perl data language, http://pdl.perl.org)

    Maido> 2. Huge memory consumption wouldn't be a problem for our
    Maido> hardware - there is still plenty of RAM left, but perl
    Maido> reports "Out of Memory" after growing to about 125 Mb in
    Maido> size.

That sounds like a resource limit set on your machine. 'limit' should
tell you what these are set to - maybe they need increasing?

Keith

-- 

-= Keith James - kdj@sanger.ac.uk - http://www.sanger.ac.uk/Users/kdj =-
The Sanger Centre, Wellcome Trust Genome Campus, Hinxton, Cambs CB10 1SA