[Biojava-l] A Simple Genbank Parser Runs Out of Memory

[Schreiber, Mark]

Hi -

Most JVMs give you only 32 or 64M of space to operate in for large
genome sized GenBank files (or big chunks of chromosomes) this is not
sufficient. Typing java -X at the prompt will give you all the extended
options for your particular JVM. The JVM will complain if you try to
give it more memory than it can find but most will happily use disk
cache if it is available.

Mark


-----Original Message-----
From: Muratet, Michael [mailto:MMuratet@resgen.com] 
Sent: Saturday, 2 March 2002 11:59 a.m.
To: 'cantey.lg@pg.com'; biojava-l@biojava.org
Subject: RE: [Biojava-l] A Simple Genbank Parser Runs Out of Memory


Larry

I forget what the default memory allocation is for the jvm, but there is
an argument -Xmx that will allow you to increase the heap size. Maybe
that will fix the problem.

Mike

> -----Original Message-----
> From: cantey.lg@pg.com [mailto:cantey.lg@pg.com]
> Sent: Friday, March 01, 2002 2:26 PM
> To: biojava-l@biojava.org
> Subject: [Biojava-l] A Simple Genbank Parser Runs Out of Memory
> 
> 
> I created a simple parser based on the Genbank demo, to test
> a proof of concept.
> I ran it on a large Genbank source file ( contains over 
> 160,000 "sequences").
> The program  processed 34,170 sequences then crashed with the
> java.lang.OutOfMemoryError .   I ran this on an NT with 768mb 
> ram, using the
> 1.2.2 JVM.
> 
> The interesting thing about this is that I was able watch the
> size of the JVM.
> For the first 34,169 sequences it reached a steady state at 
> about 14 mb with
> normal expansion and contraction.  Then it processed a 
> sequence and the JVM size
> jumped up to 85 mb and crashed.  This scenario was exactly 
> reproducible.  To
> make sure it wasn't a data issue,  I took 5 sequences before 
> and after the
> "crashing" sequence and put them in a separate file.  I was 
> able to process this
> file with no interesting problems.
> 
> The net result was that I was able to "solve" this problem by
> pre-allocating a
> larger JVM.  However, I am concerned when I see an expansion 
> of the JVM by 70 mb
> when it does a simple parse of a sequence.
> 
> Are any of you aware of perhaps garbage collecting problems
> in jdk 1.2.2 ?
> 
> Any other ideas?
> 
> Thanks and Best Regards,
> Larry Cantey
> 
> 
> _______________________________________________
> Biojava-l mailing list  -  Biojava-l@biojava.org 
> http://biojava.org/mailman/listinfo/biojava-l
> 
_______________________________________________
Biojava-l mailing list  -  Biojava-l@biojava.org
http://biojava.org/mailman/listinfo/biojava-l
=======================================================================
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
=======================================================================