[Biojava-dev] Build system changes

[Michael Heuer]

On Wed, 21 Jan 2004, Thomas Down wrote:

> Once upon a time, Len Trigg wrote:
> > Thomas Down wrote:
> > >One worry, though, is that if you watch memory usage during
> > >an 'ant runtests', it creeps up pretty rapidly even when
> > >running tests which only use a few small objects.  I'm wondering
> > >how much of the problem is actually data, and how much is due
> > >to keeping lots of classes in memory.  BioJava is a pretty big
> >
> > I suppose that could account for a lot. How much memory does it take? As
> > a data point, our main src module here has about 10% more classes than
> > biojava, and runs around 4000 tests. We give the JVM 200MB when running
> > the tests. I think we had problems with 128MB. Our stuff can be very
> > memory intensive while running (text categorization, lots of documents
> > and classifiers etc), so I would say has a higher memory requirement
> > than biojava.
>
> Okay, I think I've got to the bottom of this now.  When running
> in fork="no" mode, the Ant JUnit task normally creates a new
> ClassLoader for every test.  Now, the core of BioJava is
> pretty interdependant, so significant parts of the core are
> needed by many other classes.  In practice, this means that
> a test run ends up loading about 50 copies of AlphabetManager
> and friends, so it's not surprising that it runs out of memory!

Speaking of AlphabetManager and friends, I built a very small library that
I've used for various things when I don't need all the complexity of the
biojava core.  Alphabets are static enumerations in the style of Bloch's
_Effective Java_ with symbols, e.g. DnaAlphabet.A, RnaAlphabet.G, and
ProteinAlphabet.ALA.  Then for symbol lists I just use java collections
classes (jakarta-commons [collections], jakarta-commons-sandbox [functor],
etc).

For whatever it's worth,

> http://shore.net/~heuermh/static-alphabet.tar.gz


ps. Sorry I haven't been around the list much lately, it seems that the
list daemon turned off sending messages to me for some reason.  I just
thought everyone was on holiday.  :)

   michael


> Ant 1.6.0 has added a new option which allows all the tests
> to be run in a single ClassLoader.  That solves most of the
> memory issues, and also makes the tests run faster (factor of >2).
>
> I've just checked in a build.xml patch which enables this mode
> by default when you do an "ant runtests".  I've added an extra
> target, "runtests-fork", which gives the old behaviour.  If, for
> some reason, you can't upgrade to ant 1.6, use "runtests-fork".
>
> (In principle, sharing a ClassLoader is a little bit risky,
> since one test could modify static data and affect the result
> of a later test.  However, we don't have many static variables
> in BioJava, so I don't think this is a serious risk.  Something
> to be aware of, though).
>
>     Thomas.
> _______________________________________________
> biojava-dev mailing list
> biojava-dev at biojava.org
> http://biojava.org/mailman/listinfo/biojava-dev
>