[Biojava-dev] ExternalProcess class

[Martin Szugat]

Hi!

It's me again. I've implemented an ExternalProcess class. It encapsulates
the necessity to run multi threaded input and output handlers when calling
an external process using Runtime.exec().

The STDERR and STDOUT outputs as well as the STDIN input must be
read/written in separate threads otherwise the calling application may hang
up. A problem occurs when running an external program multiple times, e.g.
running BLAST a thousand times. In this case for each iteration three
threads are generated. Under Linux threads are implemented as processes,
thus the process "java" is started three times for each iteration. However
the thread objects are not terminated by the garbage collector and thus the
threads/processes are not terminated. Even explicitly freeing the objects
does not work (I've tested this several times). This results in an
OutOfMemoryException after a few hundreds iterations, because the numbers of
processes is limited under Linux.

I've solved this problem by using BioJava's SimpleThreadPool. Output reading
and input writing is handled by Runnable input/output handlers, e.g. using a
StreamPipe objects it is possible to redirect the STDOUT of an external
process to the STDOUT of the calling process.

The usage of the ExternalProcess class is very simple, e.g. there are some
simple static methods that encapsulate the internal complexity. The class
also supports setting the environment variables or to inherit them from the
parent process. In addition the working directory can be set or can be
inherited. Finally there is a special feature: one can define variables for
the command line arguments, e.g. "program -c %PARAMETER%"---%PARAMETER% is
replaced by the value from a Properties object with the key "PARAMETER".

The classes and interfaces are fully documentated and there is a (repeated)
Unit-Test for the ExternalProcess class. I've attached this test, the
ExternalProcess class as well as the various handler classes.

I hope you'll find these classes useful and integrate them into BioJava.

Best regards

Martin

  ____________________
 /   Martin Szugat    \
/ Author and Developer \
+--------------------------------+---------------+
|Phone: +49 (0)821 4206442       |Address:       |
|Fax:   +49 (0)821 4206443       |               |
|Mobil: +49 (0)179 7789714       |Zwerchgasse 6  |
|Email: Martin.Szugat at GMX.net    |86150 Augsburg |
|Web:   http://szugat.gmxhome.de |Germany        |
+--------------------------------+---------------+

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ReaderWriterPipe.java
Type: text/java
Size: 4012 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/ReaderWriterPipe-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: StreamPipe.java
Type: text/java
Size: 4158 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/StreamPipe-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ExternalProcess.java
Type: text/java
Size: 19109 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/ExternalProcess-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ExternalProcessTest.java
Type: text/java
Size: 4426 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/ExternalProcessTest-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OutputHandler.java
Type: text/java
Size: 1674 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/OutputHandler-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: InputHandler.java
Type: text/java
Size: 1582 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/InputHandler-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SimpleOutputHandler.java
Type: text/java
Size: 1551 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/SimpleOutputHandler-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ReaderInputHandler.java
Type: text/java
Size: 2846 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/ReaderInputHandler-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SimpleInputHandler.java
Type: text/java
Size: 2250 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/SimpleInputHandler-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WriterOutputHandler.java
Type: text/java
Size: 2085 bytes
Desc: not available
Url : http://portal.open-bio.org/pipermail/biojava-dev/attachments/20050906/654b4746/WriterOutputHandler-0001.bin