[Biojava-dev] ExternalProcess class

[Martin Szugat]

Hi Mark,

> More interesting stuff! BioJava already has two classes to launch external
> processes, ExecRunner and ProcessTools. From memory one is better than the
> other but I can't for the life of me remember which (can anyone help me
> out?). We should deprecate the lesser of the two.

That's interesting! I didn't find these classes because they are not
documented in the JavaDoc?!

> 
> Potentially yours is better than these two. Can you check out these two
> classes, if yours improves upon them then I'd be happy to check it in.

I don't know if it's better but different: You can provide your custom
(threaded) input and output handlers. I use it for parsing and writing BLAST
output/input on the fly.

I'll make a new package and send it you with the next mail.

Best regards

Martin

> 
> - Mark
> 
> 
> 
> 
> 
> "Martin Szugat" <Martin.Szugat at GMX.net>
> Sent by: biojava-dev-bounces at portal.open-bio.org
> 09/06/2005 07:55 PM
> 
> 
>         To:     <biojava-dev at biojava.org>
>         cc:     (bcc: Mark Schreiber/GP/Novartis)
>         Subject:        [Biojava-dev] ExternalProcess class
> 
> 
> Hi!
> 
> It's me again. I've implemented an ExternalProcess class. It encapsulates
> the necessity to run multi threaded input and output handlers when calling
> an external process using Runtime.exec().
> 
> The STDERR and STDOUT outputs as well as the STDIN input must be
> read/written in separate threads otherwise the calling application may
> hang
> up. A problem occurs when running an external program multiple times, e.g.
> running BLAST a thousand times. In this case for each iteration three
> threads are generated. Under Linux threads are implemented as processes,
> thus the process "java" is started three times for each iteration. However
> the thread objects are not terminated by the garbage collector and thus
> the
> threads/processes are not terminated. Even explicitly freeing the objects
> does not work (I've tested this several times). This results in an
> OutOfMemoryException after a few hundreds iterations, because the numbers
> of
> processes is limited under Linux.
> 
> I've solved this problem by using BioJava's SimpleThreadPool. Output
> reading
> and input writing is handled by Runnable input/output handlers, e.g. using
> a
> StreamPipe objects it is possible to redirect the STDOUT of an external
> process to the STDOUT of the calling process.
> 
> The usage of the ExternalProcess class is very simple, e.g. there are some
> simple static methods that encapsulate the internal complexity. The class
> also supports setting the environment variables or to inherit them from
> the
> parent process. In addition the working directory can be set or can be
> inherited. Finally there is a special feature: one can define variables
> for
> the command line arguments, e.g. "program -c %PARAMETER%"---%PARAMETER% is
> replaced by the value from a Properties object with the key "PARAMETER".
> 
> The classes and interfaces are fully documentated and there is a
> (repeated)
> Unit-Test for the ExternalProcess class. I've attached this test, the
> ExternalProcess class as well as the various handler classes.
> 
> I hope you'll find these classes useful and integrate them into BioJava.
> 
> Best regards
> 
> Martin
> 
> ____________________
> /   Martin Szugat    \
> / Author and Developer \
> +--------------------------------+---------------+
> |Phone: +49 (0)821 4206442       |Address:       |
> |Fax:   +49 (0)821 4206443       |               |
> |Mobil: +49 (0)179 7789714       |Zwerchgasse 6  |
> |Email: Martin.Szugat at GMX.net    |86150 Augsburg |
> |Web:   http://szugat.gmxhome.de |Germany        |
> +--------------------------------+---------------+
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> biojava-dev mailing list
> biojava-dev at biojava.org
> http://biojava.org/mailman/listinfo/biojava-dev
> 
> [ Attachment ''READERWRITERPIPE.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''STREAMPIPE.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''EXTERNALPROCESS.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''EXTERNALPROCESSTEST.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''OUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''INPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''SIMPLEOUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''READERINPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''SIMPLEINPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> [ Attachment ''WRITEROUTPUTHANDLER.JAVA'' removed by Mark Schreiber ]
> 
> 
> _______________________________________________
> biojava-dev mailing list
> biojava-dev at biojava.org
> http://biojava.org/mailman/listinfo/biojava-dev