[Biojava-dev] Biojava.util package?
David Felty
davfelty at gmail.com
Thu Mar 29 23:59:54 UTC 2012
>I'd suggest you step aside from the
>details of implementation. Think about what features your parser(s)
>must have and when how you are going to achieve them
Thank you for this! I now realize that I've been concentrating too
much on the implementation rather than the features. The
implementation will be important when (or if) I actually work on the
project during GSoC, but for now, I'll try to focus on features for my
proposal.
Unfortunately, I'm not very acquainted with the world of computational
biology, so I can't be sure what features would be most useful for the
file parsers. But here is what I've gathered so far from a combination
of already-existing code and people's responses:
- Simple api
- Robust
- Extensible
- Good performance
- Feature-rich
- Wide variety of parsers
- Proxy-fetching from remote databases (by ID or location)
- Local caching
- Auto-detection of data type
- Auto-detection of file format
- Lazy IO
- Random access file reading
Obviously, these are not all of equal importance, so I'll have to pick
out the most important ones for my proposal. But are there any
additional features anyone wants me to consider? Once again, I don't
have the same experience as many of you, so your input is very
helpful!
Thanks,
David
On Thu, Mar 29, 2012 at 5:49 PM, P. Troshin <to.petr at gmail.com> wrote:
> Hi David,
>
> Great to see such a discussion! You should see how important your work
> for Bio community is going to be.
>
> Now, what you need to do is to try taking into account what other
> people were suggesting and put it into your proposal. It's not going
> to be any good just to add a bunch of opinions; you need to come up
> with a coherent proposal. For this I'd suggest you step aside from the
> details of implementation. Think about what features your parser(s)
> must have and when how you are going to achieve them?
> I'd suggest that your parsers should be
> - easy to use (IMHO this is something BioJava 1 FASTA parser lacked)
> - robust
> - extensible
> - have good performance
> - most importantly, have sufficiently rich feature set so that we can
> replace other parsers (for the same format) in BioJava with yours.
>
> Do not forget to split your work in several achievable stages.
>
> I'd be careful about transferring the design from Python and
> especially a decade old Perl implementation straight to Java. While
> high level concerts may be the similar, implementation details should
> not be. It’s not that there is anything wrong with these parsers, it
> just that the languages are different. It is good to know how things
> are done elsewhere, but I'd suggest that for Java implementation you
> should be taking inspiration from some well know Java feature. For
> example, the Java Collections - a set of highly regarded tools for
> working with various collections of objects. Also do some reading on
> Java enums, your proposed implementation will definitely benefit from
> using them.
>
> Have fun,
>
> Regards,
> Peter
>
>
> On 29 March 2012 16:39, David Felty <davfelty at gmail.com> wrote:
>> Hey Andreas,
>>
>> It it wouldn't be too difficult to make a method that can infer the
>> file type using the file extension. In fact, it looks like BioPerl's
>> SeqIO does something like this. On the other hand, BioPython's SeqIO
>> takes the route of "explicit is better than implicit," and requires
>> that you explicitly give the format. Perhaps BioJava could take both
>> routes, and have an overloaded parse method that infers the file type,
>> along with the regular explicit method.
>>
>> As for non-fasta files, I implemented a couple of fasq parsers here:
>> http://pastebin.com/KLcpq8Qb
>> This would work similarly:
>>
>> InputStream is = ...
>> ProteinSequence seq = SeqIO.parse(is, SeqIO.FASTQ_SANGER, SeqIO.PROTEIN);
>>
>>
>> It looks like the other sequence readers aren't as clear-cut, so they
>> may need a bit more wrapping before they can be adapted to this
>> method. A common problem is that sequence readers don't return a
>> specific type of sequence, like with
>> org.biojava3.core.sequence.loader.UniprotProxySequenceReader, which
>> just contains the sequence data in itself. We might want to create
>> methods that convert the UniprotProxySequenceReader into sequences
>> that make more sense, like DNASequence and ProteinSequence.
>>
>> I'll look into this more later, I have to go to class.
>>
>> Regards,
>> David
>>
>> On Thu, Mar 29, 2012 at 10:39 AM, Andreas Prlic <andreas at sdsc.edu> wrote:
>>>
>>> Hi David,
>>>
>>> so far it still feels like a wrapper for what is already there. Try to
>>> take it to the next level. Why does the user still need to provide the
>>> type of file, can't this be auto-detected? What is the behaviour for
>>> non-fasta files, what can be supported and where are the limits, etc.
>>>
>>> Andreas
>>>
>>> On Thu, Mar 29, 2012 at 6:55 AM, David Felty <davfelty at gmail.com> wrote:
>>> > I've actually been working on something like this for my GSoC proposal,
>>> > here's what I came up with:
>>> >
>>> > public class SeqIO {
>>> > public static final int FASTA = 0;
>>> > public static final int FASTQ = 1;
>>> > public static final Class<DNASequence> DNA = DNASequence.class;
>>> > public static final Class<ProteinSequence> PROTEIN =
>>> > ProteinSequence.class;
>>> >
>>> > public static <S extends Sequence> Iterable<S> parse(InputStream is,
>>> > int fileFormat, Class<S> seqType) throws Exception {
>>> > switch (fileFormat) {
>>> > case FASTA:
>>> > if (seqType == DNA) {
>>> > return (Iterable<S>)
>>> > FastaReaderHelper.readFastaDNASequence(is);
>>> > } else if (seqType == PROTEIN) {
>>> > // etc...
>>> > }
>>> > break;
>>> > case FASTQ:
>>> > // etc...
>>> > }
>>> > }
>>> > }
>>> >
>>> > It would be used like so:
>>> >
>>> > InputStream is = ...
>>> > Iterable<DNASequence> seqs = SeqIO.parse(is, SeqIO.FASTA, SeqIO.DNA);
>>> > for (DNASequence s : seqs) {
>>> > // do something
>>> > }
>>> >
>>> > Obviously it's not the prettiest and a lot could be changed, but that's my
>>> > initial design. I tried to base it off BioPython's SeqIO, but static typing
>>> > and the variety of Sequence types forced me to put in some nasty generics.
>>> > Any tips would be appreciated!
>>> >
>>> > David
>>> >
>>> > On Thu, Mar 29, 2012 at 4:27 AM, Hannes Brandstätter-Müller <
>>> > biojava at hannes.oib.com> wrote:
>>> >
>>> >> Yes, something like a simplifying and unifying wrapper would be what I
>>> >> am thinking of.
>>> >>
>>> >> Hannes
>>> >>
>>> >> On Thu, Mar 29, 2012 at 05:55, Andreas Prlic <andreas at sdsc.edu> wrote:
>>> >> > Hi Hannes,
>>> >> >
>>> >> > I guess this is pretty similar to:
>>> >> >
>>> >> > http://biojava.org/wiki/BioJava:CookBook:Core:FastaReadWrite
>>> >> >
>>> >> > we have also been using "proxy" objects to fetch sequence data on the fly
>>> >> >
>>> >> > http://biojava.org/wiki/BioJava:CookBook:Core:Sequences
>>> >> >
>>> >> > As such I think we should discuss this a bit more. If we can find a
>>> >> > common api that is simple and works with both local files as well as
>>> >> > remote proxy objects, that would be nice. There should be no need to
>>> >> > change much of the existing code, but perhaps there could be a
>>> >> > simplified wrapper for what is already there.
>>> >> >
>>> >> > Andreas
>>> >> >
>>> >> > On Wed, Mar 28, 2012 at 12:04 PM, Hannes Brandstätter-Müller
>>> >> > <biojava at hannes.oib.com> wrote:
>>> >> >> Hi,
>>> >> >>
>>> >> >> I browsed around in the sister projects Biopython and Bioperl a bit,
>>> >> >> and noticed that many of the user interaction with the code goes
>>> >> >> through classes like SeqIO, SearchIO, AlignIO...
>>> >> >>
>>> >> >> So that got me thinking: how about we create similar "Interface"
>>> >> >> classes in Biojava?
>>> >> >>
>>> >> >> PROS:
>>> >> >>
>>> >> >> - easy change for programmers who switch languages
>>> >> >> - easy base interface that can be used in cookbook examples
>>> >> >> - makes code more readable if designed properly
>>> >> >> - easy access to features that are spread over the whole codebase but
>>> >> >> are connected anyway, like all file parsers
>>> >> >>
>>> >> >> CONS:
>>> >> >>
>>> >> >> - another thing to maintain
>>> >> >> - creates possible cross-dependencies (but if you don't want that,
>>> >> >> just use the existing classes directly)
>>> >> >>
>>> >> >>
>>> >> >> What are your thoughts?
>>> >> >>
>>> >> >> python from http://biopython.org/wiki/SeqIO:
>>> >> >>
>>> >> >> from Bio import SeqIO
>>> >> >> handle = open("example.fasta", "rU")
>>> >> >> for record in SeqIO.parse(handle, "fasta") :
>>> >> >> print record.id
>>> >> >> handle.close()
>>> >> >>
>>> >> >> possible equivalent in biojava (support for streaming API, Iterators,
>>> >> etc?):
>>> >> >>
>>> >> >> import org.biojava3.util.SeqIO;
>>> >> >>
>>> >> >> File file = new File("example.fasta");
>>> >> >> SeqIO seqIO = new SeqIO(file, SeqIO.FASTA);
>>> >> >> while (seqIO.hasNext()) {
>>> >> >> System.out.println(seqIO.next());
>>> >> >> }
>>> >> >> file.close();
>>> >> >>
>>> >> >> Hannes
>>> >> >> _______________________________________________
>>> >> >> biojava-dev mailing list
>>> >> >> biojava-dev at lists.open-bio.org
>>> >> >> http://lists.open-bio.org/mailman/listinfo/biojava-dev
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > -----------------------------------------------------------------------
>>> >> > Dr. Andreas Prlic
>>> >> > Senior Scientist, RCSB PDB Protein Data Bank
>>> >> > University of California, San Diego
>>> >> > (+1) 858.246.0526
>>> >> > -----------------------------------------------------------------------
>>> >>
>>> >> _______________________________________________
>>> >> biojava-dev mailing list
>>> >> biojava-dev at lists.open-bio.org
>>> >> http://lists.open-bio.org/mailman/listinfo/biojava-dev
>>> >>
>>> >
>>> > _______________________________________________
>>> > biojava-dev mailing list
>>> > biojava-dev at lists.open-bio.org
>>> > http://lists.open-bio.org/mailman/listinfo/biojava-dev
>>
>> _______________________________________________
>> biojava-dev mailing list
>> biojava-dev at lists.open-bio.org
>> http://lists.open-bio.org/mailman/listinfo/biojava-dev
More information about the biojava-dev
mailing list