[Bioperl-l] New Perl ORM for graph db Neo4j

Tue Sep 25 17:07:25 UTC 2012

Hi Joseph, 
Yes, it does its own chunking. You can control with 
$REST::Neo4p::JOB_CHUNK 
which defaults to 1024. That is a completely untested number, so any info is welcomed. The running by chunk should be transparent to the user, so if it borks let me know that too--
cheers MAJ

>-----Original Message-----
>From: Joseph Guhlin [mailto:guhli007 at umn.edu]
>Sent: Tuesday, September 25, 2012 10:53 AM
>To: 'Mark A. Jensen'
>Cc: 'Hilmar Lapp', 'Fields, Christopher J', <bioperl-l at lists.open-bio.org>
>Subject: Re: [Bioperl-l] New Perl ORM for graph db Neo4j
>
>Thanks, I'll try to run some tests on this this week. Is there a limit to
>the number of things you can do? Should I stagger at 10k or 100k or
>somewhere or do you know, or will it handle it itself?
>
>Best,
>--Joseph
>
>On Sun, Sep 23, 2012 at 4:46 PM, Mark A. Jensen <maj at fortinbras.us> wrote:
>
>> All,
>> I have added support for batch processing to REST::Neo4p, please have a
>> look at https://metacpan.org/module/REST::Neo4p::Batch.
>>
>> To use it, you write in the context of the object framework, but surround
>> your code with a "batch" block; e.g.:
>>
>> #!perl
>> # loader...
>> use REST::Neo4p;
>> use REST::Neo4p::Batch;
>>
>> open $f, shift() or die $!;
>> batch {
>>   while (<$f>) {
>>    chomp;
>>    ($name, $value) = split /\t/;
>>    REST::Neo4p::Node->new({name => $name, value => $value});
>>   } 'discard_objs';
>> exit(0);
>>
>> The code takes care of packaging the contained individual API calls and
>> sending them off to the server in neo4j's batch format.
>>
>> Please send me bugs on RT (
>> https://rt.cpan.org/Public/Bug/Report.html?Queue=REST-Neo4p)
>>
>> cheers MAJ
>>
>> >-----Original Message-----
>> >From: Mark A. Jensen [mailto:maj at fortinbras.us]
>> >Sent: Monday, September 17, 2012 03:02 PM
>> >To: 'Joseph Guhlin', 'Hilmar Lapp'
>> >Cc: 'Fields, Christopher J', <bioperl-l at lists.open-bio.org>, 'Mark A.
>> Jensen'
>> >Subject: Re: [Bioperl-l] New Perl ORM for graph db Neo4j
>> >
>> >W00t! Glad to see there is scope for this; now I can say I did one useful
>> thing for BP in two years!
>> >
>> >There is batch support (as well as traversals) in the REST api; looks
>> like the next two features I should add.
>> >
>> >cheers all MAJ
>> >
>> >Batch is an interesting problem,
>> >>-----Original Message-----
>> >>From: Joseph Guhlin [mailto:guhli007 at umn.edu]
>> >>Sent: Monday, September 17, 2012 02:21 PM
>> >>To: 'Hilmar Lapp'
>> >>Cc: 'Fields, Christopher J', <bioperl-l at lists.open-bio.org>, 'Mark A.
>> Jensen'
>> >>Subject: Re: [Bioperl-l] New Perl ORM for graph db Neo4j
>> >>
>> >>Another project using Neo4j is Bio4j. The database is generated using
>> java
>> >>but can be queried through the REST interface with CYPHER.
>> >>
>> >>I haven't had a chance to build it yet or look at it, but may do so in
>> the
>> >>near future.
>> >>
>> >>I am using Neo4j for my own project and the only thing generic enough I
>> >>could contribute with (that also overlaps with another module) is The
>> Gene
>> >>Ontology. I import it (~60k node, ~600k relationships, but I ignore a few
>> >>that others may consider key) and could clean up and contribute that
>> code.
>> >>I do use a bulk import script written in java that uses the native
>> >>libraries to process everything so would have to make it work for this
>> >>module directly (or keep the java import, as it is open source).
>> >>
>> >>--Joseph
>> >>
>> >>On Mon, Sep 17, 2012 at 12:44 PM, Hilmar Lapp <hlapp at drycafe.net> wrote:
>> >>
>> >>> Yes, indeed. And lo and behold, the Open Tree of Life (
>> >>> http://opentreeoflife.org) folks are at present building their tree
>> store
>> >>> on top of Neo4J. They're mostly Python, though. (Which I'm sure our
>> >>> Biopython colleagues will love to hear.)
>> >>>
>> >>>         -hilmar
>> >>>
>> >>> On Sep 17, 2012, at 11:28 AM, Fields, Christopher J wrote:
>> >>>
>> >>> > Following up on this, any specific modules where this would be a
>> boon?
>> >>>  I could see this being very useful for Bio::Tree (and Jason's recent
>> work
>> >>> on creating a persistent backend for storing tree info). Bio::Ontology
>> also…
>> >>> >
>> >>> > chris
>> >>> >
>> >>> > On Sep 17, 2012, at 12:08 PM, "Fields, Christopher J" <
>> >>> cjfields at illinois.edu> wrote:
>> >>> >
>> >>> >> Agree, very nice!
>> >>> >>
>> >>> >> chris
>> >>> >>
>> >>> >> On Sep 17, 2012, at 10:00 AM, Hilmar Lapp <hlapp at drycafe.net>
>> wrote:
>> >>> >>
>> >>> >>> Very nice, Mark! -hilmar
>> >>> >>>
>> >>> >>> On Sep 17, 2012, at 8:13 AM, Mark A. Jensen wrote:
>> >>> >>>
>> >>> >>>> Hello all,
>> >>> >>>> If anyone is experimenting with Neo4j (www.neo4j.org) and needs a
>> >>> full-featured Perl OO interface, please give REST::Neo4p (
>> >>> https://metacpan.org/module/REST::Neo4p) a try. Send me the bugs via
>> RT!
>> >>> >>>> cheers,
>> >>> >>>> MAJ
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> _______________________________________________
>> >>> >>>> Bioperl-l mailing list
>> >>> >>>> Bioperl-l at lists.open-bio.org
>> >>> >>>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>> >>> >>>
>> >>> >>> --
>> >>> >>> ===========================================================
>> >>> >>> : Hilmar Lapp -:- Durham, NC -:- hlapp at drycafe dot net :
>> >>> >>> ===========================================================
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>> _______________________________________________
>> >>> >>> Bioperl-l mailing list
>> >>> >>> Bioperl-l at lists.open-bio.org
>> >>> >>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>> >>> >>
>> >>> >>
>> >>> >> _______________________________________________
>> >>> >> Bioperl-l mailing list
>> >>> >> Bioperl-l at lists.open-bio.org
>> >>> >> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>> >>> >
>> >>>
>> >>> --
>> >>> ===========================================================
>> >>> : Hilmar Lapp -:- Durham, NC -:- hlapp at drycafe dot net :
>> >>> ===========================================================
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Bioperl-l mailing list
>> >>> Bioperl-l at lists.open-bio.org
>> >>> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>> >>>
>> >>
>> >>_______________________________________________
>> >>Bioperl-l mailing list
>> >>Bioperl-l at lists.open-bio.org
>> >>http://lists.open-bio.org/mailman/listinfo/bioperl-l
>> >>
>> >
>> >
>> >_______________________________________________
>> >Bioperl-l mailing list
>> >Bioperl-l at lists.open-bio.org
>> >http://lists.open-bio.org/mailman/listinfo/bioperl-l
>> >
>>
>>
>>
>