From ktym at hgc.jp Tue Dec 7 06:51:32 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Tue, 7 Dec 2010 20:51:32 +0900 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th Message-ID: Dear BioRuby developers, Here we announce the first BioRuby world wide IRC conference! The topic is: "Current and future BioRuby development" The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), which is 15pm CET (Central European Time) and 9am EST (New York) and 6am PST (California time). You are kindly invited to join! Server: irc.freenode.org Channel: #bioruby Toshiaki Katayama on behalf of the BioRuby staff. From pjotr.public14 at thebird.nl Tue Dec 7 14:42:13 2010 From: pjotr.public14 at thebird.nl (pjotr.public14 at thebird.nl) Date: Tue, 7 Dec 2010 20:42:13 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: References: Message-ID: <20101207194213.GA25866@thebird.nl> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: > Dear BioRuby developers, > > Here we announce the first BioRuby world wide IRC conference! > > The topic is: "Current and future BioRuby development" > > The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), > which is 15pm CET (Central European Time) and 9am EST (New York) and > 6am PST (California time). Great idea! I, for one would like to look at BioRuby on the JVM. Another topic would be integrationg the brilliant http://matplotlib.sourceforge.net/. Yes, it is Python, but we could bridge that. Would it be an idea to have talks? We could do that in the form of videos. I don't know what people would like to present, but if you are interested in giving, or hearing, some talk, we could make them available just before the conference day. Videos of 10-20 minutes, preferably a code walk/tutorial. Who is game? If anyone is interested I could do a talk on either 1. Ruby actors for parallelization 2. The (not yet high performance) genome browser plugin (GFF3) Maybe Raoul can present the plugin system, and Naohisa a code walk on certain modules? Anyone interested in a presentation on Rubyshell inside the browser (Rails)? If it is successful, we could make this an annual happening. Pj. From bonnalraoul at ingm.it Thu Dec 9 04:12:14 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 9 Dec 2010 10:12:14 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101207194213.GA25866@thebird.nl> References: <20101207194213.GA25866@thebird.nl> Message-ID: <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> Dear All, I'd like to present the plugin system and the scaffolder but unfortunately on Dec 19 at the meeting time I'll landing in Milan, Malpensa. Btw I'll find out a way to be connected. I like to idea of BioRubyCast +1 I'm interested on RubyShell/Rails. Il giorno 07/dic/2010, alle ore 20.42, pjotr.public14 at thebird.nl ha scritto: > On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >> Dear BioRuby developers, >> >> Here we announce the first BioRuby world wide IRC conference! >> >> The topic is: "Current and future BioRuby development" >> >> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >> which is 15pm CET (Central European Time) and 9am EST (New York) and >> 6am PST (California time). > > Great idea! I, for one would like to look at BioRuby on the JVM. > > Another topic would be integrationg the brilliant > http://matplotlib.sourceforge.net/. Yes, it is Python, but we could > bridge that. > > Would it be an idea to have talks? We could do that in the form of > videos. I don't know what people would like to present, but if you are > interested in giving, or hearing, some talk, we could make them > available just before the conference day. Videos of 10-20 minutes, > preferably a code walk/tutorial. > > Who is game? > > If anyone is interested I could do a talk on either > > 1. Ruby actors for parallelization > 2. The (not yet high performance) genome browser plugin (GFF3) > > Maybe Raoul can present the plugin system, and Naohisa a code walk on > certain modules? Anyone interested in a presentation on Rubyshell > inside the browser (Rails)? > > If it is successful, we could make this an annual happening. > > Pj. > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From mail at michaelbarton.me.uk Thu Dec 9 12:06:58 2010 From: mail at michaelbarton.me.uk (Michael Barton) Date: Thu, 9 Dec 2010 12:06:58 -0500 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> Message-ID: I have also written a genome scaffolder in Ruby. The lastest doc is located at - http://bit.ly/ho2UtW I hope to release this within the next week or two. Cheers Mike On 9 December 2010 04:12, Raoul Bonnal wrote: > Dear All, > I'd like to present the plugin system and the scaffolder but unfortunately on Dec 19 at the meeting time I'll landing in Milan, Malpensa. > Btw I'll find out a way to be connected. > > I like to idea of BioRubyCast +1 > > I'm interested on RubyShell/Rails. > > > Il giorno 07/dic/2010, alle ore 20.42, pjotr.public14 at thebird.nl ha scritto: > >> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>> Dear BioRuby developers, >>> >>> Here we announce the first BioRuby world wide IRC conference! >>> >>> The topic is: "Current and future BioRuby development" >>> >>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>> 6am PST (California time). >> >> Great idea! I, for one would like to look at BioRuby on the JVM. >> >> Another topic would be integrationg the brilliant >> http://matplotlib.sourceforge.net/. Yes, it is Python, but we could >> bridge that. >> >> Would it be an idea to have talks? We could do that in the form of >> videos. I don't know what people would like to present, but if you are >> interested in giving, or hearing, some talk, we could make them >> available just before the conference day. Videos of 10-20 minutes, >> preferably a code walk/tutorial. >> >> Who is game? >> >> If anyone is interested I could do a talk on either >> >> 1. Ruby actors for parallelization >> 2. The (not yet high performance) genome browser plugin (GFF3) >> >> Maybe Raoul can present the plugin system, and Naohisa a code walk on >> certain modules? Anyone interested in a presentation on Rubyshell >> inside the browser (Rails)? >> >> If it is successful, we could make this an annual happening. >> >> Pj. >> >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From pjotr.public14 at thebird.nl Thu Dec 9 12:45:53 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 9 Dec 2010 18:45:53 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> Message-ID: <20101209174553.GA11569@thebird.nl> On Thu, Dec 09, 2010 at 12:06:58PM -0500, Michael Barton wrote: > I have also written a genome scaffolder in Ruby. > The lastest doc is located at - http://bit.ly/ho2UtW > I hope to release this within the next week or two. That is a very nice idea - it would be great if we can get rid of gff3. And it is perfect for a plugin. I only wonder how YAML will hold up with really big files. Pj. From mail at michaelbarton.me.uk Thu Dec 9 13:10:06 2010 From: mail at michaelbarton.me.uk (Michael Barton) Date: Thu, 9 Dec 2010 13:10:06 -0500 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101209174553.GA11569@thebird.nl> References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> <20101209174553.GA11569@thebird.nl> Message-ID: <20101209181006.GA35533@nku069218.hh.nku.edu> Thanks Pjotr. I wrote it for my own needs doing genome assembly on a 7MBp microbe. At the moment I'm doing my own mini-code freeze so that I can finish the website and documentation for releasing version 0.4. Here's a bit more detail which may be of interest: The YAML format maps on to a DSL for the Scaffolder::Region class. --- some_class: some_attr: "value" The above YAML corresponds to a DSL which uses Ruby's metaprogramming. The top level name corresponds to a class and the second level indentations correspond to the attributes on the instance of that class. So the above YAML would fetch the subclass of Scaffolder::Region named SomeClass, create a new instance, and call 'some_method' on this instance with the parameter "value". All you would need to do to define this class is as follows. class SomeClass << Scaffolder::Region attribute :some_attr end I tried to design it so that the vocabulary of the YAML scaffold definition is completely extensible. See the Scaffolder::Region#attribute method for more details - http://bit.ly/eMImcn Here's the slides from a short presentation I gave on it recently also: http://bit.ly/cwH2Xm. I agree that really large YAML files may be a problem with memory requirements. However I'm working on small increments developments so far. I've yet to encounter it though but hopefully will be able to solve it when I reach it. Any suggestions from the bioruby community are very welcome. Cheers Mike On Thu, Dec 09, 2010 at 06:45:53PM +0100, Pjotr Prins wrote: > On Thu, Dec 09, 2010 at 12:06:58PM -0500, Michael Barton wrote: > > I have also written a genome scaffolder in Ruby. > > The lastest doc is located at - http://bit.ly/ho2UtW > > I hope to release this within the next week or two. > > That is a very nice idea - it would be great if we can get rid of > gff3. And it is perfect for a plugin. > > I only wonder how YAML will hold up with really big files. > > Pj. From bonnalraoul at ingm.it Fri Dec 10 06:45:16 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Fri, 10 Dec 2010 12:45:16 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101209181006.GA35533@nku069218.hh.nku.edu> References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> <20101209174553.GA11569@thebird.nl> <20101209181006.GA35533@nku069218.hh.nku.edu> Message-ID: <30100CBE-81F2-41CD-84CD-63610A9B4A77@ingm.it> Got it. It's a good idea. Sorry I use the term Scaffolder but it's a different thing it's more like a scaffold generator code for developing bioruby libraries. We can packit as a bioruby gem with my lib. Also George wrote a library, asking for the plugin guidelines time ago, we could pack it too. Il giorno 09/dic/2010, alle ore 19.10, Michael Barton ha scritto: > Thanks Pjotr. I wrote it for my own needs doing genome assembly on a 7MBp > microbe. At the moment I'm doing my own mini-code freeze so that I can finish > the website and documentation for releasing version 0.4. > > Here's a bit more detail which may be of interest: > > The YAML format maps on to a DSL for the Scaffolder::Region class. > > --- > some_class: > some_attr: "value" > > The above YAML corresponds to a DSL which uses Ruby's metaprogramming. The top > level name corresponds to a class and the second level indentations correspond > to the attributes on the instance of that class. So the above YAML would fetch > the subclass of Scaffolder::Region named SomeClass, create a new instance, and > call 'some_method' on this instance with the parameter "value". > > All you would need to do to define this class is as follows. > > class SomeClass << Scaffolder::Region > attribute :some_attr > end > > I tried to design it so that the vocabulary of the YAML scaffold definition is > completely extensible. See the Scaffolder::Region#attribute method for more > details - http://bit.ly/eMImcn > > Here's the slides from a short presentation I gave on it recently also: > http://bit.ly/cwH2Xm. > > I agree that really large YAML files may be a problem with memory requirements. > However I'm working on small increments developments so far. I've yet to > encounter it though but hopefully will be able to solve it when I reach it. > Any suggestions from the bioruby community are very welcome. > > Cheers > > Mike > > On Thu, Dec 09, 2010 at 06:45:53PM +0100, Pjotr Prins wrote: >> On Thu, Dec 09, 2010 at 12:06:58PM -0500, Michael Barton wrote: >>> I have also written a genome scaffolder in Ruby. >>> The lastest doc is located at - http://bit.ly/ho2UtW >>> I hope to release this within the next week or two. >> >> That is a very nice idea - it would be great if we can get rid of >> gff3. And it is perfect for a plugin. >> >> I only wonder how YAML will hold up with really big files. >> >> Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Sat Dec 11 04:46:54 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 11 Dec 2010 10:46:54 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) Message-ID: <20101211094654.GA19598@thebird.nl> Hi Rubyistas, We have a special community, with a special language. Ruby is one of the most fun languages to work with, and we know it. Here I want to argue that for bioinformatics JRuby is one of the most exciting developments. Not only is it pretty fast, once compiled, but it also allows easy integration with Java (and BioJava). Before you recoil in horror, it also allows integration with some really cool programming languages, i.e. Scala, Clojure and Groovy. You know Ruby is great. But it has some weaknesses too. For Bioinformatics (B) and big data (BD) the problems are: (1) Weak B functionality (2) BD performance issues (3) So so parallel computing support (for BD) (4) Only partial functional programming support (1) and (2) can be resolved by using JRuby, BioJava and the JVM. (3) and (4) can be resolved by tapping into Scala and Clojure. Let me try to explain. (1) Weak B functionality: BioRuby is a great achievement, but I have a number of criticisms. First is that it is not suitable for BD. Almost every module loads all data in RAM, and there is no concept of parallel computations in the design. Finally the development is not fast - we are suffering from the fact that we are a small community. You could argue about reasons, but I don't think we should spend energy on the past, when there is such an obvious way forward. Let me continue. (2) BD performance issues: Ruby is slow. By definition compared to statically typed and compiled languages (such as C, Java, Scala, Clojure). It is pretty amazing to see how much speed Ruby 1.9 has. But, for BD it breaks down quickly. Ruby's strength is in beauty of code, but not in raw power. (3) So so parallel computing support (for BD): Functional languages (Haskell, Erlang, Scala, Clojure) have immutable data, and abstractions for parallelization, such as shared memory and actors, which make it much easier to write parallelized code. For performance and BD, this is extremely useful. (4) Only partial functional programming support: Once you get into functional programming you realize Ruby gets in the way. Support for functional programming in Ruby is patchy, though there is some. It is no accident that I have started the BioScala project, and Jan Aerts has started the BioClojure project in 2010 (!) BioRuby has spin-offs. My experience with Scala has been great. Scala is statically typed, and very fast. It also allows beautiful code with functional programming and parallelization thrown in. For me, there is a clear path where I use Ruby and Scala on a 50/50 basis. Essentially using the best of both worlds. JRuby is key to combining them. And, you know what? It is great fun!! I would get frustrated if I was locked in either language. But now it is seamless moving between the two, thanks to the JVM. Which, btw, these days can outperform even C code. Believe me, even two years ago, I would not have thought I would *ever* champion the JVM. But as a saaientist, you go by evidence. Programming is fun. And it has not ever been this great. I want to share that with you, and I would like to use the coming holiday season, and years after, to pass that on. I would guess Jan thinks the same way about Ruby and Clojure. Who is interested in getting back into the fun of programming? Who wants to experiment and become an even more productive programmer? It could be the goal of BioRuby in 2011 to show the way to other Bio* projects of handling development in such a way that we can easily move between the strengths of dynamic programming languages and high performant functional languages. I would like that. Pj. From pjotr.public14 at thebird.nl Sun Dec 12 11:47:09 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sun, 12 Dec 2010 17:47:09 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101211094654.GA19598@thebird.nl> References: <20101211094654.GA19598@thebird.nl> Message-ID: <20101212164709.GA27907@thebird.nl> On Sat, Dec 11, 2010 at 10:46:54AM +0100, Pjotr Prins wrote: > We have a special community, with a special language. Ruby is one of > the most fun languages to work with, and we know it. The response is overwhelming. Does it mean no one has fun anymore? Or is Ruby dying a slow death in bioinformatics? Or was my message just over everyone's head? If anyone is interested I could do a screencast on using BioRuby and BioJava on the JVM. But I fear, somehow, the effort is wasted. Pj. From russell.whitaker at gmail.com Sun Dec 12 12:21:38 2010 From: russell.whitaker at gmail.com (Russell Whitaker) Date: Sun, 12 Dec 2010 09:21:38 -0800 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101212164709.GA27907@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: On Sun, Dec 12, 2010 at 8:47 AM, Pjotr Prins wrote: > On Sat, Dec 11, 2010 at 10:46:54AM +0100, Pjotr Prins wrote: >> We have a special community, with a special language. Ruby is one of >> the most fun languages to work with, and we know it. > > The response is overwhelming. Does it mean no one has fun anymore? Or > is Ruby dying a slow death in bioinformatics? Or was my message just over > everyone's head? > > If anyone is interested I could do a screencast on using BioRuby and > BioJava on the JVM. But I fear, somehow, the effort is wasted. > I, for one, would not consider that a wasted effort, and would invite others in my community to watch (assuming scheduling worked out). -- Russell Whitaker http://twitter.com/OrthoNormalRuss http://orthonormalruss.blogspot.com/ From russell.whitaker at gmail.com Sun Dec 12 13:30:06 2010 From: russell.whitaker at gmail.com (Russell Whitaker) Date: Sun, 12 Dec 2010 10:30:06 -0800 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: On Sun, Dec 12, 2010 at 10:17 AM, benjamin Almeida wrote: > I am interested too, so far I only used jruby (with bioruby) to have access > to neo4j and because it was faster. But my colleague uses scala so this is > definitely interesting to see how someone combines these two! > cheers > ben > CC'ing back to the list, since you accidentally replied privately. -- Russell Whitaker http://twitter.com/OrthoNormalRuss http://orthonormalruss.blogspot.com/ From pjotr.public14 at thebird.nl Sun Dec 12 13:40:43 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sun, 12 Dec 2010 19:40:43 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: <20101212184043.GA29332@thebird.nl> > On Sun, Dec 12, 2010 at 10:17 AM, benjamin Almeida > I am interested too, so far I only used jruby (with bioruby) to have access > to neo4j and because it was faster. But my colleague uses scala so this is > definitely interesting to see how someone combines these two! > cheers > ben >From my perspective neo4j looks interesting too - looks like a natural fit with the semantic web. Pj. From cjfields at illinois.edu Sun Dec 12 14:19:52 2010 From: cjfields at illinois.edu (Chris Fields) Date: Sun, 12 Dec 2010 13:19:52 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101212164709.GA27907@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: On Dec 12, 2010, at 10:47 AM, Pjotr Prins wrote: > On Sat, Dec 11, 2010 at 10:46:54AM +0100, Pjotr Prins wrote: >> We have a special community, with a special language. Ruby is one of >> the most fun languages to work with, and we know it. > > The response is overwhelming. Does it mean no one has fun anymore? Or > is Ruby dying a slow death in bioinformatics? Or was my message just over > everyone's head? > > If anyone is interested I could do a screencast on using BioRuby and > BioJava on the JVM. But I fear, somehow, the effort is wasted. > > Pj. Pjotr, I think, with efforts like this you'll find interest but also the need to demonstrate the utility of what you are proposing. An email may not be the best medium for planning things out along those lines. Also, users may be interested but simply may not have the time or resources to donate to such an effort unless they have a vested interest in it, what I find to be a common problem in many informatics-related community projects. Some commonly-used projects seem to have a 'bus factor' of 1 for this reason. A screencast might go a long way to helping with both of those. chris From francesco.strozzi at gmail.com Sun Dec 12 14:43:15 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Sun, 12 Dec 2010 20:43:15 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101211094654.GA19598@thebird.nl> References: <20101211094654.GA19598@thebird.nl> Message-ID: Hi Piotr, It's been a while since I started collaborating with Jan Aerts to develop the Ruby Ensembl API. It was a good chance for me to learn Ruby and develop bioinformatics programs. But I have to admit that, as you said, Ruby isn't so powerful for bioinformatics compared to other languages. So, like you, I feel the need to add another language for "real" and "though" bioinformatics work and I have started learning Python in the last months which it's really impressive. The core library is very poweful and BioPython is updated every 3 months (so it is under constant development). The performance of many common tasks (e.g. parsing a large Blast report) are superior than with BioRuby libraries, for what I have seen. But Ruby syntax and logic is simply amazing (and to me even better than Python) and it's a shame we don't have powerful solutions to bring Ruby out of the "just for the web" corner where it was put since Rails outbreak. So, from my point of view, every effort that could make Ruby shine for bioinformatics...well, is really really appreciated and I want to hear and read more! >From the technical point of view I have a couple of (newbie) questions: - I have no idea what Clojure and Scala are, so a sort of "Functional Programming for Dummies" will be appreciated. - How much Java one need to know in order to be productive with Ruby/BioRuby and Java/BioJava using JRuby? Because from my point of view, a Ruby programmer I think doesn't really want to learn Java (which is a big language but with it's own issues and is not so flexible as Ruby or Python). So in the end, I will read and hear with interest what you can tell us about your experience and future directions for Ruby/BioRuby development. Cheers Francesco On Sat, Dec 11, 2010 at 10:46, Pjotr Prins wrote: > Hi Rubyistas, > > We have a special community, with a special language. Ruby is one of > the most fun languages to work with, and we know it. > > Here I want to argue that for bioinformatics JRuby is one of the most > exciting developments. Not only is it pretty fast, once compiled, > but it also allows easy integration with Java (and BioJava). Before > you recoil in horror, it also allows integration with some really cool > programming languages, i.e. Scala, Clojure and Groovy. > > You know Ruby is great. But it has some weaknesses too. For > Bioinformatics (B) and big data (BD) the problems are: > > (1) Weak B functionality > (2) BD performance issues > (3) So so parallel computing support (for BD) > (4) Only partial functional programming support > > (1) and (2) can be resolved by using JRuby, BioJava and the JVM. (3) > and (4) can be resolved by tapping into Scala and Clojure. > > Let me try to explain. > > (1) Weak B functionality: BioRuby is a great achievement, but I have a > number of criticisms. First is that it is not suitable for BD. Almost > every module loads all data in RAM, and there is no concept of > parallel computations in the design. Finally the development is not > fast - we are suffering from the fact that we are a small community. > You could argue about reasons, but I don't think we should spend > energy on the past, when there is such an obvious way forward. Let me > continue. > > (2) BD performance issues: Ruby is slow. By definition compared to > statically typed and compiled languages (such as C, Java, Scala, > Clojure). It is pretty amazing to see how much speed Ruby 1.9 has. > But, for BD it breaks down quickly. Ruby's strength is in beauty of > code, but not in raw power. > > (3) So so parallel computing support (for BD): Functional languages > (Haskell, Erlang, Scala, Clojure) have immutable data, and > abstractions for parallelization, such as shared memory and actors, > which make it much easier to write parallelized code. For performance > and BD, this is extremely useful. > > (4) Only partial functional programming support: Once you get into > functional programming you realize Ruby gets in the way. Support for > functional programming in Ruby is patchy, though there is some. > > It is no accident that I have started the BioScala project, and Jan Aerts > has > started the BioClojure project in 2010 (!) BioRuby has spin-offs. > > My experience with Scala has been great. Scala is statically typed, > and very fast. It also allows beautiful code with functional > programming and parallelization thrown in. For me, there is a clear > path where I use Ruby and Scala on a 50/50 basis. Essentially using > the best of both worlds. JRuby is key to combining them. > > And, you know what? It is great fun!! I would get frustrated if I was > locked in either language. But now it is seamless moving between the > two, thanks to the JVM. Which, btw, these days can outperform even C > code. > > Believe me, even two years ago, I would not have thought I would > *ever* champion the JVM. But as a saaientist, you go by evidence. > > Programming is fun. And it has not ever been this great. I want to > share that with you, and I would like to use the coming holiday > season, and years after, to pass that on. I would guess Jan thinks the > same way about Ruby and Clojure. > > Who is interested in getting back into the fun of programming? Who > wants to experiment and become an even more productive programmer? > It could be the goal of BioRuby in 2011 to show the way to other Bio* > projects of handling development in such a way that we can easily move > between the strengths of dynamic programming languages and high > performant functional languages. > > I would like that. > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Mon Dec 13 06:42:38 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 13 Dec 2010 12:42:38 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> Message-ID: <20101213114238.GA32702@thebird.nl> On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: > Ruby isn't so powerful for bioinformatics compared to other > languages. So, like you, I feel the need to add another language for > "real" and "though" bioinformatics work and I have started learning > Python in the last months which it's really impressive. The core > library is very poweful and BioPython is updated every 3 months (so > it is under constant development). Main comment would be that Python is not much different from Ruby. It is interpreted, dymamic, and does not have good support for parallel programming, nor the speed required for big data (BD). Perl, Python and Ruby are really in the same boat. > The performance of many common tasks (e.g. parsing a large Blast > report) are superior than with BioRuby libraries, for what I have > seen. Here you have a good point. BioRuby is lagging behind BioPython, BioPerl and BioJava. My proposal is to use BioJava together with BioRuby. You can do something similar with Jython (Python on the JVM), though I have the impression Jython is not as active as JRuby, anymore. JRuby is getting strong industry support, at this point. > But Ruby syntax and logic is simply amazing (and to me even > better than Python) and it's a shame we don't have powerful > solutions to bring Ruby out of the "just for the web" corner where > it was put since Rails outbreak. I know people who would disagree with that last statement... To me, Ruby is a way of thinking, which is reflected in Rails. > So, from my point of view, every effort > that could make Ruby shine for bioinformatics...well, is really really > appreciated and I want to hear and read more! > From the technical point of view I have a couple of (newbie) questions: > - I have no idea what Clojure and Scala are, so a sort of "Functional > Programming for Dummies" will be appreciated. There are plenty online resources. The main Scala book is great. > - How much Java one need to know in order to be productive with > Ruby/BioRuby and Java/BioJava using JRuby? Not Java per se. Mostly it is getting used to the JVM world and docs. > Because from my point of view, a Ruby programmer I think doesn't really > want to learn Java (which is a big language but with it's own issues and is > not so flexible as Ruby or Python). I agree. I personally dislike the Java language. After a project around 2001 I vowed not to touch it again. It is not a hard language, but merely a pain. I liked any of the other Bio* languages better. > So in the end, I will read and hear with interest what you can tell us about > your experience and future directions for Ruby/BioRuby development. My suggestion would be to study a language more removed from Python, Ruby and Perl. I will try to make a screencast. Pj. From bonnalraoul at ingm.it Mon Dec 13 07:29:19 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Mon, 13 Dec 2010 13:29:19 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213114238.GA32702@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: +1 for screen cast which demostrates how jruby + scala or clojure and parallel programming can outperform a regular ruby script. I have to admit that parallel programming in Ruby is not very different from years ago however there are interesting methods for implementing it -I can't fine the title of a Ruby's book on this topic-. Unfortunately Ruby is a terrific language which suffers from the association Ruby => Rails Other languages have a lot of libraries which are very solid, mathematics, statistics and charting/plotting which I think are the main libs that we need in our everyday work. I was fascinated from JRuby and the ability to use java libraries is a +1 from my computer scientist/programmer point of view. I could use R whit a very solid tcp protocol for example. The same thing is not possible with Ruby 1.9 or 1.8 because the actual implementation lacks complex handling of dataframe. Btw... I think that we should poll here who wants to actively contribute to BioRuby ? Then I think we should address what we need to use bioruby/ruby in a more productive way or easiest way. @Francesco: which are the motivation or real case studies which drive you to shift from Ruby to Python ? Jan did a very interesting post on Clojure about how easy it's to plot data for NGS, are there alternative in ruby or is there someone here that faced the same problem and tried to find a solution with BioRuby ? A library that I started to use some time ago is http://ruby-statsample.rubyforge.org/ it's not R but I got the same results whitout the need to use R inside my script. Why is there a lot of interest of Jruby? My opinion is: Containers and Rails. I you want to use a rails application in an enterprise company you need to put it into a container: glassfish, jboss, Oracle. The only problem with Java for me is Oracle itself but... I'm quite scared about that. Ciao. Il giorno 13/dic/2010, alle ore 12.42, Pjotr Prins ha scritto: > On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: >> Ruby isn't so powerful for bioinformatics compared to other >> languages. So, like you, I feel the need to add another language for >> "real" and "though" bioinformatics work and I have started learning >> Python in the last months which it's really impressive. The core >> library is very poweful and BioPython is updated every 3 months (so >> it is under constant development). > > Main comment would be that Python is not much different from Ruby. It > is interpreted, dymamic, and does not have good support for parallel > programming, nor the speed required for big data (BD). Perl, Python > and Ruby are really in the same boat. > >> The performance of many common tasks (e.g. parsing a large Blast >> report) are superior than with BioRuby libraries, for what I have >> seen. > > Here you have a good point. BioRuby is lagging behind BioPython, > BioPerl and BioJava. > > My proposal is to use BioJava together with BioRuby. You can do > something similar with Jython (Python on the JVM), though I have the > impression Jython is not as active as JRuby, anymore. JRuby is > getting strong industry support, at this point. > >> But Ruby syntax and logic is simply amazing (and to me even >> better than Python) and it's a shame we don't have powerful >> solutions to bring Ruby out of the "just for the web" corner where >> it was put since Rails outbreak. > > I know people who would disagree with that last statement... > > To me, Ruby is a way of thinking, which is reflected in Rails. > >> So, from my point of view, every effort >> that could make Ruby shine for bioinformatics...well, is really really >> appreciated and I want to hear and read more! >> From the technical point of view I have a couple of (newbie) questions: >> - I have no idea what Clojure and Scala are, so a sort of "Functional >> Programming for Dummies" will be appreciated. > > There are plenty online resources. The main Scala book is great. > >> - How much Java one need to know in order to be productive with >> Ruby/BioRuby and Java/BioJava using JRuby? > > Not Java per se. Mostly it is getting used to the JVM world and docs. > >> Because from my point of view, a Ruby programmer I think doesn't really >> want to learn Java (which is a big language but with it's own issues and is >> not so flexible as Ruby or Python). > > I agree. I personally dislike the Java language. > > After a project around 2001 I vowed not to touch it again. It is not a > hard language, but merely a pain. I liked any of the other Bio* > languages better. > >> So in the end, I will read and hear with interest what you can tell us about >> your experience and future directions for Ruby/BioRuby development. > > My suggestion would be to study a language more removed from Python, > Ruby and Perl. > > I will try to make a screencast. > > Pj. > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From francesco.strozzi at gmail.com Mon Dec 13 08:26:34 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 13 Dec 2010 14:26:34 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: On Mon, Dec 13, 2010 at 13:29, Raoul Bonnal wrote: > > > Unfortunately Ruby is a terrific language which suffers from the > association Ruby => Rails > Yes I agree, that's what I was saying when I wrote about the "just for the web" corner. Rails is amazing and I love it, but I think now Ruby is seen more as the language for Rails and less as a robust language that could do almost the same things as Python, in my opinion. > > @Francesco: which are the motivation or real case studies which drive you > to shift from Ruby to Python ? > Basically the Bio library usage. Reading through the docs and using the different libraries, I found BioPython to be more complete and with a better performance, in particular for parsing big files (like Blast reports, assembly data, NGS files etc.). And the math libraries are amazing. With matplotlib and NumPy/SciPy you could avoid using R for what I have seen. And the documentation is vast. Also Python syntax is quite similar to Ruby in many ways, so moving from one to another wasn't so hard (even if mastering a language is not a matter of few months of practice). In the end I found myself running things in Ruby to develop web applications(!) or for the Ruby Ensembl API, and doing everything else with Python...that's the reason why I have written here to give my "+1", as I completely agree with Pjotr and I'd like to learn new ways of using Ruby (with Scala/Clojure or whatever) to do the "hard" work as well... Ciao -- Francesco From pjotr.public14 at thebird.nl Mon Dec 13 08:57:03 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 13 Dec 2010 14:57:03 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: <20101213135703.GA2024@thebird.nl> For me it was the other way round. Web programming in Python, wrote ZopeInAnger, and programmed everything else in Ruby :) I agree Python has some really nice projects for Science. I mentioned matplotlib before. On Mon, Dec 13, 2010 at 02:26:34PM +0100, Francesco Strozzi wrote: > I have written here to give my "+1", as I completely agree with > Pjotr and I'd like to learn new ways of using Ruby (with > Scala/Clojure or whatever) to do the "hard" work as well... It was not so long ago I would do the performance stuff in C. BioLib being a point in case. C/BioLib is also a valid route. I find, however, that for programming C/C++ bogs me down in implementation detail. Something I don't have in Ruby, Python or Scala. Ruby/Perl/Python wins on rapid/agile, and even dirty coding. But if it comes to execution speed and correctness - the new JVM languages deliver. Part of my excitement is mixing the two ways of programming. Note, the BioLib project is not dead. For legacy code, such as EMBOSS, it is very useful. I will make sure it works from the JVM. I have had little time for coding the last 4 months, but it should be better in 2011. Pj. From biopython at maubp.freeserve.co.uk Mon Dec 13 09:44:16 2010 From: biopython at maubp.freeserve.co.uk (Peter) Date: Mon, 13 Dec 2010 14:44:16 +0000 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213114238.GA32702@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: On Mon, Dec 13, 2010 at 11:42 AM, Pjotr Prins wrote: > > On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: >> The performance of many common tasks (e.g. parsing a large Blast >> report) are superior than with BioRuby libraries, for what I have >> seen. > > Here you have a good point. BioRuby is lagging behind BioPython, > BioPerl and BioJava. Part of how we try to handle big data files in Biopython is using Python iterators, whereby the file is loaded record by record (how depends on the file format - for BLAST we do this query by query), not all into memory in one go. I think BioPerl does something very similar in their parsers, I'm not so familiar with BioJava. > My proposal is to use BioJava together with BioRuby. You can do > something similar with Jython (Python on the JVM), though I have the > impression Jython is not as active as JRuby, anymore. JRuby is > getting strong industry support, at this point. Just for reference, we do test Biopython on Jython (i.e. on the JVM) and most things work. There is a significant downside that you can't use compiled C code, in particular this means you can't use NumPy under Jython (the numerical python library, used in matplotlib and SciPy). Nevertheless, some people are using Biopython on Jython. Peter From bonnalraoul at ingm.it Mon Dec 13 12:57:39 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Mon, 13 Dec 2010 18:57:39 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> Il giorno 13/dic/2010, alle ore 15.44, Peter ha scritto: > On Mon, Dec 13, 2010 at 11:42 AM, Pjotr Prins wrote: >> >> On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: >>> The performance of many common tasks (e.g. parsing a large Blast >>> report) are superior than with BioRuby libraries, for what I have >>> seen. >> >> Here you have a good point. BioRuby is lagging behind BioPython, >> BioPerl and BioJava. > > Part of how we try to handle big data files in Biopython is using > Python iterators, whereby the file is loaded record by record (how > depends on the file format - for BLAST we do this query by query), > not all into memory in one go. I think BioPerl does something very > similar in their parsers, I'm not so familiar with BioJava. This is the right approach. Junior Programmers tend to load everything and/or create huge hashes, at the same time the old ruby interpreter had a bad garbage collector. It's always a matter of coding if we are discussing about performances each language can have worst performances. If scala can be used to easily express the concept or better help in parallel programming why not trying it? > >> My proposal is to use BioJava together with BioRuby. You can do >> something similar with Jython (Python on the JVM), though I have the >> impression Jython is not as active as JRuby, anymore. JRuby is >> getting strong industry support, at this point. > > Just for reference, we do test Biopython on Jython (i.e. on the JVM) > and most things work. There is a significant downside that you can't > use compiled C code, in particular this means you can't use NumPy > under Jython (the numerical python library, used in matplotlib and > SciPy). Nevertheless, some people are using Biopython on Jython. > >From my point of view Python guys are doing a very good job on all fields. Unfortunately I'm in love with ruby :-) -- Ra From pjotr.public14 at thebird.nl Mon Dec 13 13:16:59 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 13 Dec 2010 19:16:59 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> Message-ID: <20101213181659.GA4269@thebird.nl> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > > Part of how we try to handle big data files in Biopython is using > > Python iterators, whereby the file is loaded record by record (how > > depends on the file format - for BLAST we do this query by query), > > not all into memory in one go. I think BioPerl does something very > > similar in their parsers, I'm not so familiar with BioJava. BioJava uses a visitor pattern. In effect an iterator. With all current implementations IO runs, then code, the IO, etc. While we are IO constrained, we are actually doing worse. What I want is an IO thread going at maximum throughput. Every item should get parcelled out for further parsing and processing, in parallel to the IO thread. We should do better, and make it a generalization. I think we can do it by using Scala and the standard BioJava iterators. With Scala it can be turned in a parallelized iterator. That is a fun project. > From my point of view Python guys are doing a very good job on all fields. > Unfortunately I'm in love with ruby :-) All you need is love :) Pj. From cjfields at illinois.edu Mon Dec 13 16:48:23 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 13 Dec 2010 15:48:23 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213181659.GA4269@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> Message-ID: <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: >>> Part of how we try to handle big data files in Biopython is using >>> Python iterators, whereby the file is loaded record by record (how >>> depends on the file format - for BLAST we do this query by query), >>> not all into memory in one go. I think BioPerl does something very >>> similar in their parsers, I'm not so familiar with BioJava. > > BioJava uses a visitor pattern. In effect an iterator. > > With all current implementations IO runs, then code, the IO, etc. > While we are IO constrained, we are actually doing worse. > > What I want is an IO thread going at maximum throughput. Every item > should get parcelled out for further parsing and processing, in > parallel to the IO thread. > > We should do better, and make it a generalization. I think we can do > it by using Scala and the standard BioJava iterators. With Scala it > can be turned in a parallelized iterator. That is a fun project. > >> From my point of view Python guys are doing a very good job on all fields. >> Unfortunately I'm in love with ruby :-) > > All you need is love :) > > Pj. At some point the choice of a language will not matter as much, as long as it is implemented in a VM (something Perl 5 cannot claim at the moment, but Perl 6 does with the Parrot VM). chris From bonnalraoul at ingm.it Mon Dec 13 19:21:39 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 14 Dec 2010 01:21:39 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> Message-ID: <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Hi Chris, Il giorno 13/dic/2010, alle ore 22.48, Chris Fields ha scritto: > On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > > At some point the choice of a language will not matter as much, as long as it is implemented in a VM (something Perl 5 cannot claim at the moment, but Perl 6 does with the Parrot VM). The real problem was ruby 1.8, something in the middle between 1.9 and the previous release. One year ago JRuby wasn't complain with the 1.9 standard. The Idea to use the JVM is to reuse other libraries, at that point all the Bio* that can run in it could share the same library, it's not too bad. Ideally we could develop a common set of libraries but probably is too complex. There is also another problem, maintaining the current installations, how? Do you know which are the performances using different JVM ? Pjotr: do you mean an IO access in a map/reduce style/way? -- Ra From cjfields at illinois.edu Mon Dec 13 21:31:52 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 13 Dec 2010 20:31:52 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Message-ID: On Dec 13, 2010, at 6:21 PM, Raoul Bonnal wrote: > Hi Chris, > > Il giorno 13/dic/2010, alle ore 22.48, Chris Fields ha scritto: > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: >> >> At some point the choice of a language will not matter as much, as long as it is implemented in a VM (something Perl 5 cannot claim at the moment, but Perl 6 does with the Parrot VM). > The real problem was ruby 1.8, something in the middle between 1.9 and the previous release. One year ago JRuby wasn't complain with the 1.9 standard. > The Idea to use the JVM is to reuse other libraries, at that point all the Bio* that can run in it could share the same library, it's not too bad. Ideally we could develop a common set of libraries but probably is too complex. > There is also another problem, maintaining the current installations, how? > Do you know which are the performances using different JVM ? No idea. The problem with lack of C-based bindings is a bit of a worry (Peter mentioned this in a previous post in this thread). -c > Pjotr: do you mean an IO access in a map/reduce style/way? > > -- > Ra From pjotr.public14 at thebird.nl Tue Dec 14 02:13:50 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 14 Dec 2010 08:13:50 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Message-ID: <20101214071350.GA6860@thebird.nl> On Tue, Dec 14, 2010 at 01:21:39AM +0100, Raoul Bonnal wrote: > Pjotr: do you mean an IO access in a map/reduce style/way? No. Mapreduce only works when you can reduce data into independent packages of work. I find it does not generally work in biological research. Way overrated. Py. From pjotr.public14 at thebird.nl Tue Dec 14 02:32:28 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 14 Dec 2010 08:32:28 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Message-ID: <20101214073228.GB6860@thebird.nl> On Mon, Dec 13, 2010 at 08:31:52PM -0600, Chris Fields wrote: > At some point the choice of a language will not matter as much, as > long as it is implemented in a VM (something Perl 5 cannot claim at > the moment, but Perl 6 does with the Parrot VM). In my opinion, multiple VMs will only serve the diaspora in bioinformatics. One VM would be optimal to make effective use of shared competencies. It is pretty clear (to me) that Perl is going it alone... Like the Erlang VM - it becomes a handicap, rather than an asset. > No idea. The problem with lack of C-based bindings is a bit of a > worry (Peter mentioned this in a previous post in this thread). It is no real problem to bind C to the JVM, using SWIG (for example). No worse than binding to Perl5. I am planning as much, since I don't want to drop the C projects from the equation. Nice point is that, a C/C++/FORTRAN lib once bound against the JVM, will work for all JVM languages. So, if, for example, the BioPython would resolve their math C bindings, and make Jython work, it would naturally translate to Java, Ruby, Scala, Clojure (which btw may be an R replacement), Groovy, and any future language on the JVM. Now, *that*, my friends, would be a major step forward in bioinformatics. Not only can we build on each others work, but also, each of us can work in a language of choice. Pj. From bonnalraoul at ingm.it Tue Dec 14 05:46:37 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 14 Dec 2010 11:46:37 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213181659.GA4269@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> Message-ID: <5D135AF4-797E-4F45-948C-59E5CC433D2B@ingm.it> Il giorno 13/dic/2010, alle ore 19.16, Pjotr Prins ha scritto: > With all current implementations IO runs, then code, the IO, etc. > While we are IO constrained, we are actually doing worse. > > What I want is an IO thread going at maximum throughput. Every item > should get parcelled out for further parsing and processing, in > parallel to the IO thread. This is not clear to me, could you make a real example ? -- Ra From pjotr.public14 at thebird.nl Tue Dec 14 07:54:31 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 14 Dec 2010 13:54:31 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <5D135AF4-797E-4F45-948C-59E5CC433D2B@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <5D135AF4-797E-4F45-948C-59E5CC433D2B@ingm.it> Message-ID: <20101214125431.GC9257@thebird.nl> On Tue, Dec 14, 2010 at 11:46:37AM +0100, Raoul Bonnal wrote: > This is not clear to me, could you make a real example ? With FASTA, GFF3 etc. part of the job is reading, part is parsing. These can be on multiple threads. With GFF3 there are dependencies. So if it is not all in memory you get multiple readers and multiple parsers. I'll do an example, at some point. Pj. From cjfields at illinois.edu Tue Dec 14 08:05:31 2010 From: cjfields at illinois.edu (Chris Fields) Date: Tue, 14 Dec 2010 07:05:31 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101214073228.GB6860@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> <20101214073228.GB6860@thebird.nl> Message-ID: <9C9CE549-854B-4EEF-9E00-B0D31C8E2D65@illinois.edu> On Dec 14, 2010, at 1:32 AM, Pjotr Prins wrote: > On Mon, Dec 13, 2010 at 08:31:52PM -0600, Chris Fields wrote: >> At some point the choice of a language will not matter as much, as >> long as it is implemented in a VM (something Perl 5 cannot claim at >> the moment, but Perl 6 does with the Parrot VM). > > In my opinion, multiple VMs will only serve the diaspora in > bioinformatics. One VM would be optimal to make effective use of > shared competencies. It is pretty clear (to me) that Perl is going > it alone... Like the Erlang VM - it becomes a handicap, rather than an > asset. Much of this is due to the complexities involved in parsing the language. It's long been said that only perl can parse perl, but that's changing with perl 6. >> No idea. The problem with lack of C-based bindings is a bit of a >> worry (Peter mentioned this in a previous post in this thread). > > It is no real problem to bind C to the JVM, using SWIG (for example). > No worse than binding to Perl5. I am planning as much, since I don't > want to drop the C projects from the equation. Nice point is that, a > C/C++/FORTRAN lib once bound against the JVM, will work for all JVM > languages. > > So, if, for example, the BioPython would resolve their math C > bindings, and make Jython work, it would naturally translate to Java, > Ruby, Scala, Clojure (which btw may be an R replacement), Groovy, and > any future language on the JVM. Now, *that*, my friends, would be a > major step forward in bioinformatics. Not only can we build on each > others work, but also, each of us can work in a language of choice. > > Pj. The same exists for Parrot (the NCI interface) and likely must exist for any successful VM. The preliminary DBI for Perl 6 uses it. chris From john.woods at marcottelab.org Wed Dec 15 18:02:27 2010 From: john.woods at marcottelab.org (John Woods) Date: Wed, 15 Dec 2010 17:02:27 -0600 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) Message-ID: Hi everyone, I wanted to put in my two cents about this (not this specific email, but the thread). I come from a CS background, and so when I began working in bioinformatics, I shifted from C++ to Python, Perl, back to Python, and finally to Ruby (and Rails!). Working in a bioinformatics lab, my experience has been that most people go with Python over Ruby because of SciPy and NumPy (and matplotlib). At least for us, it's not because of speed at all. If I need something speedy, I code it in C++ and use Rice to build a Ruby adapter. Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and with it, good numerical support). To that end, we've been working a bit to put together a spec for how a SciRuby with plotting and numeric capabilities would work. I'm not quite ready to say anything specific about it publicly just yet, but if you're interested in being involved, please write me back. Best, John O. Woods NSF Fellow The University of Texas at Austin On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields wrote: > On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > > > On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > >>> Part of how we try to handle big data files in Biopython is using > >>> Python iterators, whereby the file is loaded record by record (how > >>> depends on the file format - for BLAST we do this query by query), > >>> not all into memory in one go. I think BioPerl does something very > >>> similar in their parsers, I'm not so familiar with BioJava. > > > > BioJava uses a visitor pattern. In effect an iterator. > > > > With all current implementations IO runs, then code, the IO, etc. > > While we are IO constrained, we are actually doing worse. > > > > What I want is an IO thread going at maximum throughput. Every item > > should get parcelled out for further parsing and processing, in > > parallel to the IO thread. > > > > We should do better, and make it a generalization. I think we can do > > it by using Scala and the standard BioJava iterators. With Scala it > > can be turned in a parallelized iterator. That is a fun project. > > > >> From my point of view Python guys are doing a very good job on all > fields. > >> Unfortunately I'm in love with ruby :-) > > > > All you need is love :) > > > > Pj. > > At some point the choice of a language will not matter as much, as long as > it is implemented in a VM (something Perl 5 cannot claim at the moment, but > Perl 6 does with the Parrot VM). > > chris > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From bonnalraoul at ingm.it Wed Dec 15 19:29:44 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 16 Dec 2010 01:29:44 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: Message-ID: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> +1 for me. Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there is also an optimized version. John, in the web era a way to easily publish plots on the web not just as static images would be great. Probably I could not attend the first meeting, work trip. Btw if someone is around these day I'll try to be in the channel. Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: > Hi everyone, > > I wanted to put in my two cents about this (not this specific email, but the > thread). > > I come from a CS background, and so when I began working in bioinformatics, > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby (and > Rails!). > > Working in a bioinformatics lab, my experience has been that most people go > with Python over Ruby because of SciPy and NumPy (and matplotlib). At least > for us, it's not because of speed at all. If I need something speedy, I code > it in C++ and use Rice to build a Ruby adapter. > > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and > with it, good numerical support). > > To that end, we've been working a bit to put together a spec for how a > SciRuby with plotting and numeric capabilities would work. > > I'm not quite ready to say anything specific about it publicly just yet, but > if you're interested in being involved, please write me back. > > Best, > John O. Woods > NSF Fellow > The University of Texas at Austin > > > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields wrote: > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: >> >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: >>>>> Part of how we try to handle big data files in Biopython is using >>>>> Python iterators, whereby the file is loaded record by record (how >>>>> depends on the file format - for BLAST we do this query by query), >>>>> not all into memory in one go. I think BioPerl does something very >>>>> similar in their parsers, I'm not so familiar with BioJava. >>> >>> BioJava uses a visitor pattern. In effect an iterator. >>> >>> With all current implementations IO runs, then code, the IO, etc. >>> While we are IO constrained, we are actually doing worse. >>> >>> What I want is an IO thread going at maximum throughput. Every item >>> should get parcelled out for further parsing and processing, in >>> parallel to the IO thread. >>> >>> We should do better, and make it a generalization. I think we can do >>> it by using Scala and the standard BioJava iterators. With Scala it >>> can be turned in a parallelized iterator. That is a fun project. >>> >>>> From my point of view Python guys are doing a very good job on all >> fields. >>>> Unfortunately I'm in love with ruby :-) >>> >>> All you need is love :) >>> >>> Pj. >> >> At some point the choice of a language will not matter as much, as long as >> it is implemented in a VM (something Perl 5 cannot claim at the moment, but >> Perl 6 does with the Parrot VM). >> >> chris >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From john.woods at marcottelab.org Wed Dec 15 19:50:54 2010 From: john.woods at marcottelab.org (John Woods) Date: Wed, 15 Dec 2010 18:50:54 -0600 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> Message-ID: I saw ruby-statsample, and it looks like one of many things we'd (ideally) like to integrate. I'm glad you brought up publishing to the web. We spent a not-insignificant amount of time debating graphics platform, and ultimately we decided that HTML5 canvas, and SVG-type graphics, were probably the ideal. For the moment, we're looking at protovis. Interestingly, Claudio Bustos has been working on a Ruby protovis port (he also wrote ruby-statsample, I believe). The other big piece is NArray. Masahiro Tanaka is apparently rewriting it with a goal of getting it incorporated into Ruby core. John Prince has been working on documentation for the existing NArray, in part to see if it's got everything SciRuby would need. But ultimately, the NArray rewrite/core integration is the key. It's years from being finished, supposedly -- but perhaps if there are some eager volunteers we could speed that up. =) So it seems like most of the pieces are present -- it's just a matter of tying them together. And I just said I wasn't going to get into detail -- but there, you just got me going. =) John On Wed, Dec 15, 2010 at 6:29 PM, Raoul Bonnal wrote: > +1 for me. > Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there > is also an optimized version. > > John, in the web era a way to easily publish plots on the web not just as > static images would be great. > > Probably I could not attend the first meeting, work trip. Btw if someone is > around these day I'll try to be in the channel. > > Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: > > > Hi everyone, > > > > I wanted to put in my two cents about this (not this specific email, but > the > > thread). > > > > I come from a CS background, and so when I began working in > bioinformatics, > > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby > (and > > Rails!). > > > > Working in a bioinformatics lab, my experience has been that most people > go > > with Python over Ruby because of SciPy and NumPy (and matplotlib). At > least > > for us, it's not because of speed at all. If I need something speedy, I > code > > it in C++ and use Rice to build a Ruby adapter. > > > > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have > been > > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby > (and > > with it, good numerical support). > > > > To that end, we've been working a bit to put together a spec for how a > > SciRuby with plotting and numeric capabilities would work. > > > > I'm not quite ready to say anything specific about it publicly just yet, > but > > if you're interested in being involved, please write me back. > > > > Best, > > John O. Woods > > NSF Fellow > > The University of Texas at Austin > > > > > > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields > wrote: > > > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > >> > >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > >>>>> Part of how we try to handle big data files in Biopython is using > >>>>> Python iterators, whereby the file is loaded record by record (how > >>>>> depends on the file format - for BLAST we do this query by query), > >>>>> not all into memory in one go. I think BioPerl does something very > >>>>> similar in their parsers, I'm not so familiar with BioJava. > >>> > >>> BioJava uses a visitor pattern. In effect an iterator. > >>> > >>> With all current implementations IO runs, then code, the IO, etc. > >>> While we are IO constrained, we are actually doing worse. > >>> > >>> What I want is an IO thread going at maximum throughput. Every item > >>> should get parcelled out for further parsing and processing, in > >>> parallel to the IO thread. > >>> > >>> We should do better, and make it a generalization. I think we can do > >>> it by using Scala and the standard BioJava iterators. With Scala it > >>> can be turned in a parallelized iterator. That is a fun project. > >>> > >>>> From my point of view Python guys are doing a very good job on all > >> fields. > >>>> Unfortunately I'm in love with ruby :-) > >>> > >>> All you need is love :) > >>> > >>> Pj. > >> > >> At some point the choice of a language will not matter as much, as long > as > >> it is implemented in a VM (something Perl 5 cannot claim at the moment, > but > >> Perl 6 does with the Parrot VM). > >> > >> chris > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > From pjotr.public14 at thebird.nl Thu Dec 16 08:26:58 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 16 Dec 2010 14:26:58 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: Message-ID: <20101216132658.GA21601@thebird.nl> On Wed, Dec 15, 2010 at 05:02:27PM -0600, John Woods wrote: > Working in a bioinformatics lab, my experience has been that most people go > with Python over Ruby because of SciPy and NumPy (and matplotlib). At least > for us, it's not because of speed at all. If I need something speedy, I code > it in C++ and use Rice to build a Ruby adapter. That is what I always did. Reason to switch to functional languages are that they are stronger with parallelization abstractions (and shared memory). C++ can do parallelization, but it is harder work and more dangerous. > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and > with it, good numerical support). Great! Be good if it supports JRuby! > To that end, we've been working a bit to put together a spec for how a > SciRuby with plotting and numeric capabilities would work. I did a GSL port to Ruby https://github.com/pjotrp/ascilib using SWIG. It may not interest you, but it does support Perl too, and probably soon the JVM. Using SWIG is not that hard, and the advantage is a wider target. > I'm not quite ready to say anything specific about it publicly just yet, but > if you're interested in being involved, please write me back. It is a good thing, thanks, looking forward to it. Pj. From bonnalraoul at ingm.it Thu Dec 16 12:20:53 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 16 Dec 2010 18:20:53 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: Message-ID: <20101216172053.6ae641ea@mail.ingm.it> Hi Claudio, I like your work on statsample. As others pointed out iterators are the keys so probably we need to invest some time in designg. Could we organize this math/stat package like a plugin for bioruby, what do you think? Then about graphics attaching the visualization concept to some object would be great not only static but also for HTML5 +1 for all these topis from me. Rserve can be used efficiently for parallelization, it's ability to have session and user authentication. Distributed Programming with Ruby it's a must. _____ From: Claudio Bustos [mailto:clbustos at gmail.com] To: John Woods [mailto:john.woods at marcottelab.org] Cc: Raoul Bonnal [mailto:bonnalraoul at ingm.it], Chris Fields [mailto:cjfields at illinois.edu], bioruby at lists.open-bio.org Sent: Thu, 16 Dec 2010 04:59:51 +0100 Subject: Re: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) Rubyvis fellows: I'm Claudio, developer of statsample, rubyvis and rserve-client for ruby. I will be very happy if I can help in any way in bioruby project. Statsample: I'll start a Master on Statistics next year, so the library will be actively developed. I'm very interested in implementation of iterator based datasets, to allow statsample process big datasets. Rubyvis: I will finish port of all Layout on January, I think. What about creating a builder for rubyvis to create protovis scripts using the same set of instructions to create static images?. We can have out cake and eat it :) Regards -- Claudio Bustos Mg. en Psicolog?a clbustos at gmail.com From pjotr.public14 at thebird.nl Thu Dec 16 13:22:04 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 16 Dec 2010 19:22:04 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101216172053.6ae641ea@mail.ingm.it> References: <20101216172053.6ae641ea@mail.ingm.it> Message-ID: <20101216182204.GB24908@thebird.nl> On Thu, Dec 16, 2010 at 06:20:53PM +0100, Raoul Bonnal wrote: > +1 for all these topis from me. Topi is a hat http://www.thefreedictionary.com/topi (also in Bengali) I hope everyone can make the first world-wide BioRuby IRC conference on Sunday! I am looking forward to it, and hope to have a 10min screencast on JRuby ready on Saturday. Pj. From bonnalraoul at ingm.it Thu Dec 16 13:47:42 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 16 Dec 2010 19:47:42 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101216182204.GB24908@thebird.nl> References: <20101216172053.6ae641ea@mail.ingm.it> <20101216182204.GB24908@thebird.nl> Message-ID: <5F9A7F53-A292-4E09-8774-2269213AC3EC@ingm.it> On 16/dic/2010, at 19.22, Pjotr Prins wrote: > On Thu, Dec 16, 2010 at 06:20:53PM +0100, Raoul Bonnal wrote: >> +1 for all these topis from me. > > Topi is a hat > > http://www.thefreedictionary.com/topi > > (also in Bengali) Thanks Pjotr. http://vimeo.com/14435288 -- Ra From andrew.j.grimm at gmail.com Sat Dec 18 01:04:37 2010 From: andrew.j.grimm at gmail.com (Andrew Grimm) Date: Sat, 18 Dec 2010 17:04:37 +1100 Subject: [BioRuby] IRC conference archives Message-ID: Will there be archives of the IRC conference? The conference appears to be at 1am Monday Sydney time. If not, I guess I could leave my computer on all night... Andrew From francesco.strozzi at gmail.com Sat Dec 18 02:13:10 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Sat, 18 Dec 2010 08:13:10 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: Message-ID: +1 for this. I will try to connect to the correct time but I may miss the first part. It will be nice to have an archive with the IRC conference and the materials presented (screencasts etc.). Thanks Francesco On Sat, Dec 18, 2010 at 07:04, Andrew Grimm wrote: > Will there be archives of the IRC conference? The conference appears > to be at 1am Monday Sydney time. > > If not, I guess I could leave my computer on all night... > > Andrew > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From pjotr.public14 at thebird.nl Sat Dec 18 07:10:24 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 18 Dec 2010 13:10:24 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: Message-ID: <20101218121024.GA456@thebird.nl> We will set up logging. No worries. On Sat, Dec 18, 2010 at 05:04:37PM +1100, Andrew Grimm wrote: > Will there be archives of the IRC conference? The conference appears > to be at 1am Monday Sydney time. > > If not, I guess I could leave my computer on all night... > > Andrew > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Sat Dec 18 10:59:38 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 18 Dec 2010 16:59:38 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <5F9A7F53-A292-4E09-8774-2269213AC3EC@ingm.it> References: <20101216172053.6ae641ea@mail.ingm.it> <20101216182204.GB24908@thebird.nl> <5F9A7F53-A292-4E09-8774-2269213AC3EC@ingm.it> Message-ID: <20101218155938.GA1756@thebird.nl> Below is the screencast on JRuby Raoul sent: > http://vimeo.com/14435288 It is pretty good, as it shows the steps of running JRuby, and how to leverage JVM tools. Note the following, JRuby has 1. Full threads support 2. Compiles Ruby to bytecode, which is optimized to native code, when run often enough (the exception being 'eval' statements). 3. You get the performance and debugging toolset that come with the JVM, including a profiler that is not much slower than native 4. You can ship Ruby code with all (gem) dependencies included in a JAR, which will run on ANY JVM. Well, Dalvik excepted ;). The main message is that you get Ruby with all Java/JVM goodies. Pj. From pjotr.public14 at thebird.nl Sun Dec 19 07:53:13 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sun, 19 Dec 2010 13:53:13 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: References: Message-ID: <20101219125313.GA6033@thebird.nl> Today is the day!! I use the bitchx client, with logging. Pj. On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: > Dear BioRuby developers, > > Here we announce the first BioRuby world wide IRC conference! > > The topic is: "Current and future BioRuby development" > > The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), > which is 15pm CET (Central European Time) and 9am EST (New York) and > 6am PST (California time). > > You are kindly invited to join! > > Server: irc.freenode.org > Channel: #bioruby > > Toshiaki Katayama on behalf of the BioRuby staff. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From ktym at hgc.jp Sun Dec 19 13:49:36 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Mon, 20 Dec 2010 03:49:36 +0900 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101219125313.GA6033@thebird.nl> References: <20101219125313.GA6033@thebird.nl> Message-ID: <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> Hi all, Today, we had >10 participants and discussed about * Current and future of BioRuby plugin system * BioRuby with JVM for high performance computing * Regularly continue the IRC meeting? etc. Details will be reported from Pjotr. We agreed to continue this IRC meeting on every Thursday as a "core time", and we may also keep the #bioruby channel open for everyone as much as possible, I just setup an IRC bot to log discussions. >From now on, the log of this channel will be made public at http://bioruby.org/irc/ so please join us (and please pay attention to privacy as a netiquette as well ;-). Toshiaki On 2010/12/19, at 21:53, Pjotr Prins wrote: > Today is the day!! > > I use the bitchx client, with logging. > > Pj. > > On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >> Dear BioRuby developers, >> >> Here we announce the first BioRuby world wide IRC conference! >> >> The topic is: "Current and future BioRuby development" >> >> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >> which is 15pm CET (Central European Time) and 9am EST (New York) and >> 6am PST (California time). >> >> You are kindly invited to join! >> >> Server: irc.freenode.org >> Channel: #bioruby >> >> Toshiaki Katayama on behalf of the BioRuby staff. >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Tue Dec 21 04:18:37 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 21 Dec 2010 10:18:37 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101218121024.GA456@thebird.nl> References: <20101218121024.GA456@thebird.nl> Message-ID: <20101221091837.GA21147@thebird.nl> Below the first IRC conference in a long time. We are keeping the #bioruby room open on freenode. Every Thursday 2pm (GMT) we'll try to be online. This Thursday we can look at developing new plugins. For anyone interested in some Ruby programming over the holidays. Pj. IRC log started Sun Dec 19 13:51:39 2010 good "morning"... hi there Hello everyone! the first world wide BioRuby IRC conference :) Hi hi Hi all. Would it be useful to have very quick introductions from everyone? Less than or equal to 10 words? tktym should open first Hmm, I'm Toshiaki Katayama, one of founders of the BioRuby project. Nice to see you. Pjotr Prins: 50% BioRuby, 50% other - next year lots of NGS Jan Aerts. Ruby + beginnings of Clojure. Lots of NGS now + visualization. I'm Naohisa Goto, a core developer, and a release manager of BioRuby My work -- comparative and evolutionary functional genomics, software and algorithm development mostly in Ruby and Java (www.cmzmasek.net) I'm Hiro Mishima, using Ruby and Bioruby for human genetics, genomics, and NGS analysis Mitsuteru Nakao: a BioRuby founder, TogoWS, next year lots of SemanticWeb :-) George Githinji, PhD student, exploring sequence diversity and immunoinformatics next year hello hello, we've just finished self introductions from each other arg -sorry I was stuck with something. Is there a log already? I'm not sure. Pjotr? I am logging my miniintro: I'm mostly a bioruby user. I'm an evolutionary biologist trying to understand the genetic basis of social life in ant colonies.... after getting fed up with perl's weirdness I decided to use bioruby for most of my scripting ant colonies! sounds interesting OK, how do we start discussion? Is Raoul coming on the call? Would be useful for the plugin stuff. Probably, he is in Spain for his vacation. As for the plug-in system, we (ngoto, mn and me) tried to setup bio-geek and started to make the first plugin, last week. How did that go? And what plugin was that? Installation procedure was fine. I thought we should use a name "biogem" instead of the "biogeek". ruby <-> gem ; bioruby <-> biogem for symmetry I'd vote for that as well. and bioruby plugins are actually RubyGems. OK, I'll talk with Raoul on this. Then, biogem (biogeek) command created a template plugin directory structure. It is working very well, but we may need more templates with a lot of comments instead of empty files. the plugin system is looking good from a simple version (ruby only, 1 file) to a complex version (many files and may contain C code) The version we tried is the former, simple one, which was kind of Bio::ColorScheme extension. more info: http://bioruby.open-bio.org/wiki/Plugins but I'm not sure these descriptions are the latest version ;-) It seems not up-to-date obsolete OK, I'll contribute my installation log there. Anyway, my overall feeling about the plugin system was good and it seems a way to go. Additionally, I'd like to discuss about the inclusion of workflows (with data) as plugins. workflows sounds interesting... how do you see that? there are a lot of methods described in journal articles, and probably some of them can be coded in (Bio)Ruby if the procedure can be distributed as a plugin, (with example data) user can try the workflow with ease and with his own data as well That'd be really cool. Would fit in nicely with the new journal Open Research Computation. (disclaimer: am an editor) That'd be really cood -- agreed! integrating git for history? One of the issues will be that many workflows rely an very specific site-specific setups. I think this kind of plugins are something like R's package, which is one of the strength of R and BioConductor. yes, which also allows people to submit to journalx Jan, yes, but we can start with generic ones. I think what is required now is a few examples NGS-related would be nice :-) But that is at the moment not the strongest field in bioruby... Something phylo? gff3 yes, NGS with stats and/or visualization would be a candidate. GFF3 too. what NGS formats are you using? products of Google Summer of Code 2010: NeXML, gene duplocation algorithm fastq -> sam/bam -> pileup or VCF. @pjotr: biolib? biolib has sam/bam (samtools) biojava too biolib version is not complete What was in my mind : analysis methods (>200?) implemented in G-language http://www.g-language.org/wiki/documents picard = java library on top of samtools All: I have to rush off, unfortunately. @pjotr: will you send us the logs? yes - both biolib and picard are samtools I have made screencasts for JRuby but I have trouble getting the sound track right silent movie now anyway, coming soon can we watch the silent version? heh maybe not that good, it needs info OK I could not spare time to watch the movie, Raoul sent us a link before. http://vimeo.com/14435288 Hmm, "Rails on the JVM with JRuby" it is mostly about JRuby and tools worth watching? it seems the length of the movie > 1h! depends it gives an idea of JRuby and the JVM tools I think the JVM is a good way forward for BioRuby for BioRuby developers or users? both developers benefit from existing libraries and tools picard a point in case users benefit from packaged jars you can package everything together, including Is a ruby program packaged by using jar? jruby, scala, biojava, bioscala and gems (!) I'm still not sure because I suppose that most of BioRuby users are not that geeks ;) yes - that you can see in the video, all in one tktym: it is easy for the users, even one-click installs on most platforms OK, potentially it sounds very interesting. only thing that causes problems are gems with C libs but, I think we can avoid that for most use cases also the JVM is very fast these days there is no reason to program in C any more, except a few border line cases so, what kind of help do you need? I am definately heading this way for big data no help, per se I'm still not sure because I suppose that most of BioRuby users are not that geeks -- I would very about too... I am interested in finding people who are interested ChristianZmasek: what are you saying? Anyway, we can try both with plugins. Some plugins for high-end users can depend on JVM to BioScala but we can still use BioRuby as it is now. sure we can take it case by case right ruby 1.9 is pretty fast jruby will win with parallel programming that is my main focus pjotr: to me it sounds as if these recent proposals (also on the mailinglist) are moving away from what many like about Ruby: simplicity and elegance. ChristianZmasek: I think bioruby on its own is a lost cause we just don't have the momentum biopython is going to be successful bioperl may live for some time biojava is doing really well so where are we? wouldn't worry that by spreading out the few developers over more "projects" would make it even worse? ultimately bioruby is interpreter agnostic don't you think it is possible to speed up many/some things from within bioruby (more effiecient coding) than relying on numerous "outside" tools? a lot needs to happen with bioruby, to make it suitable for big data not only that I remember somebody writing about some module for numeric methonds in ruby -- that sound interesting... ruby/python/perl are not fast enough for churning 1TB (it is not only an IO problem) rbnum would be useful, though we have quite a bit of that already ruby/python/perl are not fast enough for 1TB -- agreed! Wouldn't even think about using scripting languages for that amount of data... how about Java/JVM? Java/Scala are strongly typed that means they can be highly optimized for speed The JVM used to be bad with memory but that has mostly gone now but I predict that people will go by use cases I feel the amount of data itself is not a problem of coding. We can use DBs and indexing systems optimized for that. If so, all we need inside bioruby is binding for those external data store? if I write a fast GFF3 Scala lib, that you can use from bioruby are you going to reimplement that? tktym: that requires DBs etc. do you really think a JVM dependency is harder for users? I always use DBs or TokyoCabinet or other indexing systems for speed. Most machines have a JVM now. 1. is it user friendly? anyway I wrote: Difficulty of depending JVM is something equivalent to have external DBs for ordinary users in this context. It depends, and it would be a requirement for some cases. sorry, pressed the wrong key ;) I think JVM is one way to go, but I also suppose plugins and workflow applications will attract people if we can focus our efforts to making really useful ones for daily research. yes Anyway, we can continue both approach. i disagree that JVM makes it harder. It makes it even easier that what we have now you'll see. OK, I'm looking forward your movie :) well, they are dev focussed yes I'll need to make another one (#4) ;) We spent some time on the topic of JVM, I'd like to take from others. yes Just for statistics and to plan the next IRC meeting, can I ask where are you connecting from? tktym, kozo2, ngoto, mn are from Japan. mishima as well? I remember someone from Australia wanted to join this meeting if the time is sane. :) (As I joined conferences in Japan for past two weeks, I could not reply the thread on the mailing list but I just finished to read them yesterday.) We have 12 members (+2 raoul and jan) on this channel. i'm in jakarta indonesia COOL 30 degrees celcius actually :) I am near Amsterdam, Netherlands. hi welcome :) I am late, sorry : ( Today, we talked about BioRuby plugin system (current status and future) and BioRuby with JVM for high performance computing. Probably, Pjotr will put a log on the web? I am in California tktym: yes ok Chris, what time in your time zone? ChristianZmasek: early bird I can be productive today! yannickwurm: what are your bioruby needs? pjotr: work with ants is a bit different because you need to do everything yourself (eg: there is no established GO annotation) are you sequencing yannickwurm: that's the same in livestock animals for me so I've been parsing a lot of stuff (blasts...) and creating small pipelines to stick different tools together, and sometimes run things parallel (using only ruby threads for now... maybe the rqueue in the future) we sequenced and assembled our genome (will be published in PNAS in january :)) we all seem to be doing pipelines so have been dealing more and more with Illumina sequence (fastq), 4 sequence (SFF), their assembly, their mapping (for RNAseq stuff) I am in "Pacific Time Zone" yannickwurm: from Indonesia? I'm usually at the borderline between "shell script" and "pipeline" (most are one-off) I'm a postdoc in switzerland but in september my partner and I moved to indonesia ah, holiday too quick ;) no - well be here for a year or 2 I'm still using the Swiss cluster though (www.vital-it.ch) :) yannickwurm: I am Swiss as well gruetzi :) I have lived in Bangladesh, Italy and UK Hoi! piotr: n Italy also? Where? (I'm italian) Roma My wife was working with UN FAO May happen again I see Where are you in Italy? Milano Near Raoul near Raoul :-) Should meet up at ISMB Vienna I am planning to take the motor bike - Guzzi in july? Is with ECCB2011 if I remember yannickwurm: where in Indonesia it is kinda big jakarta crazy yannickwurm: I have an Indonesian friend there biorubyist? :) I'll hire him :p heh, no programmer we'll take this offline :) It is getting late for some of us (Japan) Yep. 1AM now Maybe we should do this more often? Shall we regularly continue this kind of meeting on IRC? yes, I have got to get going too yannickwurm: quick question do you know about the Hymenoptera Anatomy Ontology? (looking at your solenopsis now) who would not join in next time? mjy: heard about it, but I'm not a taxonomist so haven't pushed much to find out more the thing is yannickwurm: we should chat sometime, it's much than taxonomy the 'core' team does meet up occasionaly online erm much *more* we feel it is useful to broaden the meetings but the timezones are difficult pjotr: with fixed agenda? or as 'bioruby outreach'? I think of it as a meeting of minds so, it is more outreach ok will you send a copy of the log through the mailing list ? already it is useful francesco_strozz: sure already it is useful -- yes one thing we could do is weeklies keep IRC going anyway, but meet up, say one day a week tktym: what is a good time for you, to be logged in? ngoto: same probably not Friday night :) ! whoops maybe Thursday? We are basically online and we can setup a IRC bot to always keep this channel open and to make the log on web (that we had been doing before the Skype era), so all we need to set "core time" - Thursday maybe fine. 11pm JST, 3pm CET, 6am PST every Thursday, does not have to be active conversation, just representation of BioRuby Time to discuss developments, frustrations, and encourage new developments? ChristianZmasek: OK? and "I'm a n00b user why doesnt XXX work?" ? exactly anything :) that's sounds good :-) OK -- Th morning (for me)! that would be good - and important to advertise on the website the key point is, the conversation log will be made public on the web because the first "im going to try something" is when people either stay or leave we need to recruit new contributors, and find ways of leveraging existing code and projects I hope everyone on this channel will help answering those newbie questions :) like rtfm rtfc is more likely with BioRuby ;) Students may have been exposed to RoR so, yes. Hehe - anyway I really gotta run. I would like to just say that I am thankful for all the efforts that have been and continue to be put into bioruby. While many aspects clearly have potential for improvement, I differ with Pjotr's view that bioruby may be a lost cause. My vision is limited, but I have the feeling that bioruby has a solid base and hope that positive energy will lead to more contributions & improvement in the code, as well as m visibility thanks & good night! \quit() eh ok i'm going to control-c piotr2: sorry if this was already discussed but, why don't put (maybe on the website), the "open issues", so libraries that need improvements or new libraries to be written so people can see what is needed and maybe propose to help or write parts of bioruby ? and important to advertise on the website -- agreed francesco: https://github.com/bioruby/bioruby/issues it would be good, yes, but experience tells me it is hard for people to (start) contributing existing issues can be found on the above page, tktym: didn't know???.thanks :-) assigning "new libraries to be written" for others is a hard task There is more to it: I have quite a few branches of bioruby myself, which never make it into mainline we need to mentor them, we need to know their ability and motivation etc. that is one reason to have plugins some such chances were Google SoC so we can have 'experimental' code with bioruby tktym: yes I know, can be a hard task I think development is driven by the 'itch' that needs scratching pj: yes, I believe plugin system will casually invite more people to contribute their code to bioruby it is easy to fork bioruby or plugins but it does not necessarily translate into active and coordinated contribution I still need to write to promised phylogenetics related tutorials... :x ChristianZmasek: I would like that :) but let's continue with these IRC conferences, at least to encourage contribution we are a social species... like ant... :) Is there a "use-case" list for desired bioruby features? I still need to do some cleanup on the Google Summer of Code - "duplications" project and then it needs to be merged... :x mjy: probably, no merging is the hard bit (mad laughter) anyway... mjy: http://bioruby.open-bio.org/wiki/Plugins#Candidate_Whish_List let me ask, who would like to contribute to BioRuby itself here? me me I would at some point- don't know if I have the skills yet :) me (if I meet the required skills, otherwise I will improve skills and then come back to say "me") heh, that is quite a few who have not commited yet I don't think skills should be that critical we all learn as we try plugins can help - even to replace core code maybe each of us should start thinking in terms of creating a plugin yes pjotrp: am I correct in surmising that plugins are just code with a Bioruby namespace? mjy: these items in the plugin wish list are mainly libraries for file formats and external programs, but I would like to propose researchers to provide "workflow" plugins to reproduce the scientific procedure. yes just read the "What is a plugin" it can be really loosely connected, or tight bioruby I wish to encourage you to come up with an idea for a plugin we will support you pjotrp: ok- so I have several gems that I used jeweler for- it would be nice there was a "jeweler" for bioruby plugins if you konw what I mean tktym: I think workflows could be a great plus???.if you think about all the pipelines we need to write to run, for example, sequencing data That will come. francesco: yes, that will be a one of the goals :) tktym: is the idea gridengine or TORQUE? to cluster compute stuff? tktym: I think workflows could be a great plus??? yes! tktym: a workflow system design could also be a paper anyway, let's continue Thursday? Currently, real usages of the bioruby library are buried in the test codes, but we can bring it to front as a plugin. I'm fine with Thursday Besides, I'll setup "nadoka" http://www.atdot.net/nadoka/ as before. bye! And thanks! which is a IRC bot written in Ruby :) by the core developer of Ruby-1.9, Sasada-san !! I'll get the audio on the screen casts done bye, see on the next irc call bye for now bye and thanks! hello bioruby hello again (testing) OK, I think http://bioruby.org/irc/ is ready for you :) Updated http://bioruby.open-bio.org/wiki/BioRuby_IRC_conference Updated http://bioruby.open-bio.org/wiki/Plugins#Develop_a_plugin and added http://bioruby.open-bio.org/wiki/BiogemInstallation test IRC log ended Sun Dec 19 22:50:34 2010 From bonnalraoul at ingm.it Tue Dec 21 05:13:40 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 11:13:40 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> Message-ID: <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> Dear John, do you have an idea, plans on how to arrange all this stuffs ? During this holiday I'll work on the plugin system, code name bio-gem. Googling around I found this framework http://apotomo.de/ which seems to be very promising for building rich web applications. Why using widget ? Suppose to associate a widget to a biodata you can publish is immediately or interact with it easily, just an idea. I think there is another widget for rails but I can't remember the name. On 16/dic/2010, at 01.50, John Woods wrote: > I saw ruby-statsample, and it looks like one of many things we'd (ideally) like to integrate. > > I'm glad you brought up publishing to the web. We spent a not-insignificant amount of time debating graphics platform, and ultimately we decided that HTML5 canvas, and SVG-type graphics, were probably the ideal. For the moment, we're looking at protovis. Interestingly, Claudio Bustos has been working on a Ruby protovis port (he also wrote ruby-statsample, I believe). > > The other big piece is NArray. Masahiro Tanaka is apparently rewriting it with a goal of getting it incorporated into Ruby core. John Prince has been working on documentation for the existing NArray, in part to see if it's got everything SciRuby would need. But ultimately, the NArray rewrite/core integration is the key. It's years from being finished, supposedly -- but perhaps if there are some eager volunteers we could speed that up. =) > > So it seems like most of the pieces are present -- it's just a matter of tying them together. > > And I just said I wasn't going to get into detail -- but there, you just got me going. =) > > John > > On Wed, Dec 15, 2010 at 6:29 PM, Raoul Bonnal wrote: > +1 for me. > Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there is also an optimized version. > > John, in the web era a way to easily publish plots on the web not just as static images would be great. > > Probably I could not attend the first meeting, work trip. Btw if someone is around these day I'll try to be in the channel. > > Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: > > > Hi everyone, > > > > I wanted to put in my two cents about this (not this specific email, but the > > thread). > > > > I come from a CS background, and so when I began working in bioinformatics, > > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby (and > > Rails!). > > > > Working in a bioinformatics lab, my experience has been that most people go > > with Python over Ruby because of SciPy and NumPy (and matplotlib). At least > > for us, it's not because of speed at all. If I need something speedy, I code > > it in C++ and use Rice to build a Ruby adapter. > > > > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been > > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and > > with it, good numerical support). > > > > To that end, we've been working a bit to put together a spec for how a > > SciRuby with plotting and numeric capabilities would work. > > > > I'm not quite ready to say anything specific about it publicly just yet, but > > if you're interested in being involved, please write me back. > > > > Best, > > John O. Woods > > NSF Fellow > > The University of Texas at Austin > > > > > > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields wrote: > > > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > >> > >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > >>>>> Part of how we try to handle big data files in Biopython is using > >>>>> Python iterators, whereby the file is loaded record by record (how > >>>>> depends on the file format - for BLAST we do this query by query), > >>>>> not all into memory in one go. I think BioPerl does something very > >>>>> similar in their parsers, I'm not so familiar with BioJava. > >>> > >>> BioJava uses a visitor pattern. In effect an iterator. > >>> > >>> With all current implementations IO runs, then code, the IO, etc. > >>> While we are IO constrained, we are actually doing worse. > >>> > >>> What I want is an IO thread going at maximum throughput. Every item > >>> should get parcelled out for further parsing and processing, in > >>> parallel to the IO thread. > >>> > >>> We should do better, and make it a generalization. I think we can do > >>> it by using Scala and the standard BioJava iterators. With Scala it > >>> can be turned in a parallelized iterator. That is a fun project. > >>> > >>>> From my point of view Python guys are doing a very good job on all > >> fields. > >>>> Unfortunately I'm in love with ruby :-) > >>> > >>> All you need is love :) > >>> > >>> Pj. > >> > >> At some point the choice of a language will not matter as much, as long as > >> it is implemented in a VM (something Perl 5 cannot claim at the moment, but > >> Perl 6 does with the Parrot VM). > >> > >> chris > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > From biopython at maubp.freeserve.co.uk Tue Dec 21 06:15:34 2010 From: biopython at maubp.freeserve.co.uk (Peter) Date: Tue, 21 Dec 2010 11:15:34 +0000 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101221091837.GA21147@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: On Tue, Dec 21, 2010 at 9:18 AM, Pjotr Prins wrote: > > Below the first IRC conference in a long time. We are keeping the > #bioruby room open on freenode. Every Thursday 2pm (GMT) we'll try to > be online. > > This Thursday we can look at developing new plugins. For anyone > interested in some Ruby programming over the holidays. > > Pj. > > IRC log started Sun Dec 19 13:51:39 2010 > good "morning"... > ... > Additionally, I'd like to discuss about the inclusion of workflows (with data) as plugins. > workflows sounds interesting... > how do you see that? > there are a lot of methods described in journal articles, and probably some of them can be coded in (Bio)Ruby > if the procedure can be distributed as a plugin, > (with example data) > user can try the workflow with ease and with his own data as well > ... Hi all, I just read over the IRC log, and one of the topics that was discussed was workflows (in the form of BioRuby plugins). Looking at workflows a different way, perhaps there are some existing BioRuby scripts which might work really well as in workflow tool in a framework like Taverna or Galaxy? http://www.taverna.org.uk/ http://usegalaxy.org/ We're using Galaxy where I work, and it comes with tool wrappers covering Python, Perl, and R scripts, plus compiled code. I'm sure one could call (Bio)Ruby scripts too. Peter From benjamin.q.almeida at gmail.com Tue Dec 21 06:38:38 2010 From: benjamin.q.almeida at gmail.com (benjamin Almeida) Date: Tue, 21 Dec 2010 12:38:38 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: Hi all, thanks for the log, I couldn't make it, but I'll try thursday as of workflows I kind a like wrapping things in objects and just ask my way through delegate dependencies, but I would join forces on a nice workflow tool in ruby. galaxy is used here (Vienna) too ben From bonnalraoul at ingm.it Tue Dec 21 07:33:18 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 13:33:18 +0100 Subject: [BioRuby] NGS, transcriptome, workflows Message-ID: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> Hi all, I read in the irc's log that a lot of the memebers are or will start working on ngs data. I'll re-start ( I worked on 454) with the Illumina platform, in few week, do you have some consolidate workflow to follow for transcriptome analysis ? papers, blogs, etc.. everything is ok :-) I'm also interested about the workflow discussion but... are the workflow intended to let the not bioinformaticians analyze complex datasets or automate some tedious and repetitive task that we (bioinformaticians) must do every time ? In both cases our life will be probably better :-) -- Ra From missy at be.to Tue Dec 21 09:29:02 2010 From: missy at be.to (MISHIMA, Hiroyuki) Date: Tue, 21 Dec 2010 23:29:02 +0900 Subject: [BioRuby] NGS, transcriptome, workflows In-Reply-To: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> Message-ID: <4D10B9AE.2010206@be.to> Hi all, I would like to say something about workflow automation in Ruby. Recently I am interested in using Parallel Workflow extention for Rake (Pwrake) for NextGen sequencer data processing. As you may know, Rake is Ruby Make, a build tool. Pwrake is developped by Masahiro Tanaka at University of Tsukuba. He is also the author of "NArray", very fast matrix calculation engine for Ruby. Although Pwrake and regular Rake are compatible in syntax, Pwrake automatically detects workflow steps that can be run in parallel. Pwrake's parallelization model is "process based". Because I am just a *user* of bioinformatics packages (like BWA/GATK/DINDEL etc..), it is what I need. Pwrake invokes processes via ssh and supports the Gfarm large-scale distributed filesystem. Of course, it works well on a multi-processor Linux box. Although Pwrake is developed for astronomy science, its goal is also common in bioinformatics. I think that some helper methods may simplify Rakefiles for bioinformatics, and such helper methods are good for a BioRuby plugin. FYI, Pwrake on github: https://github.com/masa16/Pwrake/ Presentation at RubyConfX: http://www.slideshare.net/masa16tanaka/ruby-conftanaka16 Presentation at PRAGMA18: http://goc.pragma-grid.net/pragma-doc/pragma18/Cool_Things/pwrake.pptx Thanks, Hiro. Raoul Bonnal wrote(2010/12/21 21:33): > Hi all, I read in the irc's log that a lot of the memebers are or > will start working on ngs data. I'll re-start ( I worked on 454) with > the Illumina platform, in few week, do you have some consolidate > workflow to follow for transcriptome analysis ? > > papers, blogs, etc.. everything is ok :-) > > I'm also interested about the workflow discussion but... are the > workflow intended to let the not bioinformaticians analyze complex > datasets or automate some tedious and repetitive task that we > (bioinformaticians) must do every time ? In both cases our life will > be probably better :-) > > -- Ra -- MISHIMA, Hiroyuki, DDS, Ph.D. COE Research Fellow Department of Human Genetics Nagasaki University Graduate School of Biomedical Sciences From bonnalraoul at ingm.it Tue Dec 21 10:44:03 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 16:44:03 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: <52CB9CE6-ABE6-4FA9-8983-1F4ECBC6FD49@ingm.it> Hello Guys, something that has not been mentioned during the conference is the possibility to create meta plugins/gems specific for a particular field or release. Pjotr had the original idea because he was worried about compatibility between the plugins and the difference releases of bioruby. He invented the term blessed and we are thinking to mark as blessed the most important plugins, "certified" for working with a specific version of bioruby. Probably use have already discussed this but it's also possible to create meta packages for NGS, for workflows, for imaging, for parallel computation, for JVM (some plugin installable only if a JVM in available) I think that these meta packages, at least at the beginning will be supervised by core developers, just to start and organize them but in the future other people could takeover them. Another important thing is to avoid the proliferation of namespaces, try to look at the core bioruby and find if the your namespace fits in some place, then it would be more easy for the people find the code, classes and modules logically. -- Ra : ml Helius : irc helios : githup ilpuccio : skype On 21/dic/2010, at 12.38, benjamin Almeida wrote: > Hi all, > thanks for the log, I couldn't make it, but I'll try thursday > as of workflows I kind a like wrapping things in objects and just ask my way > through delegate dependencies, but I would join forces on a nice workflow > tool in ruby. > galaxy is used here (Vienna) too > > ben > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From john.woods at marcottelab.org Tue Dec 21 11:46:08 2010 From: john.woods at marcottelab.org (John Woods) Date: Tue, 21 Dec 2010 11:46:08 -0500 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> Message-ID: Apotomo looks like it could be very useful. I think widgets are definitely the way to go. One thing I want to make clear is that although many of us are biologists, SciRuby should be a standalone project. This is not simply intended as a bioinformatics tool. It should be something general purpose, that any scientist/statistician would feel comfortable using. With that said, I think it's appropriate to write plugins that bridge BioRuby and SciRuby. Can you be more specific with your question? I'm planning on taking a look at what we already have later today and updating the spec. (Unfortunately, our lab website is being upgraded, so I can't link you to the current spec right now.) Incidentally, I just set up a Google Group for SciRuby development. It's public. http://groups.google.com/group/sciruby-dev Best, John On Tue, Dec 21, 2010 at 5:13 AM, Raoul Bonnal wrote: > Dear John, > do you have an idea, plans on how to arrange all this stuffs ? > During this holiday I'll work on the plugin system, code name bio-gem. > > Googling around I found this framework http://apotomo.de/ which seems to > be very promising for building rich web applications. > Why using widget ? > Suppose to associate a widget to a biodata you can publish is immediately > or interact with it easily, just an idea. > I think there is another widget for rails but I can't remember the name. > > On 16/dic/2010, at 01.50, John Woods wrote: > > I saw ruby-statsample, and it looks like one of many things we'd (ideally) > like to integrate. > > I'm glad you brought up publishing to the web. We spent a not-insignificant > amount of time debating graphics platform, and ultimately we decided that > HTML5 canvas, and SVG-type graphics, were probably the ideal. For the > moment, we're looking at protovis. Interestingly, Claudio Bustos has been > working on a Ruby protovis port (he also > wrote ruby-statsample, I believe). > > The other big piece is NArray. Masahiro Tanaka is apparently rewriting it > with a goal of getting it incorporated into Ruby core. John Prince has been > working on documentation for the existing NArray, in part to see if it's got > everything SciRuby would need. But ultimately, the NArray rewrite/core > integration is the key. It's years from being finished, supposedly -- but > perhaps if there are some eager volunteers we could speed that up. =) > > So it seems like most of the pieces are present -- it's just a matter of > tying them together. > > And I just said I wasn't going to get into detail -- but there, you just > got me going. =) > > John > > On Wed, Dec 15, 2010 at 6:29 PM, Raoul Bonnal wrote: > >> +1 for me. >> Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there >> is also an optimized version. >> >> John, in the web era a way to easily publish plots on the web not just as >> static images would be great. >> >> Probably I could not attend the first meeting, work trip. Btw if someone >> is around these day I'll try to be in the channel. >> >> Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: >> >> > Hi everyone, >> > >> > I wanted to put in my two cents about this (not this specific email, but >> the >> > thread). >> > >> > I come from a CS background, and so when I began working in >> bioinformatics, >> > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby >> (and >> > Rails!). >> > >> > Working in a bioinformatics lab, my experience has been that most people >> go >> > with Python over Ruby because of SciPy and NumPy (and matplotlib). At >> least >> > for us, it's not because of speed at all. If I need something speedy, I >> code >> > it in C++ and use Rice to build a Ruby adapter. >> > >> > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have >> been >> > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby >> (and >> > with it, good numerical support). >> > >> > To that end, we've been working a bit to put together a spec for how a >> > SciRuby with plotting and numeric capabilities would work. >> > >> > I'm not quite ready to say anything specific about it publicly just yet, >> but >> > if you're interested in being involved, please write me back. >> > >> > Best, >> > John O. Woods >> > NSF Fellow >> > The University of Texas at Austin >> > >> > >> > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields >> wrote: >> > >> >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: >> >> >> >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: >> >>>>> Part of how we try to handle big data files in Biopython is using >> >>>>> Python iterators, whereby the file is loaded record by record (how >> >>>>> depends on the file format - for BLAST we do this query by query), >> >>>>> not all into memory in one go. I think BioPerl does something very >> >>>>> similar in their parsers, I'm not so familiar with BioJava. >> >>> >> >>> BioJava uses a visitor pattern. In effect an iterator. >> >>> >> >>> With all current implementations IO runs, then code, the IO, etc. >> >>> While we are IO constrained, we are actually doing worse. >> >>> >> >>> What I want is an IO thread going at maximum throughput. Every item >> >>> should get parcelled out for further parsing and processing, in >> >>> parallel to the IO thread. >> >>> >> >>> We should do better, and make it a generalization. I think we can do >> >>> it by using Scala and the standard BioJava iterators. With Scala it >> >>> can be turned in a parallelized iterator. That is a fun project. >> >>> >> >>>> From my point of view Python guys are doing a very good job on all >> >> fields. >> >>>> Unfortunately I'm in love with ruby :-) >> >>> >> >>> All you need is love :) >> >>> >> >>> Pj. >> >> >> >> At some point the choice of a language will not matter as much, as long >> as >> >> it is implemented in a VM (something Perl 5 cannot claim at the moment, >> but >> >> Perl 6 does with the Parrot VM). >> >> >> >> chris >> >> _______________________________________________ >> >> BioRuby Project - http://www.bioruby.org/ >> >> BioRuby mailing list >> >> BioRuby at lists.open-bio.org >> >> http://lists.open-bio.org/mailman/listinfo/bioruby >> >> >> > _______________________________________________ >> > BioRuby Project - http://www.bioruby.org/ >> > BioRuby mailing list >> > BioRuby at lists.open-bio.org >> > http://lists.open-bio.org/mailman/listinfo/bioruby >> >> >> > > From bonnalraoul at ingm.it Tue Dec 21 12:02:06 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 18:02:06 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> Message-ID: <3B4E0689-EAB1-4079-9137-A21D560BF953@ingm.it> Dear John, On 21/dic/2010, at 17.46, John Woods wrote: > Apotomo looks like it could be very useful. I think widgets are definitely the way to go. > > One thing I want to make clear is that although many of us are biologists, SciRuby should be a standalone project. This is not simply intended as a bioinformatics tool. It should be something general purpose, that any scientist/statistician would feel comfortable using. With that said, I think it's appropriate to write plugins that bridge BioRuby and SciRuby. It's fine and logical, i agree with you. > Can you be more specific with your question? I'm planning on taking a look at what we already have later today and updating the spec. (Unfortunately, our lab website is being upgraded, so I can't link you to the current spec right now.) I mean a road map, but I'll wait for the new web site. > > Incidentally, I just set up a Google Group for SciRuby development. It's public. > http://groups.google.com/group/sciruby-dev Good to know. -- Ra From pjotr.public14 at thebird.nl Tue Dec 21 12:04:51 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 21 Dec 2010 18:04:51 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101221091837.GA21147@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: <20101221170451.GA25645@thebird.nl> On Tue, Dec 21, 2010 at 10:18:37AM +0100, Pjotr Prins wrote: > This Thursday we can look at developing new plugins. For anyone > interested in some Ruby programming over the holidays. Some of the list members are interested in contributing new bioruby code. I would like to ask here for candidates for new plugins. My vote would be for example: - bio-blast: BLAST parser plugin based on libxml that does not load everything in RAM - bio-samtools: Samtools plugin (based on biolib?) Who would be interested in writing that? I can help with the implementation and design, if required. Any other ideas here? Pj. From bonnalraoul at ingm.it Tue Dec 21 12:50:24 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 18:50:24 +0100 Subject: [BioRuby] Workflows and Parallelization In-Reply-To: <4D10B9AE.2010206@be.to> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> <4D10B9AE.2010206@be.to> Message-ID: <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> My original request was on analysis approach on NGS and transcriptome. btw Very interesting Hiro, searching on github there is also this project: https://github.com/grosser/parallel Run any code in parallel Processes(> use all CPUs) or Threads(> speedup blocking operations). Best suited for map-reduce or e.g. parallel downloads/uploads. Processes Speedup through multiple CPUs Speedup for blocking operations Protects global data Extra memory used ( very low on REE through copy_on_write_friendly ) Child processes are killed when your main process is killed through Ctrl+c or kill -2 Threads Speedup for blocking operations Global data can be modified No extra memory used Processes/Threads are workers, they grab the next piece of work when they finish On 21/dic/2010, at 15.29, MISHIMA, Hiroyuki wrote: > Hi all, > > I would like to say something about workflow automation in Ruby. > > Recently I am interested in using Parallel Workflow extention for Rake > (Pwrake) for NextGen sequencer data processing. As you may know, Rake is Ruby Make, a build tool. > > Pwrake is developped by Masahiro Tanaka at University of Tsukuba. He is > also the author of "NArray", very fast matrix calculation engine for > Ruby. Although Pwrake and regular Rake are compatible in syntax, Pwrake > automatically detects workflow steps that can be run in parallel. > > Pwrake's parallelization model is "process based". Because I am just a > *user* of bioinformatics packages (like BWA/GATK/DINDEL etc..), it is > what I need. > > Pwrake invokes processes via ssh and supports the Gfarm large-scale > distributed filesystem. Of course, it works well on a multi-processor > Linux box. > > Although Pwrake is developed for astronomy science, its goal is also > common in bioinformatics. > > I think that some helper methods may simplify Rakefiles for bioinformatics, and such helper methods are good for a BioRuby plugin. > > FYI, > Pwrake on github: > https://github.com/masa16/Pwrake/ > > Presentation at RubyConfX: > http://www.slideshare.net/masa16tanaka/ruby-conftanaka16 > > Presentation at PRAGMA18: > http://goc.pragma-grid.net/pragma-doc/pragma18/Cool_Things/pwrake.pptx > > Thanks, > Hiro. > > Raoul Bonnal wrote(2010/12/21 21:33): >> Hi all, I read in the irc's log that a lot of the memebers are or >> will start working on ngs data. I'll re-start ( I worked on 454) with >> the Illumina platform, in few week, do you have some consolidate >> workflow to follow for transcriptome analysis ? >> >> papers, blogs, etc.. everything is ok :-) >> >> I'm also interested about the workflow discussion but... are the >> workflow intended to let the not bioinformaticians analyze complex >> datasets or automate some tedious and repetitive task that we >> (bioinformaticians) must do every time ? In both cases our life will >> be probably better :-) >> >> -- Ra > -- > MISHIMA, Hiroyuki, DDS, Ph.D. > COE Research Fellow > Department of Human Genetics > Nagasaki University Graduate School of Biomedical Sciences > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Wed Dec 22 05:13:44 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 22 Dec 2010 11:13:44 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101221170451.GA25645@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> Message-ID: <20101222101344.GB29739@thebird.nl> Here my first 4 min. screencast on BioRuby on the JVM: http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm (temporary download link) I'll try to put a few more together in the coming month(s): 2) Calling JRuby from Scala (8 min) 3) Calling BioJava from JRuby and Jython (8 min) 4) JRuby parallelization using Actors (BioRuby performance) And, maybe later (partly depending on demand) 5) One-click install of JRuby applications on non-Ruby machines 6) JRuby BioRuby plugins 7) Using C modules from JRuby 8) Using JVM profiling and debugging tools with JRuby 9) Clustering BioRuby with Terracotta Now, if someone could clone me... Pj. From yannick.wurm at unil.ch Wed Dec 22 05:06:20 2010 From: yannick.wurm at unil.ch (Yannick Wurm) Date: Wed, 22 Dec 2010 17:06:20 +0700 Subject: [BioRuby] rake Message-ID: <6CFECABA-E9B4-40CE-9B3B-F1AFBAA38031@unil.ch> Hiro-san's comment about Pwrake for workflows convinced me to finally have a look at Rake. So I was trying to make a Rakefile that gives me a tree from the gblocks run on the codon alignment from the backtranslated protein alignment from the protein sequence from nucleotide squences. But I think I quickly bumped into a killer limitation with rules: that they apparently only do one level of inference And I quote from http://onestepback.org/articles/buildingwithrake/rulelimitations.html > > rule ".c" => [".y"] do |t| yacc(t.source) end > rule ".o" => [".c"] do |t| compile_c(t.source) end > If lex.y exists ? Rake will not Build lex.c from lex.y and lex.o from lex.c. Is this limitation still true? Or is something else wrong with my code? Do you have a workaround? And question two: lets say I want a generic rule that cleans up fasta sequences (so that if I require 'aasdflkjsalkfjasdlkj.fasta', the file aasdflkjsalkfjasdlkj should be run through Emboss' seqret. Is that possible? The following hasn't been working: > rule ".fasta" => "" do |task| > sh "seqret -sequence #{task.prerequisites.join} -outseq #{task.name}" > end Thanks & kind regards, Yannick From missy at be.to Wed Dec 22 20:11:42 2010 From: missy at be.to (MISHIMA, Hiroyuki) Date: Thu, 23 Dec 2010 10:11:42 +0900 Subject: [BioRuby] Workflows and Parallelization In-Reply-To: <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> <4D10B9AE.2010206@be.to> <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> Message-ID: <4D12A1CE.4040702@be.to> Hi Raoul and all, Unfortunately I have not had experience in analyzing transcriptome using NGS. Recently I am working on exome mutation analysis and micro RNA analysis using NGS. I will let you know if I find good material for NGS + transcriptome analysis. Thank you for information about the Parallel library. It can be foundations for the workflow management. And I would like to add Raoul's comment about merits of workflow automation. I think it has three merits. First and second are as Rauol mentioned. It is good for wet-side researchers to simplify the complex tasks, and it is good for dry-side researchers to simplify the repetitive tasks. Third is that it is good for workflow sharing. My ideas are like the followings: Rspec (Ruby's test workframe) is the executable specification. It is better than paper specification. Rakefile (or some scripts) can be executable experiment note. It must be better than paper experiment note. And it is good to share protocols in "experiment notes". By the way, I afraid that talking about Rake/Pwrake and workflow automation in the BioRuby mailing list may be kind of out-of-focus. Is it OK? Thanks, Hiro. Raoul Bonnal wrote (2010/12/22 2:50): > My original request was on analysis approach on NGS and transcriptome. > > btw > > Very interesting Hiro, > > searching on github there is also this project: > https://github.com/grosser/parallel -- MISHIMA, Hiroyuki, DDS, Ph.D. COE Research Fellow Department of Human Genetics Nagasaki University Graduate School of Biomedical Sciences From missy at be.to Wed Dec 22 20:13:16 2010 From: missy at be.to (MISHIMA, Hiroyuki) Date: Thu, 23 Dec 2010 10:13:16 +0900 Subject: [BioRuby] rake In-Reply-To: <6CFECABA-E9B4-40CE-9B3B-F1AFBAA38031@unil.ch> References: <6CFECABA-E9B4-40CE-9B3B-F1AFBAA38031@unil.ch> Message-ID: <4D12A22C.5010602@be.to> Hi Yannick and all, I will try to answer your questions.... Yannick Wurm wrote (2010/12/22 19:06): > Hiro-san's comment about Pwrake for workflows convinced me to > finally have a look at Rake. > > So I was trying to make a Rakefile that gives me a tree from the > gblocks run on the cordon alignment from the backtranslated protein > alignment from the protein sequence from nucleotide squences. But I > think I quickly bumped into a killer limitation with rules: that > they apparently only do one level of inference > > And I quote from > http://onestepback.org/articles/buildingwithrake/rulelimitations.html > >> rule ".c" => [".y"] do |t| yacc(t.source) end >> rule ".o" => [".c"] do |t| compile_c(t.source) end >> If lex.y exists >> ? Rake will not Build lex.c from lex.y and lex.o from lex.c. > > Is this limitation still true? Or is something else wrong with my > code? Do you have a workaround? Rakefile: rule ".c" => [".y"] do |t| puts "run yacc(#{t.source})" touch t.source.ext('c') end rule ".o" => [".c"] do |t| puts "run compile_c(#{t.source})" touch t.source.ext('o') end $ touch lex.y $ rake lex.o run yacc(lex.y) touch lex.c run compile_c(lex.c) touch lex.o Hmm.. it seems to work well.. Do I make misconception about the slide? > And question two: lets say I want a generic rule that cleans up fasta > sequences (so that if I require 'aasdflkjsalkfjasdlkj.fasta', the > file aasdflkjsalkfjasdlkj should be run through Emboss' seqret. Is > that possible? The following hasn't been working: >> rule ".fasta" => "" do |task| >> sh "seqret -sequence >> #{task.prerequisites.join} -outseq #{task.name}" >> end Try dynamic definition using the "file" method. Rekefile (put all the files to be cleaned into the source directory): SOURCES = FileList["source/*"] SOURCES.each do |src| file "#{src}.fasta" => src do |t| sh ["segret", "-sequence #{t.prerequisites.join(" ")}", "-outseq #{t.name}", ].join(" ") end end task :default => SOURCES Sometimes the "rule" methods can be complicated because it basically assumes that all files have one extension (not ".tar.gz" but ".tgz"). Recently I prefer to use "file" instead of "rule". Because Rakefiles are Ruby codes themselves, dynamic definition using the file methods are more flexible. However, I think some hack introducing some new methods (Rake DSL syntax) can make the description simpler... Thanks, Hiro -- MISHIMA, Hiroyuki, DDS, Ph.D. COE Research Fellow Department of Human Genetics Nagasaki University Graduate School of Biomedical Sciences From bonnalraoul at ingm.it Thu Dec 23 04:47:47 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 23 Dec 2010 10:47:47 +0100 Subject: [BioRuby] Workflows and Parallelization In-Reply-To: <4D12A1CE.4040702@be.to> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> <4D10B9AE.2010206@be.to> <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> <4D12A1CE.4040702@be.to> Message-ID: <0B6FAF54-370F-4CB3-8752-1E5F80DAC569@ingm.it> Hi Hiro, On 23/dic/2010, at 02.11, MISHIMA, Hiroyuki wrote: > Unfortunately I have not had experience in analyzing transcriptome using > NGS. Recently I am working on exome mutation analysis and micro RNA > analysis using NGS. I will let you know if I find good material for NGS > + transcriptome analysis. Actually the focus of my institute is mainly on mirna, so I'm also interested on techniques for analyzing NGS(illumina) and microRNA. Btw. > By the way, I afraid that talking about Rake/Pwrake and workflow > automation in the BioRuby mailing list may be kind of out-of-focus. Is it OK? Don't be afraid, was just a matter of topic (subject) :-) During the irc meeting they talked about workflows, so this is the right place -- R.J.P.B. From bonnalraoul at ingm.it Thu Dec 23 05:06:39 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 23 Dec 2010 11:06:39 +0100 Subject: [BioRuby] Wiki Message-ID: Hello, Wiki updated * Plugins: removed old doc & updated http://bioruby.open-bio.org/wiki/Plugins * Added section BioRuby::Ideas::Workflows http://bioruby.open-bio.org/wiki/Main_Page#Ideas http://bioruby.open-bio.org/wiki/Workflows please contribute to this page. Hiro, I reported a piece of your mail, please check if everything is ok. -- R.J.P.B. From ktym at hgc.jp Thu Dec 23 07:09:47 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Thu, 23 Dec 2010 21:09:47 +0900 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> Message-ID: Hi all, The 2nd IRC conf will be started in less than 2 hours from now. Toshiaki On 2010/12/20, at 3:49, Toshiaki Katayama wrote: > Hi all, > > Today, we had >10 participants and discussed about > > * Current and future of BioRuby plugin system > * BioRuby with JVM for high performance computing > * Regularly continue the IRC meeting? > > etc. Details will be reported from Pjotr. > > We agreed to continue this IRC meeting on every Thursday as a "core time", > and we may also keep the #bioruby channel open for everyone > as much as possible, I just setup an IRC bot to log discussions. > > From now on, the log of this channel will be made public at > > http://bioruby.org/irc/ > > so please join us (and please pay attention to privacy > as a netiquette as well ;-). > > Toshiaki > > > On 2010/12/19, at 21:53, Pjotr Prins wrote: > >> Today is the day!! >> >> I use the bitchx client, with logging. >> >> Pj. >> >> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>> Dear BioRuby developers, >>> >>> Here we announce the first BioRuby world wide IRC conference! >>> >>> The topic is: "Current and future BioRuby development" >>> >>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>> 6am PST (California time). >>> >>> You are kindly invited to join! >>> >>> Server: irc.freenode.org >>> Channel: #bioruby >>> >>> Toshiaki Katayama on behalf of the BioRuby staff. >>> _______________________________________________ >>> BioRuby Project - http://www.bioruby.org/ >>> BioRuby mailing list >>> BioRuby at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/bioruby > From pjotr.public14 at thebird.nl Thu Dec 23 15:53:10 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 23 Dec 2010 21:53:10 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101222101344.GB29739@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> Message-ID: <20101223205310.GA9623@thebird.nl> Interesting post on parallel programming, "High Level Concurrency with JRuby and Akka Actors" http://metaphysicaldeveloper.wordpress.com/2010/12/16/high-level-concurrency-with-jruby-and-akka-actors/ Something we'll use soon. Note that Akka is written in Scala, by one of the brightest people: Jonas Bon?r, Pj. From ktym at hgc.jp Thu Dec 23 19:21:29 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Fri, 24 Dec 2010 09:21:29 +0900 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> Message-ID: <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> Hi, Last night, conversation was not so active but we talked about * The actual usage of the BioGem, newly created BioRuby's plugin development system * NGS support including samtools The log is available at http://bioruby.org/irc/?date=2010-12 and I also wrote a step-by-step documentation on the use of BioGem at http://bioruby.open-bio.org/wiki/BiogemDevelopment Enjoy. Toshiaki On 2010/12/23, at 21:09, Toshiaki Katayama wrote: > Hi all, > > The 2nd IRC conf will be started in less than 2 hours from now. > > Toshiaki > > On 2010/12/20, at 3:49, Toshiaki Katayama wrote: > >> Hi all, >> >> Today, we had >10 participants and discussed about >> >> * Current and future of BioRuby plugin system >> * BioRuby with JVM for high performance computing >> * Regularly continue the IRC meeting? >> >> etc. Details will be reported from Pjotr. >> >> We agreed to continue this IRC meeting on every Thursday as a "core time", >> and we may also keep the #bioruby channel open for everyone >> as much as possible, I just setup an IRC bot to log discussions. >> >> From now on, the log of this channel will be made public at >> >> http://bioruby.org/irc/ >> >> so please join us (and please pay attention to privacy >> as a netiquette as well ;-). >> >> Toshiaki >> >> >> On 2010/12/19, at 21:53, Pjotr Prins wrote: >> >>> Today is the day!! >>> >>> I use the bitchx client, with logging. >>> >>> Pj. >>> >>> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>>> Dear BioRuby developers, >>>> >>>> Here we announce the first BioRuby world wide IRC conference! >>>> >>>> The topic is: "Current and future BioRuby development" >>>> >>>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>>> 6am PST (California time). >>>> >>>> You are kindly invited to join! >>>> >>>> Server: irc.freenode.org >>>> Channel: #bioruby >>>> >>>> Toshiaki Katayama on behalf of the BioRuby staff. >>>> _______________________________________________ >>>> BioRuby Project - http://www.bioruby.org/ >>>> BioRuby mailing list >>>> BioRuby at lists.open-bio.org >>>> http://lists.open-bio.org/mailman/listinfo/bioruby >> > From pjotr.public14 at thebird.nl Fri Dec 24 02:34:36 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 24 Dec 2010 08:34:36 +0100 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> Message-ID: <20101224073436.GA12220@thebird.nl> The plugin system appears to be functional - great stuff Raoul. Another thing is that the IRC channel will be attended to the coming weeks by a number of developers. If you feel like little coding and/or experimentation during the holidays, we are there to help. Pj. On Fri, Dec 24, 2010 at 09:21:29AM +0900, Toshiaki Katayama wrote: > Hi, > > Last night, conversation was not so active but we talked about > > * The actual usage of the BioGem, newly created BioRuby's plugin development system > * NGS support including samtools > > The log is available at > > http://bioruby.org/irc/?date=2010-12 > > and I also wrote a step-by-step documentation on the use of BioGem at > > http://bioruby.open-bio.org/wiki/BiogemDevelopment > > Enjoy. > > Toshiaki > > > On 2010/12/23, at 21:09, Toshiaki Katayama wrote: > > > Hi all, > > > > The 2nd IRC conf will be started in less than 2 hours from now. > > > > Toshiaki > > > > On 2010/12/20, at 3:49, Toshiaki Katayama wrote: > > > >> Hi all, > >> > >> Today, we had >10 participants and discussed about > >> > >> * Current and future of BioRuby plugin system > >> * BioRuby with JVM for high performance computing > >> * Regularly continue the IRC meeting? > >> > >> etc. Details will be reported from Pjotr. > >> > >> We agreed to continue this IRC meeting on every Thursday as a "core time", > >> and we may also keep the #bioruby channel open for everyone > >> as much as possible, I just setup an IRC bot to log discussions. > >> > >> From now on, the log of this channel will be made public at > >> > >> http://bioruby.org/irc/ > >> > >> so please join us (and please pay attention to privacy > >> as a netiquette as well ;-). > >> > >> Toshiaki > >> > >> > >> On 2010/12/19, at 21:53, Pjotr Prins wrote: > >> > >>> Today is the day!! > >>> > >>> I use the bitchx client, with logging. > >>> > >>> Pj. > >>> > >>> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: > >>>> Dear BioRuby developers, > >>>> > >>>> Here we announce the first BioRuby world wide IRC conference! > >>>> > >>>> The topic is: "Current and future BioRuby development" > >>>> > >>>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), > >>>> which is 15pm CET (Central European Time) and 9am EST (New York) and > >>>> 6am PST (California time). > >>>> > >>>> You are kindly invited to join! > >>>> > >>>> Server: irc.freenode.org > >>>> Channel: #bioruby > >>>> > >>>> Toshiaki Katayama on behalf of the BioRuby staff. > >>>> _______________________________________________ > >>>> BioRuby Project - http://www.bioruby.org/ > >>>> BioRuby mailing list > >>>> BioRuby at lists.open-bio.org > >>>> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From bonnalraoul at ingm.it Fri Dec 24 06:08:04 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Fri, 24 Dec 2010 12:08:04 +0100 Subject: [BioRuby] Plugins, Biogem and Christmas 2010 Message-ID: <3F1D10FB-B121-4819-A519-87A6637DD8F6@ingm.it> Hello guys, BioRuby plugin system was firstly announced at [BOSC 2010] and will be implemented by the Christmas 2010. Hopefully. :) -- Yes, we made it! Check out the BiogemInstallation and BiogemDevelopment sections. This is our present for you, today is December 24th, 2010. BioGem helps Bioinformaticians start developing plugins/modules for BioRuby creating a scaffold/template and a gem package. We are very happy to release it officially to all our community, as promised. Thanks to everyone for the support and the effort. You can follow the project here https://github.com/helios/bioruby-gem http://bioruby.open-bio.org/wiki/Plugins Start using it ? gem install bio-gem http://bioruby.open-bio.org/wiki/BiogemInstallation http://bioruby.open-bio.org/wiki/BiogemDevelopment Happy Christmas from BioRuby Developer Team -- R.J.P.B. From francesco.strozzi at gmail.com Mon Dec 27 06:25:26 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 12:25:26 +0100 Subject: [BioRuby] Bio Assembly Plugin Message-ID: Hi Bioruby coders, I want to start a Bioruby Assembly Plugin, where I'd like to put parsers for common used formats like ACE, CAF and maybe MAF that is used by the Mira assembler. For what I have seen the Bioruby Core does not provide this type of parsers yet. If someone else is doing something similar please let me know, so we can avoid duplicate the same code... Cheers -- Francesco From francesco.strozzi at gmail.com Mon Dec 27 06:54:01 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 12:54:01 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Thanks! I will have a look. On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO wrote: > Hi, > > http://github.com/chmille4/bioruby > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > On Mon, 27 Dec 2010 12:25:26 +0100 > Francesco Strozzi wrote: > > > Hi Bioruby coders, > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put parsers > for > > common used formats like ACE, CAF and maybe MAF that is used by the Mira > > assembler. For what I have seen the Bioruby Core does not provide this > type > > of parsers yet. If someone else is doing something similar please let me > > know, so we can avoid duplicate the same code... > > > > Cheers > > > > -- > > > > Francesco > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > Naohisa Goto > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > -- Francesco From ngoto at gen-info.osaka-u.ac.jp Mon Dec 27 06:39:06 2010 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Mon, 27 Dec 2010 20:39:06 +0900 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: Message-ID: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Hi, http://github.com/chmille4/bioruby http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html On Mon, 27 Dec 2010 12:25:26 +0100 Francesco Strozzi wrote: > Hi Bioruby coders, > > I want to start a Bioruby Assembly Plugin, where I'd like to put parsers for > common used formats like ACE, CAF and maybe MAF that is used by the Mira > assembler. For what I have seen the Bioruby Core does not provide this type > of parsers yet. If someone else is doing something similar please let me > know, so we can avoid duplicate the same code... > > Cheers > > -- > > Francesco > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org From chmille4 at gmail.com Mon Dec 27 08:11:45 2010 From: chmille4 at gmail.com (Chase Miller) Date: Mon, 27 Dec 2010 08:11:45 -0500 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Hi Francesco Currently, Bio::Assembly only has an ace parser, but I it should be straightforward to add a CAF ( or whatever ) parser. Take a look at the code (https://github.com/chmille4/bioruby) and let me know if you have any questions. I'm in the process of turning this into a plugin. I was thinking of turning it into a standalone ace parser plugin, but maybe it makes more sense to keep it as a bio-assembly plugin, especially if we can add another parser :) . Chase On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < francesco.strozzi at gmail.com> wrote: > Thanks! I will have a look. > > > On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO >wrote: > > > Hi, > > > > http://github.com/chmille4/bioruby > > > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > > > On Mon, 27 Dec 2010 12:25:26 +0100 > > Francesco Strozzi wrote: > > > > > Hi Bioruby coders, > > > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put > parsers > > for > > > common used formats like ACE, CAF and maybe MAF that is used by the > Mira > > > assembler. For what I have seen the Bioruby Core does not provide this > > type > > > of parsers yet. If someone else is doing something similar please let > me > > > know, so we can avoid duplicate the same code... > > > > > > Cheers > > > > > > -- > > > > > > Francesco > > > _______________________________________________ > > > BioRuby Project - http://www.bioruby.org/ > > > BioRuby mailing list > > > BioRuby at lists.open-bio.org > > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > Naohisa Goto > > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > > > > > -- > > Francesco > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From francesco.strozzi at gmail.com Mon Dec 27 08:28:25 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 14:28:25 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Hi Chase, I will have a look for sure. Good to know you will create a plugin for this, so it will be easy to share your code with BioRuby and add new features. A Bio::Assembly module does not exist in BioRuby at the moment, so it will make more sense also for me to keep the plugin as a general Bio::Assembly "container" where you or me or everybody else can add new parsers for different file formats. Cheers P.S. It seems the plugin system is getting ground very fast! :-) On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: > Hi Francesco > > Currently, Bio::Assembly only has an ace parser, but I it should be > straightforward to add a CAF ( or whatever ) parser. Take a look at the > code (https://github.com/chmille4/bioruby) and let me know if you have any > questions. > > I'm in the process of turning this into a plugin. I was thinking of turning > it into a standalone ace parser plugin, but maybe it makes more sense to > keep it as a bio-assembly plugin, especially if we can add another parser :) > . > > Chase > > > > On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > francesco.strozzi at gmail.com> wrote: > >> Thanks! I will have a look. >> >> >> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO > >wrote: >> >> > Hi, >> > >> > http://github.com/chmille4/bioruby >> > >> > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html >> > >> > On Mon, 27 Dec 2010 12:25:26 +0100 >> > Francesco Strozzi wrote: >> > >> > > Hi Bioruby coders, >> > > >> > > I want to start a Bioruby Assembly Plugin, where I'd like to put >> parsers >> > for >> > > common used formats like ACE, CAF and maybe MAF that is used by the >> Mira >> > > assembler. For what I have seen the Bioruby Core does not provide this >> > type >> > > of parsers yet. If someone else is doing something similar please let >> me >> > > know, so we can avoid duplicate the same code... >> > > >> > > Cheers >> > > >> > > -- >> > > >> > > Francesco >> > > _______________________________________________ >> > > BioRuby Project - http://www.bioruby.org/ >> > > BioRuby mailing list >> > > BioRuby at lists.open-bio.org >> > > http://lists.open-bio.org/mailman/listinfo/bioruby >> > >> > Naohisa Goto >> > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org >> > >> >> >> >> -- >> >> Francesco >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > > -- Francesco From chmille4 at gmail.com Mon Dec 27 12:00:44 2010 From: chmille4 at gmail.com (Chase Miller) Date: Mon, 27 Dec 2010 12:00:44 -0500 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Francesco, Looking over the code, it looks like I never got around to generalizing bio::assembly to work with multiple parsers. This is long overdue and I'm going to work on it today. Chase On Dec 27, 2010, at 8:28 AM, Francesco Strozzi wrote: > Hi Chase, > I will have a look for sure. Good to know you will create a plugin for this, so it will be easy to share your code with BioRuby and add new features. A Bio::Assembly module does not exist in BioRuby at the moment, so it will make more sense also for me to keep the plugin as a general Bio::Assembly "container" where you or me or everybody else can add new parsers for different file formats. > > Cheers > > P.S. It seems the plugin system is getting ground very fast! :-) > > On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: > Hi Francesco > > Currently, Bio::Assembly only has an ace parser, but I it should be straightforward to add a CAF ( or whatever ) parser. Take a look at the code (https://github.com/chmille4/bioruby) and let me know if you have any questions. > > I'm in the process of turning this into a plugin. I was thinking of turning it into a standalone ace parser plugin, but maybe it makes more sense to keep it as a bio-assembly plugin, especially if we can add another parser :) . > > Chase > > > > On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi wrote: > Thanks! I will have a look. > > > On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO wrote: > > > Hi, > > > > http://github.com/chmille4/bioruby > > > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > > > On Mon, 27 Dec 2010 12:25:26 +0100 > > Francesco Strozzi wrote: > > > > > Hi Bioruby coders, > > > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put parsers > > for > > > common used formats like ACE, CAF and maybe MAF that is used by the Mira > > > assembler. For what I have seen the Bioruby Core does not provide this > > type > > > of parsers yet. If someone else is doing something similar please let me > > > know, so we can avoid duplicate the same code... > > > > > > Cheers > > > > > > -- > > > > > > Francesco > > > _______________________________________________ > > > BioRuby Project - http://www.bioruby.org/ > > > BioRuby mailing list > > > BioRuby at lists.open-bio.org > > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > Naohisa Goto > > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > > > > > -- > > Francesco > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > -- > > Francesco > From pjotr.public14 at thebird.nl Mon Dec 27 12:13:57 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 27 Dec 2010 18:13:57 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101222101344.GB29739@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> Message-ID: <20101227171357.GA32230@thebird.nl> Here the second 8 minute screencast, where I call into BioRuby from Scala. A similar procedure would work for Java and Clojure, calling into (Bio)Ruby libraries. It just works (tm). http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm (temporary download link) Note how Scala resembles Ruby. Just to be clear: for me Scala does not replace Ruby. I use both languages. Scala is for performance, mostly. Essentially a replacement for C, in my book. It is a much nicer language than C, C++, or Java (shudder). On Wed, Dec 22, 2010 at 11:13:44AM +0100, Pjotr Prins wrote: > Here my first 4 min. screencast on BioRuby on the JVM: > > http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm > > (temporary download link) > > I'll try to put a few more together in the coming month(s): > > 3) Calling BioJava from JRuby and Jython (8 min) > 4) JRuby parallelization using Actors (BioRuby performance) > > And, maybe later (partly depending on demand) > > 5) One-click install of JRuby applications on non-Ruby machines > 6) JRuby BioRuby plugins > 7) Using C modules from JRuby > 8) Using JVM profiling and debugging tools with JRuby > 9) Clustering BioRuby with Terracotta > > Now, if someone could clone me... > > Pj. From georgkam at gmail.com Mon Dec 27 13:11:46 2010 From: georgkam at gmail.com (George Githinji) Date: Mon, 27 Dec 2010 21:11:46 +0300 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Hi, This definitely is intresting as sometime back I was looking for an CAF parser. I have some local dirty CAF parser. It would be interesting to develop a community CAF parser. Thanks George On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller wrote: > Francesco, > > Looking over the code, it looks like I never got around to generalizing > bio::assembly to work with multiple parsers. This is long overdue and I'm > going to work on it today. > > Chase > > On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < > francesco.strozzi at gmail.com> wrote: > > > Hi Chase, > > I will have a look for sure. Good to know you will create a plugin for > this, so it will be easy to share your code with BioRuby and add new > features. A Bio::Assembly module does not exist in BioRuby at the moment, so > it will make more sense also for me to keep the plugin as a general > Bio::Assembly "container" where you or me or everybody else can add new > parsers for different file formats. > > > > Cheers > > > > P.S. It seems the plugin system is getting ground very fast! :-) > > > > On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: > > Hi Francesco > > > > Currently, Bio::Assembly only has an ace parser, but I it should be > straightforward to add a CAF ( or whatever ) parser. Take a look at the > code (https://github.com/chmille4/bioruby) and let me know if you have any > questions. > > > > I'm in the process of turning this into a plugin. I was thinking of > turning it into a standalone ace parser plugin, but maybe it makes more > sense to keep it as a bio-assembly plugin, especially if we can add another > parser :) . > > > > Chase > > > > > > > > On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > francesco.strozzi at gmail.com> wrote: > > Thanks! I will have a look. > > > > > > On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < > ngoto at gen-info.osaka-u.ac.jp>wrote: > > > > > Hi, > > > > > > http://github.com/chmille4/bioruby > > > > > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > > > > > On Mon, 27 Dec 2010 12:25:26 +0100 > > > Francesco Strozzi wrote: > > > > > > > Hi Bioruby coders, > > > > > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put > parsers > > > for > > > > common used formats like ACE, CAF and maybe MAF that is used by the > Mira > > > > assembler. For what I have seen the Bioruby Core does not provide > this > > > type > > > > of parsers yet. If someone else is doing something similar please let > me > > > > know, so we can avoid duplicate the same code... > > > > > > > > Cheers > > > > > > > > -- > > > > > > > > Francesco > > > > _______________________________________________ > > > > BioRuby Project - http://www.bioruby.org/ > > > > BioRuby mailing list > > > > BioRuby at lists.open-bio.org > > > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > > Naohisa Goto > > > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > > > > > > > > > > -- > > > > Francesco > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > > > > > > -- > > > > Francesco > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- --------------- Sincerely George KEMRI/Wellcome-Trust Research Program Skype: george_g2 Blog: http://biorelated.wordpress.com/ From francesco.strozzi at gmail.com Mon Dec 27 14:55:27 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 20:55:27 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101227171357.GA32230@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> <20101227171357.GA32230@thebird.nl> Message-ID: Hi Pjotr, I have some problem with the screencast audio. It seems broken and I don't hear anything but bad noises. Is anyone having the same issue? I'm on a Mac using VLC and the previous screencast was fine. Thanks On Mon, Dec 27, 2010 at 18:13, Pjotr Prins wrote: > Here the second 8 minute screencast, where I call into BioRuby from > Scala. A similar procedure would work for Java and Clojure, calling > into (Bio)Ruby libraries. It just works (tm). > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > (temporary download link) > > Note how Scala resembles Ruby. > > Just to be clear: for me Scala does not replace Ruby. I use both > languages. Scala is for performance, mostly. Essentially a > replacement for C, in my book. It is a much nicer language than C, > C++, or Java (shudder). > > On Wed, Dec 22, 2010 at 11:13:44AM +0100, Pjotr Prins wrote: > > Here my first 4 min. screencast on BioRuby on the JVM: > > > > http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm > > > > (temporary download link) > > > > I'll try to put a few more together in the coming month(s): > > > > 3) Calling BioJava from JRuby and Jython (8 min) > > 4) JRuby parallelization using Actors (BioRuby performance) > > > > And, maybe later (partly depending on demand) > > > > 5) One-click install of JRuby applications on non-Ruby machines > > 6) JRuby BioRuby plugins > > 7) Using C modules from JRuby > > 8) Using JVM profiling and debugging tools with JRuby > > 9) Clustering BioRuby with Terracotta > > > > Now, if someone could clone me... > > > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- Francesco From pjotr.public14 at thebird.nl Mon Dec 27 15:13:56 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 27 Dec 2010 21:13:56 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> <20101227171357.GA32230@thebird.nl> Message-ID: <20101227201356.GA658@thebird.nl> Works for me. md5: > md5sum JRuby_from_Scala_or_Java.ogm a28646e48527c268a26469c4ef271238 JRuby_from_Scala_or_Java.ogm if it does not work can make one again... Pj. On Mon, Dec 27, 2010 at 08:55:27PM +0100, Francesco Strozzi wrote: > Hi Pjotr, > > I have some problem with the screencast audio. It seems broken and I don't > hear anything but bad noises. Is anyone having the same issue? I'm on a Mac > using VLC and the previous screencast was fine. > > Thanks > > On Mon, Dec 27, 2010 at 18:13, Pjotr Prins wrote: > > > Here the second 8 minute screencast, where I call into BioRuby from > > Scala. A similar procedure would work for Java and Clojure, calling > > into (Bio)Ruby libraries. It just works (tm). > > > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > > > (temporary download link) > > > > Note how Scala resembles Ruby. > > > > Just to be clear: for me Scala does not replace Ruby. I use both > > languages. Scala is for performance, mostly. Essentially a > > replacement for C, in my book. It is a much nicer language than C, > > C++, or Java (shudder). > > > > On Wed, Dec 22, 2010 at 11:13:44AM +0100, Pjotr Prins wrote: > > > Here my first 4 min. screencast on BioRuby on the JVM: > > > > > > http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm > > > > > > (temporary download link) > > > > > > I'll try to put a few more together in the coming month(s): > > > > > > 3) Calling BioJava from JRuby and Jython (8 min) > > > 4) JRuby parallelization using Actors (BioRuby performance) > > > > > > And, maybe later (partly depending on demand) > > > > > > 5) One-click install of JRuby applications on non-Ruby machines > > > 6) JRuby BioRuby plugins > > > 7) Using C modules from JRuby > > > 8) Using JVM profiling and debugging tools with JRuby > > > 9) Clustering BioRuby with Terracotta > > > > > > Now, if someone could clone me... > > > > > > Pj. > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > > -- > > Francesco From pjotr.public14 at thebird.nl Tue Dec 28 04:29:23 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 28 Dec 2010 10:29:23 +0100 Subject: [BioRuby] BioRuby Screencast Message-ID: <20101228092923.GA2089@thebird.nl> Ok, try again. Now it is ogg proper - the file is smaller too. 204d140c57f00ff16a19343742712f4c JRuby_from_Scala_or_Java.ogm On Mon, Dec 27, 2010 at 06:13:57PM +0100, Pjotr Prins wrote: > Here the second 8 minute screencast, where I call into BioRuby from > Scala. A similar procedure would work for Java and Clojure, calling > into (Bio)Ruby libraries. It just works (tm). > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > (temporary download link) > > Note how Scala resembles Ruby. > > Just to be clear: for me Scala does not replace Ruby. I use both > languages. Scala is for performance, mostly. Essentially a > replacement for C, in my book. It is a much nicer language than C, > C++, or Java (shudder). From francesco.strozzi at gmail.com Tue Dec 28 04:35:57 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Tue, 28 Dec 2010 10:35:57 +0100 Subject: [BioRuby] BioRuby Screencast In-Reply-To: <20101228092923.GA2089@thebird.nl> References: <20101228092923.GA2089@thebird.nl> Message-ID: Works perfectly now! Thanks Cheers On Tue, Dec 28, 2010 at 10:29, Pjotr Prins wrote: > Ok, try again. Now it is ogg proper - the file is smaller too. > > 204d140c57f00ff16a19343742712f4c JRuby_from_Scala_or_Java.ogm > > On Mon, Dec 27, 2010 at 06:13:57PM +0100, Pjotr Prins wrote: > > Here the second 8 minute screencast, where I call into BioRuby from > > Scala. A similar procedure would work for Java and Clojure, calling > > into (Bio)Ruby libraries. It just works (tm). > > > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > > > (temporary download link) > > > > Note how Scala resembles Ruby. > > > > Just to be clear: for me Scala does not replace Ruby. I use both > > languages. Scala is for performance, mostly. Essentially a > > replacement for C, in my book. It is a much nicer language than C, > > C++, or Java (shudder). > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- Francesco From bonnalraoul at ingm.it Tue Dec 28 05:18:25 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 28 Dec 2010 11:18:25 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> On 27/dic/2010, at 19.11, George Githinji wrote: > Hi, > This definitely is intresting as sometime back I was looking for an CAF > parser. > I have some local dirty CAF parser. It would be interesting to develop a > community CAF parser. I think so. I agree with Francesco, I suggest to start designing a general Assembler and use adapters like Ace, CAF an others in the way we did with Bio::Sequence. Bio::Assembler Bio::Assembler::Ace Bio::Assembler::CAF and we need to keep in mind that we can not store everything in memory :-) > Thanks > George > > > On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller wrote: > >> Francesco, >> >> Looking over the code, it looks like I never got around to generalizing >> bio::assembly to work with multiple parsers. This is long overdue and I'm >> going to work on it today. >> >> Chase >> >> On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < >> francesco.strozzi at gmail.com> wrote: >> >>> Hi Chase, >>> I will have a look for sure. Good to know you will create a plugin for >> this, so it will be easy to share your code with BioRuby and add new >> features. A Bio::Assembly module does not exist in BioRuby at the moment, so >> it will make more sense also for me to keep the plugin as a general >> Bio::Assembly "container" where you or me or everybody else can add new >> parsers for different file formats. >>> >>> Cheers >>> >>> P.S. It seems the plugin system is getting ground very fast! :-) >>> >>> On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: >>> Hi Francesco >>> >>> Currently, Bio::Assembly only has an ace parser, but I it should be >> straightforward to add a CAF ( or whatever ) parser. Take a look at the >> code (https://github.com/chmille4/bioruby) and let me know if you have any >> questions. >>> >>> I'm in the process of turning this into a plugin. I was thinking of >> turning it into a standalone ace parser plugin, but maybe it makes more >> sense to keep it as a bio-assembly plugin, especially if we can add another >> parser :) . >>> >>> Chase >>> >>> >>> >>> On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < >> francesco.strozzi at gmail.com> wrote: >>> Thanks! I will have a look. >>> >>> >>> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < >> ngoto at gen-info.osaka-u.ac.jp>wrote: >>> >>>> Hi, >>>> >>>> http://github.com/chmille4/bioruby >>>> >>>> http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html >>>> >>>> On Mon, 27 Dec 2010 12:25:26 +0100 >>>> Francesco Strozzi wrote: >>>> >>>>> Hi Bioruby coders, >>>>> >>>>> I want to start a Bioruby Assembly Plugin, where I'd like to put >> parsers >>>> for >>>>> common used formats like ACE, CAF and maybe MAF that is used by the >> Mira >>>>> assembler. For what I have seen the Bioruby Core does not provide >> this >>>> type >>>>> of parsers yet. If someone else is doing something similar please let >> me >>>>> know, so we can avoid duplicate the same code... >>>>> >>>>> Cheers >>>>> >>>>> -- >>>>> >>>>> Francesco >>>>> _______________________________________________ >>>>> BioRuby Project - http://www.bioruby.org/ >>>>> BioRuby mailing list >>>>> BioRuby at lists.open-bio.org >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby >>>> >>>> Naohisa Goto >>>> ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org >>>> >>> >>> >>> >>> -- >>> >>> Francesco >>> _______________________________________________ >>> BioRuby Project - http://www.bioruby.org/ >>> BioRuby mailing list >>> BioRuby at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/bioruby >>> >>> >>> >>> >>> -- >>> >>> Francesco >>> >> >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > > > > -- > --------------- > Sincerely > George > KEMRI/Wellcome-Trust Research Program > Skype: george_g2 > Blog: http://biorelated.wordpress.com/ > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby -- R.J.P.B. From francesco.strozzi at gmail.com Tue Dec 28 08:56:41 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Tue, 28 Dec 2010 14:56:41 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> Message-ID: Iteretors will be the way, as discussed in another thread :-) Ciao Francesco On Tue, Dec 28, 2010 at 11:18, Raoul Bonnal wrote: > > On 27/dic/2010, at 19.11, George Githinji wrote: > > > Hi, > > This definitely is intresting as sometime back I was looking for an CAF > > parser. > > I have some local dirty CAF parser. It would be interesting to develop a > > community CAF parser. > > I think so. > I agree with Francesco, I suggest to start designing a general Assembler > and use adapters like Ace, CAF an others in the way we did with > Bio::Sequence. > > Bio::Assembler > Bio::Assembler::Ace > Bio::Assembler::CAF > > and we need to keep in mind that we can not store everything in memory :-) > > > > > Thanks > > George > > > > > > On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller > wrote: > > > >> Francesco, > >> > >> Looking over the code, it looks like I never got around to generalizing > >> bio::assembly to work with multiple parsers. This is long overdue and > I'm > >> going to work on it today. > >> > >> Chase > >> > >> On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >> > >>> Hi Chase, > >>> I will have a look for sure. Good to know you will create a plugin for > >> this, so it will be easy to share your code with BioRuby and add new > >> features. A Bio::Assembly module does not exist in BioRuby at the > moment, so > >> it will make more sense also for me to keep the plugin as a general > >> Bio::Assembly "container" where you or me or everybody else can add new > >> parsers for different file formats. > >>> > >>> Cheers > >>> > >>> P.S. It seems the plugin system is getting ground very fast! :-) > >>> > >>> On Mon, Dec 27, 2010 at 14:11, Chase Miller > wrote: > >>> Hi Francesco > >>> > >>> Currently, Bio::Assembly only has an ace parser, but I it should be > >> straightforward to add a CAF ( or whatever ) parser. Take a look at the > >> code (https://github.com/chmille4/bioruby) and let me know if you have > any > >> questions. > >>> > >>> I'm in the process of turning this into a plugin. I was thinking of > >> turning it into a standalone ace parser plugin, but maybe it makes more > >> sense to keep it as a bio-assembly plugin, especially if we can add > another > >> parser :) . > >>> > >>> Chase > >>> > >>> > >>> > >>> On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >>> Thanks! I will have a look. > >>> > >>> > >>> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < > >> ngoto at gen-info.osaka-u.ac.jp>wrote: > >>> > >>>> Hi, > >>>> > >>>> http://github.com/chmille4/bioruby > >>>> > >>>> > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > >>>> > >>>> On Mon, 27 Dec 2010 12:25:26 +0100 > >>>> Francesco Strozzi wrote: > >>>> > >>>>> Hi Bioruby coders, > >>>>> > >>>>> I want to start a Bioruby Assembly Plugin, where I'd like to put > >> parsers > >>>> for > >>>>> common used formats like ACE, CAF and maybe MAF that is used by the > >> Mira > >>>>> assembler. For what I have seen the Bioruby Core does not provide > >> this > >>>> type > >>>>> of parsers yet. If someone else is doing something similar please let > >> me > >>>>> know, so we can avoid duplicate the same code... > >>>>> > >>>>> Cheers > >>>>> > >>>>> -- > >>>>> > >>>>> Francesco > >>>>> _______________________________________________ > >>>>> BioRuby Project - http://www.bioruby.org/ > >>>>> BioRuby mailing list > >>>>> BioRuby at lists.open-bio.org > >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>>> > >>>> Naohisa Goto > >>>> ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > >>>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> _______________________________________________ > >>> BioRuby Project - http://www.bioruby.org/ > >>> BioRuby mailing list > >>> BioRuby at lists.open-bio.org > >>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> > >> > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > > > > > > > -- > > --------------- > > Sincerely > > George > > KEMRI/Wellcome-Trust Research Program > > Skype: george_g2 > > Blog: http://biorelated.wordpress.com/ > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > -- > R.J.P.B. > > > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- Francesco From chmille4 at gmail.com Tue Dec 28 09:16:08 2010 From: chmille4 at gmail.com (Chase Miller) Date: Tue, 28 Dec 2010 09:16:08 -0500 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> Message-ID: > I agree with Francesco, I suggest to start designing a general Assembler > and use adapters like Ace, CAF an others in the way we did with > Bio::Sequence. > > Bio::Assembler > Bio::Assembler::Ace > Bio::Assembler::CAF > My code currently does this, but not in the way Bio::Sequence does. I based my solution off this stackoverflow example ( http://stackoverflow.com/questions/746207/ruby-design-pattern-how-to-make-an-extensible-factory-class ). I'll take look at Bio::Sequence and see if it's easy to refactor. Here's what I have so far (suggestions very welcome!): code: https://github.com/chmille4/bioruby-assembly usage example: https://gist.github.com/615296 > and we need to keep in mind that we can not store everything in memory :-) > By this do you mean streaming, where the entire file is eventually read into memory if all the data is accessed? Or are you talking about something else, perhaps creating an index file similar to a project like https://github.com/ekg/fastahack. Cheers, Chase > > > > > Thanks > > George > > > > > > On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller > wrote: > > > >> Francesco, > >> > >> Looking over the code, it looks like I never got around to generalizing > >> bio::assembly to work with multiple parsers. This is long overdue and > I'm > >> going to work on it today. > >> > >> Chase > >> > >> On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >> > >>> Hi Chase, > >>> I will have a look for sure. Good to know you will create a plugin for > >> this, so it will be easy to share your code with BioRuby and add new > >> features. A Bio::Assembly module does not exist in BioRuby at the > moment, so > >> it will make more sense also for me to keep the plugin as a general > >> Bio::Assembly "container" where you or me or everybody else can add new > >> parsers for different file formats. > >>> > >>> Cheers > >>> > >>> P.S. It seems the plugin system is getting ground very fast! :-) > >>> > >>> On Mon, Dec 27, 2010 at 14:11, Chase Miller > wrote: > >>> Hi Francesco > >>> > >>> Currently, Bio::Assembly only has an ace parser, but I it should be > >> straightforward to add a CAF ( or whatever ) parser. Take a look at the > >> code (https://github.com/chmille4/bioruby) and let me know if you have > any > >> questions. > >>> > >>> I'm in the process of turning this into a plugin. I was thinking of > >> turning it into a standalone ace parser plugin, but maybe it makes more > >> sense to keep it as a bio-assembly plugin, especially if we can add > another > >> parser :) . > >>> > >>> Chase > >>> > >>> > >>> > >>> On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >>> Thanks! I will have a look. > >>> > >>> > >>> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < > >> ngoto at gen-info.osaka-u.ac.jp>wrote: > >>> > >>>> Hi, > >>>> > >>>> http://github.com/chmille4/bioruby > >>>> > >>>> > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > >>>> > >>>> On Mon, 27 Dec 2010 12:25:26 +0100 > >>>> Francesco Strozzi wrote: > >>>> > >>>>> Hi Bioruby coders, > >>>>> > >>>>> I want to start a Bioruby Assembly Plugin, where I'd like to put > >> parsers > >>>> for > >>>>> common used formats like ACE, CAF and maybe MAF that is used by the > >> Mira > >>>>> assembler. For what I have seen the Bioruby Core does not provide > >> this > >>>> type > >>>>> of parsers yet. If someone else is doing something similar please let > >> me > >>>>> know, so we can avoid duplicate the same code... > >>>>> > >>>>> Cheers > >>>>> > >>>>> -- > >>>>> > >>>>> Francesco > >>>>> _______________________________________________ > >>>>> BioRuby Project - http://www.bioruby.org/ > >>>>> BioRuby mailing list > >>>>> BioRuby at lists.open-bio.org > >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>>> > >>>> Naohisa Goto > >>>> ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > >>>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> _______________________________________________ > >>> BioRuby Project - http://www.bioruby.org/ > >>> BioRuby mailing list > >>> BioRuby at lists.open-bio.org > >>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> > >> > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > > > > > > > -- > > --------------- > > Sincerely > > George > > KEMRI/Wellcome-Trust Research Program > > Skype: george_g2 > > Blog: http://biorelated.wordpress.com/ > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > -- > R.J.P.B. > > > > > From pjotr.public14 at thebird.nl Tue Dec 28 12:33:45 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 28 Dec 2010 18:33:45 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <20101228173345.GA4446@thebird.nl> On Mon, Dec 27, 2010 at 02:28:25PM +0100, Francesco Strozzi wrote: > P.S. It seems the plugin system is getting ground very fast! :-) I just cut my first bioruby-gff3 plugin gem using the tutorials :). Raoul has done a great job. The next step is to make 'blessed' gems available from bioruby itself. I should be online for the Thursday IRC session. Pj. From ktym at hgc.jp Thu Dec 30 23:44:31 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Fri, 31 Dec 2010 13:44:31 +0900 Subject: [BioRuby] The third BioRuby IRC conference on Dec 30th In-Reply-To: <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> Message-ID: <6487068C-FB06-4F17-8308-A70A3324F819@hgc.jp> Hi, Last night, we discussed about the "blessed" plugins, which will be a set of recommended plugins. You can read the IRC log at the following page as always: http://bioruby.org/irc/?date=2010-12 We agreed to maintain two "meta" packages of plugins * "bio-full" --> not reviewd but includes all plugins * "bio-core" --> reviewed/selected/recommended plugins and Raoul provided the first example of the "bio-full" meta package at https://github.com/helios/bioruby-full Although, the name of these meta packages and the system to maintain those in detail are still under discussion. Additionally, I created a toy plugin "bio-hello" for fun (and hopefully a working example of the BioRuby plugin). % gem install bio-hello % biohello.rb -x -c "A happy new year!" gcataacatgcacctccttattaaaatgaatggtaatatgaagcaaga gc c--g a---t t----a a----t a---t c--g at at c--g g---c t----a g----c g---c a--t gc cg t--a t---a a----t t----a t---a a--t at ta t--a a---t c----g t----a t---a a--t cg gc t--a a---t a----t t----a a---t t--a gc ta t--a c---g g----c t----a t---a c--g ta % ruby -rubygems -r bio -e 'p Bio::Sequence::NA.new("gcataacatgcacctccttattaaaatgaatggtaatatgaagcaaga").translate' ==> "A*HAPPY*NEW*YEAR" Please see https://github.com/ktym/bioruby-hello/blob/master/lib/bio-hello.rb and try % biohello.rb --help for more details. I hope you enjoy! Toshiaki On 2010/12/24, at 9:21, Toshiaki Katayama wrote: > Hi, > > Last night, conversation was not so active but we talked about > > * The actual usage of the BioGem, newly created BioRuby's plugin development system > * NGS support including samtools > > The log is available at > > http://bioruby.org/irc/?date=2010-12 > > and I also wrote a step-by-step documentation on the use of BioGem at > > http://bioruby.open-bio.org/wiki/BiogemDevelopment > > Enjoy. > > Toshiaki > > > On 2010/12/23, at 21:09, Toshiaki Katayama wrote: > >> Hi all, >> >> The 2nd IRC conf will be started in less than 2 hours from now. >> >> Toshiaki >> >> On 2010/12/20, at 3:49, Toshiaki Katayama wrote: >> >>> Hi all, >>> >>> Today, we had >10 participants and discussed about >>> >>> * Current and future of BioRuby plugin system >>> * BioRuby with JVM for high performance computing >>> * Regularly continue the IRC meeting? >>> >>> etc. Details will be reported from Pjotr. >>> >>> We agreed to continue this IRC meeting on every Thursday as a "core time", >>> and we may also keep the #bioruby channel open for everyone >>> as much as possible, I just setup an IRC bot to log discussions. >>> >>> From now on, the log of this channel will be made public at >>> >>> http://bioruby.org/irc/ >>> >>> so please join us (and please pay attention to privacy >>> as a netiquette as well ;-). >>> >>> Toshiaki >>> >>> >>> On 2010/12/19, at 21:53, Pjotr Prins wrote: >>> >>>> Today is the day!! >>>> >>>> I use the bitchx client, with logging. >>>> >>>> Pj. >>>> >>>> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>>>> Dear BioRuby developers, >>>>> >>>>> Here we announce the first BioRuby world wide IRC conference! >>>>> >>>>> The topic is: "Current and future BioRuby development" >>>>> >>>>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>>>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>>>> 6am PST (California time). >>>>> >>>>> You are kindly invited to join! >>>>> >>>>> Server: irc.freenode.org >>>>> Channel: #bioruby >>>>> >>>>> Toshiaki Katayama on behalf of the BioRuby staff. >>>>> _______________________________________________ >>>>> BioRuby Project - http://www.bioruby.org/ >>>>> BioRuby mailing list >>>>> BioRuby at lists.open-bio.org >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby >>> >> > From pjotr.public14 at thebird.nl Fri Dec 31 09:40:04 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 31 Dec 2010 15:40:04 +0100 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: <20101224073436.GA12220@thebird.nl> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> <20101224073436.GA12220@thebird.nl> Message-ID: <20101231144004.GB16875@thebird.nl> Today I fixed a few bug in the GFF3 parser. The problem was the use of phase in CDSs. By default it is not used, and now everything appears to pass. It is available as a bio-gff3 bioruby plugin on rubygems. https://rubygems.org/gems/bio-gff3 or my github repo https://github.com/pjotrp/bioruby-gff3-plugin which has the README. For example, fetch mRNA and CDS information from GFF3 files and output to FASTA: gff3-fetch mrna test/data/gff/test.gff3 gff3-fetch cds test/data/gff/test.gff3 On Fri, Dec 24, 2010 at 08:34:36AM +0100, Pjotr Prins wrote: > The plugin system appears to be functional - great stuff Raoul. From ktym at hgc.jp Tue Dec 7 11:51:32 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Tue, 7 Dec 2010 20:51:32 +0900 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th Message-ID: Dear BioRuby developers, Here we announce the first BioRuby world wide IRC conference! The topic is: "Current and future BioRuby development" The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), which is 15pm CET (Central European Time) and 9am EST (New York) and 6am PST (California time). You are kindly invited to join! Server: irc.freenode.org Channel: #bioruby Toshiaki Katayama on behalf of the BioRuby staff. From pjotr.public14 at thebird.nl Tue Dec 7 19:42:13 2010 From: pjotr.public14 at thebird.nl (pjotr.public14 at thebird.nl) Date: Tue, 7 Dec 2010 20:42:13 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: References: Message-ID: <20101207194213.GA25866@thebird.nl> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: > Dear BioRuby developers, > > Here we announce the first BioRuby world wide IRC conference! > > The topic is: "Current and future BioRuby development" > > The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), > which is 15pm CET (Central European Time) and 9am EST (New York) and > 6am PST (California time). Great idea! I, for one would like to look at BioRuby on the JVM. Another topic would be integrationg the brilliant http://matplotlib.sourceforge.net/. Yes, it is Python, but we could bridge that. Would it be an idea to have talks? We could do that in the form of videos. I don't know what people would like to present, but if you are interested in giving, or hearing, some talk, we could make them available just before the conference day. Videos of 10-20 minutes, preferably a code walk/tutorial. Who is game? If anyone is interested I could do a talk on either 1. Ruby actors for parallelization 2. The (not yet high performance) genome browser plugin (GFF3) Maybe Raoul can present the plugin system, and Naohisa a code walk on certain modules? Anyone interested in a presentation on Rubyshell inside the browser (Rails)? If it is successful, we could make this an annual happening. Pj. From bonnalraoul at ingm.it Thu Dec 9 09:12:14 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 9 Dec 2010 10:12:14 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101207194213.GA25866@thebird.nl> References: <20101207194213.GA25866@thebird.nl> Message-ID: <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> Dear All, I'd like to present the plugin system and the scaffolder but unfortunately on Dec 19 at the meeting time I'll landing in Milan, Malpensa. Btw I'll find out a way to be connected. I like to idea of BioRubyCast +1 I'm interested on RubyShell/Rails. Il giorno 07/dic/2010, alle ore 20.42, pjotr.public14 at thebird.nl ha scritto: > On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >> Dear BioRuby developers, >> >> Here we announce the first BioRuby world wide IRC conference! >> >> The topic is: "Current and future BioRuby development" >> >> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >> which is 15pm CET (Central European Time) and 9am EST (New York) and >> 6am PST (California time). > > Great idea! I, for one would like to look at BioRuby on the JVM. > > Another topic would be integrationg the brilliant > http://matplotlib.sourceforge.net/. Yes, it is Python, but we could > bridge that. > > Would it be an idea to have talks? We could do that in the form of > videos. I don't know what people would like to present, but if you are > interested in giving, or hearing, some talk, we could make them > available just before the conference day. Videos of 10-20 minutes, > preferably a code walk/tutorial. > > Who is game? > > If anyone is interested I could do a talk on either > > 1. Ruby actors for parallelization > 2. The (not yet high performance) genome browser plugin (GFF3) > > Maybe Raoul can present the plugin system, and Naohisa a code walk on > certain modules? Anyone interested in a presentation on Rubyshell > inside the browser (Rails)? > > If it is successful, we could make this an annual happening. > > Pj. > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From mail at michaelbarton.me.uk Thu Dec 9 17:06:58 2010 From: mail at michaelbarton.me.uk (Michael Barton) Date: Thu, 9 Dec 2010 12:06:58 -0500 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> Message-ID: I have also written a genome scaffolder in Ruby. The lastest doc is located at - http://bit.ly/ho2UtW I hope to release this within the next week or two. Cheers Mike On 9 December 2010 04:12, Raoul Bonnal wrote: > Dear All, > I'd like to present the plugin system and the scaffolder but unfortunately on Dec 19 at the meeting time I'll landing in Milan, Malpensa. > Btw I'll find out a way to be connected. > > I like to idea of BioRubyCast +1 > > I'm interested on RubyShell/Rails. > > > Il giorno 07/dic/2010, alle ore 20.42, pjotr.public14 at thebird.nl ha scritto: > >> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>> Dear BioRuby developers, >>> >>> Here we announce the first BioRuby world wide IRC conference! >>> >>> The topic is: "Current and future BioRuby development" >>> >>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>> 6am PST (California time). >> >> Great idea! I, for one would like to look at BioRuby on the JVM. >> >> Another topic would be integrationg the brilliant >> http://matplotlib.sourceforge.net/. Yes, it is Python, but we could >> bridge that. >> >> Would it be an idea to have talks? We could do that in the form of >> videos. I don't know what people would like to present, but if you are >> interested in giving, or hearing, some talk, we could make them >> available just before the conference day. Videos of 10-20 minutes, >> preferably a code walk/tutorial. >> >> Who is game? >> >> If anyone is interested I could do a talk on either >> >> 1. Ruby actors for parallelization >> 2. The (not yet high performance) genome browser plugin (GFF3) >> >> Maybe Raoul can present the plugin system, and Naohisa a code walk on >> certain modules? Anyone interested in a presentation on Rubyshell >> inside the browser (Rails)? >> >> If it is successful, we could make this an annual happening. >> >> Pj. >> >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From pjotr.public14 at thebird.nl Thu Dec 9 17:45:53 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 9 Dec 2010 18:45:53 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> Message-ID: <20101209174553.GA11569@thebird.nl> On Thu, Dec 09, 2010 at 12:06:58PM -0500, Michael Barton wrote: > I have also written a genome scaffolder in Ruby. > The lastest doc is located at - http://bit.ly/ho2UtW > I hope to release this within the next week or two. That is a very nice idea - it would be great if we can get rid of gff3. And it is perfect for a plugin. I only wonder how YAML will hold up with really big files. Pj. From mail at michaelbarton.me.uk Thu Dec 9 18:10:06 2010 From: mail at michaelbarton.me.uk (Michael Barton) Date: Thu, 9 Dec 2010 13:10:06 -0500 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101209174553.GA11569@thebird.nl> References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> <20101209174553.GA11569@thebird.nl> Message-ID: <20101209181006.GA35533@nku069218.hh.nku.edu> Thanks Pjotr. I wrote it for my own needs doing genome assembly on a 7MBp microbe. At the moment I'm doing my own mini-code freeze so that I can finish the website and documentation for releasing version 0.4. Here's a bit more detail which may be of interest: The YAML format maps on to a DSL for the Scaffolder::Region class. --- some_class: some_attr: "value" The above YAML corresponds to a DSL which uses Ruby's metaprogramming. The top level name corresponds to a class and the second level indentations correspond to the attributes on the instance of that class. So the above YAML would fetch the subclass of Scaffolder::Region named SomeClass, create a new instance, and call 'some_method' on this instance with the parameter "value". All you would need to do to define this class is as follows. class SomeClass << Scaffolder::Region attribute :some_attr end I tried to design it so that the vocabulary of the YAML scaffold definition is completely extensible. See the Scaffolder::Region#attribute method for more details - http://bit.ly/eMImcn Here's the slides from a short presentation I gave on it recently also: http://bit.ly/cwH2Xm. I agree that really large YAML files may be a problem with memory requirements. However I'm working on small increments developments so far. I've yet to encounter it though but hopefully will be able to solve it when I reach it. Any suggestions from the bioruby community are very welcome. Cheers Mike On Thu, Dec 09, 2010 at 06:45:53PM +0100, Pjotr Prins wrote: > On Thu, Dec 09, 2010 at 12:06:58PM -0500, Michael Barton wrote: > > I have also written a genome scaffolder in Ruby. > > The lastest doc is located at - http://bit.ly/ho2UtW > > I hope to release this within the next week or two. > > That is a very nice idea - it would be great if we can get rid of > gff3. And it is perfect for a plugin. > > I only wonder how YAML will hold up with really big files. > > Pj. From bonnalraoul at ingm.it Fri Dec 10 11:45:16 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Fri, 10 Dec 2010 12:45:16 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101209181006.GA35533@nku069218.hh.nku.edu> References: <20101207194213.GA25866@thebird.nl> <74FD3023-E9CF-4B61-9043-A110F086DA87@ingm.it> <20101209174553.GA11569@thebird.nl> <20101209181006.GA35533@nku069218.hh.nku.edu> Message-ID: <30100CBE-81F2-41CD-84CD-63610A9B4A77@ingm.it> Got it. It's a good idea. Sorry I use the term Scaffolder but it's a different thing it's more like a scaffold generator code for developing bioruby libraries. We can packit as a bioruby gem with my lib. Also George wrote a library, asking for the plugin guidelines time ago, we could pack it too. Il giorno 09/dic/2010, alle ore 19.10, Michael Barton ha scritto: > Thanks Pjotr. I wrote it for my own needs doing genome assembly on a 7MBp > microbe. At the moment I'm doing my own mini-code freeze so that I can finish > the website and documentation for releasing version 0.4. > > Here's a bit more detail which may be of interest: > > The YAML format maps on to a DSL for the Scaffolder::Region class. > > --- > some_class: > some_attr: "value" > > The above YAML corresponds to a DSL which uses Ruby's metaprogramming. The top > level name corresponds to a class and the second level indentations correspond > to the attributes on the instance of that class. So the above YAML would fetch > the subclass of Scaffolder::Region named SomeClass, create a new instance, and > call 'some_method' on this instance with the parameter "value". > > All you would need to do to define this class is as follows. > > class SomeClass << Scaffolder::Region > attribute :some_attr > end > > I tried to design it so that the vocabulary of the YAML scaffold definition is > completely extensible. See the Scaffolder::Region#attribute method for more > details - http://bit.ly/eMImcn > > Here's the slides from a short presentation I gave on it recently also: > http://bit.ly/cwH2Xm. > > I agree that really large YAML files may be a problem with memory requirements. > However I'm working on small increments developments so far. I've yet to > encounter it though but hopefully will be able to solve it when I reach it. > Any suggestions from the bioruby community are very welcome. > > Cheers > > Mike > > On Thu, Dec 09, 2010 at 06:45:53PM +0100, Pjotr Prins wrote: >> On Thu, Dec 09, 2010 at 12:06:58PM -0500, Michael Barton wrote: >>> I have also written a genome scaffolder in Ruby. >>> The lastest doc is located at - http://bit.ly/ho2UtW >>> I hope to release this within the next week or two. >> >> That is a very nice idea - it would be great if we can get rid of >> gff3. And it is perfect for a plugin. >> >> I only wonder how YAML will hold up with really big files. >> >> Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Sat Dec 11 09:46:54 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 11 Dec 2010 10:46:54 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) Message-ID: <20101211094654.GA19598@thebird.nl> Hi Rubyistas, We have a special community, with a special language. Ruby is one of the most fun languages to work with, and we know it. Here I want to argue that for bioinformatics JRuby is one of the most exciting developments. Not only is it pretty fast, once compiled, but it also allows easy integration with Java (and BioJava). Before you recoil in horror, it also allows integration with some really cool programming languages, i.e. Scala, Clojure and Groovy. You know Ruby is great. But it has some weaknesses too. For Bioinformatics (B) and big data (BD) the problems are: (1) Weak B functionality (2) BD performance issues (3) So so parallel computing support (for BD) (4) Only partial functional programming support (1) and (2) can be resolved by using JRuby, BioJava and the JVM. (3) and (4) can be resolved by tapping into Scala and Clojure. Let me try to explain. (1) Weak B functionality: BioRuby is a great achievement, but I have a number of criticisms. First is that it is not suitable for BD. Almost every module loads all data in RAM, and there is no concept of parallel computations in the design. Finally the development is not fast - we are suffering from the fact that we are a small community. You could argue about reasons, but I don't think we should spend energy on the past, when there is such an obvious way forward. Let me continue. (2) BD performance issues: Ruby is slow. By definition compared to statically typed and compiled languages (such as C, Java, Scala, Clojure). It is pretty amazing to see how much speed Ruby 1.9 has. But, for BD it breaks down quickly. Ruby's strength is in beauty of code, but not in raw power. (3) So so parallel computing support (for BD): Functional languages (Haskell, Erlang, Scala, Clojure) have immutable data, and abstractions for parallelization, such as shared memory and actors, which make it much easier to write parallelized code. For performance and BD, this is extremely useful. (4) Only partial functional programming support: Once you get into functional programming you realize Ruby gets in the way. Support for functional programming in Ruby is patchy, though there is some. It is no accident that I have started the BioScala project, and Jan Aerts has started the BioClojure project in 2010 (!) BioRuby has spin-offs. My experience with Scala has been great. Scala is statically typed, and very fast. It also allows beautiful code with functional programming and parallelization thrown in. For me, there is a clear path where I use Ruby and Scala on a 50/50 basis. Essentially using the best of both worlds. JRuby is key to combining them. And, you know what? It is great fun!! I would get frustrated if I was locked in either language. But now it is seamless moving between the two, thanks to the JVM. Which, btw, these days can outperform even C code. Believe me, even two years ago, I would not have thought I would *ever* champion the JVM. But as a saaientist, you go by evidence. Programming is fun. And it has not ever been this great. I want to share that with you, and I would like to use the coming holiday season, and years after, to pass that on. I would guess Jan thinks the same way about Ruby and Clojure. Who is interested in getting back into the fun of programming? Who wants to experiment and become an even more productive programmer? It could be the goal of BioRuby in 2011 to show the way to other Bio* projects of handling development in such a way that we can easily move between the strengths of dynamic programming languages and high performant functional languages. I would like that. Pj. From pjotr.public14 at thebird.nl Sun Dec 12 16:47:09 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sun, 12 Dec 2010 17:47:09 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101211094654.GA19598@thebird.nl> References: <20101211094654.GA19598@thebird.nl> Message-ID: <20101212164709.GA27907@thebird.nl> On Sat, Dec 11, 2010 at 10:46:54AM +0100, Pjotr Prins wrote: > We have a special community, with a special language. Ruby is one of > the most fun languages to work with, and we know it. The response is overwhelming. Does it mean no one has fun anymore? Or is Ruby dying a slow death in bioinformatics? Or was my message just over everyone's head? If anyone is interested I could do a screencast on using BioRuby and BioJava on the JVM. But I fear, somehow, the effort is wasted. Pj. From russell.whitaker at gmail.com Sun Dec 12 17:21:38 2010 From: russell.whitaker at gmail.com (Russell Whitaker) Date: Sun, 12 Dec 2010 09:21:38 -0800 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101212164709.GA27907@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: On Sun, Dec 12, 2010 at 8:47 AM, Pjotr Prins wrote: > On Sat, Dec 11, 2010 at 10:46:54AM +0100, Pjotr Prins wrote: >> We have a special community, with a special language. Ruby is one of >> the most fun languages to work with, and we know it. > > The response is overwhelming. Does it mean no one has fun anymore? Or > is Ruby dying a slow death in bioinformatics? Or was my message just over > everyone's head? > > If anyone is interested I could do a screencast on using BioRuby and > BioJava on the JVM. But I fear, somehow, the effort is wasted. > I, for one, would not consider that a wasted effort, and would invite others in my community to watch (assuming scheduling worked out). -- Russell Whitaker http://twitter.com/OrthoNormalRuss http://orthonormalruss.blogspot.com/ From russell.whitaker at gmail.com Sun Dec 12 18:30:06 2010 From: russell.whitaker at gmail.com (Russell Whitaker) Date: Sun, 12 Dec 2010 10:30:06 -0800 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: On Sun, Dec 12, 2010 at 10:17 AM, benjamin Almeida wrote: > I am interested too, so far I only used jruby (with bioruby) to have access > to neo4j and because it was faster. But my colleague uses scala so this is > definitely interesting to see how someone combines these two! > cheers > ben > CC'ing back to the list, since you accidentally replied privately. -- Russell Whitaker http://twitter.com/OrthoNormalRuss http://orthonormalruss.blogspot.com/ From pjotr.public14 at thebird.nl Sun Dec 12 18:40:43 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sun, 12 Dec 2010 19:40:43 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: <20101212184043.GA29332@thebird.nl> > On Sun, Dec 12, 2010 at 10:17 AM, benjamin Almeida > I am interested too, so far I only used jruby (with bioruby) to have access > to neo4j and because it was faster. But my colleague uses scala so this is > definitely interesting to see how someone combines these two! > cheers > ben >From my perspective neo4j looks interesting too - looks like a natural fit with the semantic web. Pj. From cjfields at illinois.edu Sun Dec 12 19:19:52 2010 From: cjfields at illinois.edu (Chris Fields) Date: Sun, 12 Dec 2010 13:19:52 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101212164709.GA27907@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101212164709.GA27907@thebird.nl> Message-ID: On Dec 12, 2010, at 10:47 AM, Pjotr Prins wrote: > On Sat, Dec 11, 2010 at 10:46:54AM +0100, Pjotr Prins wrote: >> We have a special community, with a special language. Ruby is one of >> the most fun languages to work with, and we know it. > > The response is overwhelming. Does it mean no one has fun anymore? Or > is Ruby dying a slow death in bioinformatics? Or was my message just over > everyone's head? > > If anyone is interested I could do a screencast on using BioRuby and > BioJava on the JVM. But I fear, somehow, the effort is wasted. > > Pj. Pjotr, I think, with efforts like this you'll find interest but also the need to demonstrate the utility of what you are proposing. An email may not be the best medium for planning things out along those lines. Also, users may be interested but simply may not have the time or resources to donate to such an effort unless they have a vested interest in it, what I find to be a common problem in many informatics-related community projects. Some commonly-used projects seem to have a 'bus factor' of 1 for this reason. A screencast might go a long way to helping with both of those. chris From francesco.strozzi at gmail.com Sun Dec 12 19:43:15 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Sun, 12 Dec 2010 20:43:15 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101211094654.GA19598@thebird.nl> References: <20101211094654.GA19598@thebird.nl> Message-ID: Hi Piotr, It's been a while since I started collaborating with Jan Aerts to develop the Ruby Ensembl API. It was a good chance for me to learn Ruby and develop bioinformatics programs. But I have to admit that, as you said, Ruby isn't so powerful for bioinformatics compared to other languages. So, like you, I feel the need to add another language for "real" and "though" bioinformatics work and I have started learning Python in the last months which it's really impressive. The core library is very poweful and BioPython is updated every 3 months (so it is under constant development). The performance of many common tasks (e.g. parsing a large Blast report) are superior than with BioRuby libraries, for what I have seen. But Ruby syntax and logic is simply amazing (and to me even better than Python) and it's a shame we don't have powerful solutions to bring Ruby out of the "just for the web" corner where it was put since Rails outbreak. So, from my point of view, every effort that could make Ruby shine for bioinformatics...well, is really really appreciated and I want to hear and read more! >From the technical point of view I have a couple of (newbie) questions: - I have no idea what Clojure and Scala are, so a sort of "Functional Programming for Dummies" will be appreciated. - How much Java one need to know in order to be productive with Ruby/BioRuby and Java/BioJava using JRuby? Because from my point of view, a Ruby programmer I think doesn't really want to learn Java (which is a big language but with it's own issues and is not so flexible as Ruby or Python). So in the end, I will read and hear with interest what you can tell us about your experience and future directions for Ruby/BioRuby development. Cheers Francesco On Sat, Dec 11, 2010 at 10:46, Pjotr Prins wrote: > Hi Rubyistas, > > We have a special community, with a special language. Ruby is one of > the most fun languages to work with, and we know it. > > Here I want to argue that for bioinformatics JRuby is one of the most > exciting developments. Not only is it pretty fast, once compiled, > but it also allows easy integration with Java (and BioJava). Before > you recoil in horror, it also allows integration with some really cool > programming languages, i.e. Scala, Clojure and Groovy. > > You know Ruby is great. But it has some weaknesses too. For > Bioinformatics (B) and big data (BD) the problems are: > > (1) Weak B functionality > (2) BD performance issues > (3) So so parallel computing support (for BD) > (4) Only partial functional programming support > > (1) and (2) can be resolved by using JRuby, BioJava and the JVM. (3) > and (4) can be resolved by tapping into Scala and Clojure. > > Let me try to explain. > > (1) Weak B functionality: BioRuby is a great achievement, but I have a > number of criticisms. First is that it is not suitable for BD. Almost > every module loads all data in RAM, and there is no concept of > parallel computations in the design. Finally the development is not > fast - we are suffering from the fact that we are a small community. > You could argue about reasons, but I don't think we should spend > energy on the past, when there is such an obvious way forward. Let me > continue. > > (2) BD performance issues: Ruby is slow. By definition compared to > statically typed and compiled languages (such as C, Java, Scala, > Clojure). It is pretty amazing to see how much speed Ruby 1.9 has. > But, for BD it breaks down quickly. Ruby's strength is in beauty of > code, but not in raw power. > > (3) So so parallel computing support (for BD): Functional languages > (Haskell, Erlang, Scala, Clojure) have immutable data, and > abstractions for parallelization, such as shared memory and actors, > which make it much easier to write parallelized code. For performance > and BD, this is extremely useful. > > (4) Only partial functional programming support: Once you get into > functional programming you realize Ruby gets in the way. Support for > functional programming in Ruby is patchy, though there is some. > > It is no accident that I have started the BioScala project, and Jan Aerts > has > started the BioClojure project in 2010 (!) BioRuby has spin-offs. > > My experience with Scala has been great. Scala is statically typed, > and very fast. It also allows beautiful code with functional > programming and parallelization thrown in. For me, there is a clear > path where I use Ruby and Scala on a 50/50 basis. Essentially using > the best of both worlds. JRuby is key to combining them. > > And, you know what? It is great fun!! I would get frustrated if I was > locked in either language. But now it is seamless moving between the > two, thanks to the JVM. Which, btw, these days can outperform even C > code. > > Believe me, even two years ago, I would not have thought I would > *ever* champion the JVM. But as a saaientist, you go by evidence. > > Programming is fun. And it has not ever been this great. I want to > share that with you, and I would like to use the coming holiday > season, and years after, to pass that on. I would guess Jan thinks the > same way about Ruby and Clojure. > > Who is interested in getting back into the fun of programming? Who > wants to experiment and become an even more productive programmer? > It could be the goal of BioRuby in 2011 to show the way to other Bio* > projects of handling development in such a way that we can easily move > between the strengths of dynamic programming languages and high > performant functional languages. > > I would like that. > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Mon Dec 13 11:42:38 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 13 Dec 2010 12:42:38 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> Message-ID: <20101213114238.GA32702@thebird.nl> On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: > Ruby isn't so powerful for bioinformatics compared to other > languages. So, like you, I feel the need to add another language for > "real" and "though" bioinformatics work and I have started learning > Python in the last months which it's really impressive. The core > library is very poweful and BioPython is updated every 3 months (so > it is under constant development). Main comment would be that Python is not much different from Ruby. It is interpreted, dymamic, and does not have good support for parallel programming, nor the speed required for big data (BD). Perl, Python and Ruby are really in the same boat. > The performance of many common tasks (e.g. parsing a large Blast > report) are superior than with BioRuby libraries, for what I have > seen. Here you have a good point. BioRuby is lagging behind BioPython, BioPerl and BioJava. My proposal is to use BioJava together with BioRuby. You can do something similar with Jython (Python on the JVM), though I have the impression Jython is not as active as JRuby, anymore. JRuby is getting strong industry support, at this point. > But Ruby syntax and logic is simply amazing (and to me even > better than Python) and it's a shame we don't have powerful > solutions to bring Ruby out of the "just for the web" corner where > it was put since Rails outbreak. I know people who would disagree with that last statement... To me, Ruby is a way of thinking, which is reflected in Rails. > So, from my point of view, every effort > that could make Ruby shine for bioinformatics...well, is really really > appreciated and I want to hear and read more! > From the technical point of view I have a couple of (newbie) questions: > - I have no idea what Clojure and Scala are, so a sort of "Functional > Programming for Dummies" will be appreciated. There are plenty online resources. The main Scala book is great. > - How much Java one need to know in order to be productive with > Ruby/BioRuby and Java/BioJava using JRuby? Not Java per se. Mostly it is getting used to the JVM world and docs. > Because from my point of view, a Ruby programmer I think doesn't really > want to learn Java (which is a big language but with it's own issues and is > not so flexible as Ruby or Python). I agree. I personally dislike the Java language. After a project around 2001 I vowed not to touch it again. It is not a hard language, but merely a pain. I liked any of the other Bio* languages better. > So in the end, I will read and hear with interest what you can tell us about > your experience and future directions for Ruby/BioRuby development. My suggestion would be to study a language more removed from Python, Ruby and Perl. I will try to make a screencast. Pj. From bonnalraoul at ingm.it Mon Dec 13 12:29:19 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Mon, 13 Dec 2010 13:29:19 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213114238.GA32702@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: +1 for screen cast which demostrates how jruby + scala or clojure and parallel programming can outperform a regular ruby script. I have to admit that parallel programming in Ruby is not very different from years ago however there are interesting methods for implementing it -I can't fine the title of a Ruby's book on this topic-. Unfortunately Ruby is a terrific language which suffers from the association Ruby => Rails Other languages have a lot of libraries which are very solid, mathematics, statistics and charting/plotting which I think are the main libs that we need in our everyday work. I was fascinated from JRuby and the ability to use java libraries is a +1 from my computer scientist/programmer point of view. I could use R whit a very solid tcp protocol for example. The same thing is not possible with Ruby 1.9 or 1.8 because the actual implementation lacks complex handling of dataframe. Btw... I think that we should poll here who wants to actively contribute to BioRuby ? Then I think we should address what we need to use bioruby/ruby in a more productive way or easiest way. @Francesco: which are the motivation or real case studies which drive you to shift from Ruby to Python ? Jan did a very interesting post on Clojure about how easy it's to plot data for NGS, are there alternative in ruby or is there someone here that faced the same problem and tried to find a solution with BioRuby ? A library that I started to use some time ago is http://ruby-statsample.rubyforge.org/ it's not R but I got the same results whitout the need to use R inside my script. Why is there a lot of interest of Jruby? My opinion is: Containers and Rails. I you want to use a rails application in an enterprise company you need to put it into a container: glassfish, jboss, Oracle. The only problem with Java for me is Oracle itself but... I'm quite scared about that. Ciao. Il giorno 13/dic/2010, alle ore 12.42, Pjotr Prins ha scritto: > On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: >> Ruby isn't so powerful for bioinformatics compared to other >> languages. So, like you, I feel the need to add another language for >> "real" and "though" bioinformatics work and I have started learning >> Python in the last months which it's really impressive. The core >> library is very poweful and BioPython is updated every 3 months (so >> it is under constant development). > > Main comment would be that Python is not much different from Ruby. It > is interpreted, dymamic, and does not have good support for parallel > programming, nor the speed required for big data (BD). Perl, Python > and Ruby are really in the same boat. > >> The performance of many common tasks (e.g. parsing a large Blast >> report) are superior than with BioRuby libraries, for what I have >> seen. > > Here you have a good point. BioRuby is lagging behind BioPython, > BioPerl and BioJava. > > My proposal is to use BioJava together with BioRuby. You can do > something similar with Jython (Python on the JVM), though I have the > impression Jython is not as active as JRuby, anymore. JRuby is > getting strong industry support, at this point. > >> But Ruby syntax and logic is simply amazing (and to me even >> better than Python) and it's a shame we don't have powerful >> solutions to bring Ruby out of the "just for the web" corner where >> it was put since Rails outbreak. > > I know people who would disagree with that last statement... > > To me, Ruby is a way of thinking, which is reflected in Rails. > >> So, from my point of view, every effort >> that could make Ruby shine for bioinformatics...well, is really really >> appreciated and I want to hear and read more! >> From the technical point of view I have a couple of (newbie) questions: >> - I have no idea what Clojure and Scala are, so a sort of "Functional >> Programming for Dummies" will be appreciated. > > There are plenty online resources. The main Scala book is great. > >> - How much Java one need to know in order to be productive with >> Ruby/BioRuby and Java/BioJava using JRuby? > > Not Java per se. Mostly it is getting used to the JVM world and docs. > >> Because from my point of view, a Ruby programmer I think doesn't really >> want to learn Java (which is a big language but with it's own issues and is >> not so flexible as Ruby or Python). > > I agree. I personally dislike the Java language. > > After a project around 2001 I vowed not to touch it again. It is not a > hard language, but merely a pain. I liked any of the other Bio* > languages better. > >> So in the end, I will read and hear with interest what you can tell us about >> your experience and future directions for Ruby/BioRuby development. > > My suggestion would be to study a language more removed from Python, > Ruby and Perl. > > I will try to make a screencast. > > Pj. > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From francesco.strozzi at gmail.com Mon Dec 13 13:26:34 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 13 Dec 2010 14:26:34 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: On Mon, Dec 13, 2010 at 13:29, Raoul Bonnal wrote: > > > Unfortunately Ruby is a terrific language which suffers from the > association Ruby => Rails > Yes I agree, that's what I was saying when I wrote about the "just for the web" corner. Rails is amazing and I love it, but I think now Ruby is seen more as the language for Rails and less as a robust language that could do almost the same things as Python, in my opinion. > > @Francesco: which are the motivation or real case studies which drive you > to shift from Ruby to Python ? > Basically the Bio library usage. Reading through the docs and using the different libraries, I found BioPython to be more complete and with a better performance, in particular for parsing big files (like Blast reports, assembly data, NGS files etc.). And the math libraries are amazing. With matplotlib and NumPy/SciPy you could avoid using R for what I have seen. And the documentation is vast. Also Python syntax is quite similar to Ruby in many ways, so moving from one to another wasn't so hard (even if mastering a language is not a matter of few months of practice). In the end I found myself running things in Ruby to develop web applications(!) or for the Ruby Ensembl API, and doing everything else with Python...that's the reason why I have written here to give my "+1", as I completely agree with Pjotr and I'd like to learn new ways of using Ruby (with Scala/Clojure or whatever) to do the "hard" work as well... Ciao -- Francesco From pjotr.public14 at thebird.nl Mon Dec 13 13:57:03 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 13 Dec 2010 14:57:03 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: <20101213135703.GA2024@thebird.nl> For me it was the other way round. Web programming in Python, wrote ZopeInAnger, and programmed everything else in Ruby :) I agree Python has some really nice projects for Science. I mentioned matplotlib before. On Mon, Dec 13, 2010 at 02:26:34PM +0100, Francesco Strozzi wrote: > I have written here to give my "+1", as I completely agree with > Pjotr and I'd like to learn new ways of using Ruby (with > Scala/Clojure or whatever) to do the "hard" work as well... It was not so long ago I would do the performance stuff in C. BioLib being a point in case. C/BioLib is also a valid route. I find, however, that for programming C/C++ bogs me down in implementation detail. Something I don't have in Ruby, Python or Scala. Ruby/Perl/Python wins on rapid/agile, and even dirty coding. But if it comes to execution speed and correctness - the new JVM languages deliver. Part of my excitement is mixing the two ways of programming. Note, the BioLib project is not dead. For legacy code, such as EMBOSS, it is very useful. I will make sure it works from the JVM. I have had little time for coding the last 4 months, but it should be better in 2011. Pj. From biopython at maubp.freeserve.co.uk Mon Dec 13 14:44:16 2010 From: biopython at maubp.freeserve.co.uk (Peter) Date: Mon, 13 Dec 2010 14:44:16 +0000 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213114238.GA32702@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: On Mon, Dec 13, 2010 at 11:42 AM, Pjotr Prins wrote: > > On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: >> The performance of many common tasks (e.g. parsing a large Blast >> report) are superior than with BioRuby libraries, for what I have >> seen. > > Here you have a good point. BioRuby is lagging behind BioPython, > BioPerl and BioJava. Part of how we try to handle big data files in Biopython is using Python iterators, whereby the file is loaded record by record (how depends on the file format - for BLAST we do this query by query), not all into memory in one go. I think BioPerl does something very similar in their parsers, I'm not so familiar with BioJava. > My proposal is to use BioJava together with BioRuby. You can do > something similar with Jython (Python on the JVM), though I have the > impression Jython is not as active as JRuby, anymore. JRuby is > getting strong industry support, at this point. Just for reference, we do test Biopython on Jython (i.e. on the JVM) and most things work. There is a significant downside that you can't use compiled C code, in particular this means you can't use NumPy under Jython (the numerical python library, used in matplotlib and SciPy). Nevertheless, some people are using Biopython on Jython. Peter From bonnalraoul at ingm.it Mon Dec 13 17:57:39 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Mon, 13 Dec 2010 18:57:39 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> Message-ID: <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> Il giorno 13/dic/2010, alle ore 15.44, Peter ha scritto: > On Mon, Dec 13, 2010 at 11:42 AM, Pjotr Prins wrote: >> >> On Sun, Dec 12, 2010 at 08:43:15PM +0100, Francesco Strozzi wrote: >>> The performance of many common tasks (e.g. parsing a large Blast >>> report) are superior than with BioRuby libraries, for what I have >>> seen. >> >> Here you have a good point. BioRuby is lagging behind BioPython, >> BioPerl and BioJava. > > Part of how we try to handle big data files in Biopython is using > Python iterators, whereby the file is loaded record by record (how > depends on the file format - for BLAST we do this query by query), > not all into memory in one go. I think BioPerl does something very > similar in their parsers, I'm not so familiar with BioJava. This is the right approach. Junior Programmers tend to load everything and/or create huge hashes, at the same time the old ruby interpreter had a bad garbage collector. It's always a matter of coding if we are discussing about performances each language can have worst performances. If scala can be used to easily express the concept or better help in parallel programming why not trying it? > >> My proposal is to use BioJava together with BioRuby. You can do >> something similar with Jython (Python on the JVM), though I have the >> impression Jython is not as active as JRuby, anymore. JRuby is >> getting strong industry support, at this point. > > Just for reference, we do test Biopython on Jython (i.e. on the JVM) > and most things work. There is a significant downside that you can't > use compiled C code, in particular this means you can't use NumPy > under Jython (the numerical python library, used in matplotlib and > SciPy). Nevertheless, some people are using Biopython on Jython. > >From my point of view Python guys are doing a very good job on all fields. Unfortunately I'm in love with ruby :-) -- Ra From pjotr.public14 at thebird.nl Mon Dec 13 18:16:59 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 13 Dec 2010 19:16:59 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> Message-ID: <20101213181659.GA4269@thebird.nl> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > > Part of how we try to handle big data files in Biopython is using > > Python iterators, whereby the file is loaded record by record (how > > depends on the file format - for BLAST we do this query by query), > > not all into memory in one go. I think BioPerl does something very > > similar in their parsers, I'm not so familiar with BioJava. BioJava uses a visitor pattern. In effect an iterator. With all current implementations IO runs, then code, the IO, etc. While we are IO constrained, we are actually doing worse. What I want is an IO thread going at maximum throughput. Every item should get parcelled out for further parsing and processing, in parallel to the IO thread. We should do better, and make it a generalization. I think we can do it by using Scala and the standard BioJava iterators. With Scala it can be turned in a parallelized iterator. That is a fun project. > From my point of view Python guys are doing a very good job on all fields. > Unfortunately I'm in love with ruby :-) All you need is love :) Pj. From cjfields at illinois.edu Mon Dec 13 21:48:23 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 13 Dec 2010 15:48:23 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213181659.GA4269@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> Message-ID: <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: >>> Part of how we try to handle big data files in Biopython is using >>> Python iterators, whereby the file is loaded record by record (how >>> depends on the file format - for BLAST we do this query by query), >>> not all into memory in one go. I think BioPerl does something very >>> similar in their parsers, I'm not so familiar with BioJava. > > BioJava uses a visitor pattern. In effect an iterator. > > With all current implementations IO runs, then code, the IO, etc. > While we are IO constrained, we are actually doing worse. > > What I want is an IO thread going at maximum throughput. Every item > should get parcelled out for further parsing and processing, in > parallel to the IO thread. > > We should do better, and make it a generalization. I think we can do > it by using Scala and the standard BioJava iterators. With Scala it > can be turned in a parallelized iterator. That is a fun project. > >> From my point of view Python guys are doing a very good job on all fields. >> Unfortunately I'm in love with ruby :-) > > All you need is love :) > > Pj. At some point the choice of a language will not matter as much, as long as it is implemented in a VM (something Perl 5 cannot claim at the moment, but Perl 6 does with the Parrot VM). chris From bonnalraoul at ingm.it Tue Dec 14 00:21:39 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 14 Dec 2010 01:21:39 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> Message-ID: <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Hi Chris, Il giorno 13/dic/2010, alle ore 22.48, Chris Fields ha scritto: > On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > > At some point the choice of a language will not matter as much, as long as it is implemented in a VM (something Perl 5 cannot claim at the moment, but Perl 6 does with the Parrot VM). The real problem was ruby 1.8, something in the middle between 1.9 and the previous release. One year ago JRuby wasn't complain with the 1.9 standard. The Idea to use the JVM is to reuse other libraries, at that point all the Bio* that can run in it could share the same library, it's not too bad. Ideally we could develop a common set of libraries but probably is too complex. There is also another problem, maintaining the current installations, how? Do you know which are the performances using different JVM ? Pjotr: do you mean an IO access in a map/reduce style/way? -- Ra From cjfields at illinois.edu Tue Dec 14 02:31:52 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 13 Dec 2010 20:31:52 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Message-ID: On Dec 13, 2010, at 6:21 PM, Raoul Bonnal wrote: > Hi Chris, > > Il giorno 13/dic/2010, alle ore 22.48, Chris Fields ha scritto: > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: >> >> At some point the choice of a language will not matter as much, as long as it is implemented in a VM (something Perl 5 cannot claim at the moment, but Perl 6 does with the Parrot VM). > The real problem was ruby 1.8, something in the middle between 1.9 and the previous release. One year ago JRuby wasn't complain with the 1.9 standard. > The Idea to use the JVM is to reuse other libraries, at that point all the Bio* that can run in it could share the same library, it's not too bad. Ideally we could develop a common set of libraries but probably is too complex. > There is also another problem, maintaining the current installations, how? > Do you know which are the performances using different JVM ? No idea. The problem with lack of C-based bindings is a bit of a worry (Peter mentioned this in a previous post in this thread). -c > Pjotr: do you mean an IO access in a map/reduce style/way? > > -- > Ra From pjotr.public14 at thebird.nl Tue Dec 14 07:13:50 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 14 Dec 2010 08:13:50 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Message-ID: <20101214071350.GA6860@thebird.nl> On Tue, Dec 14, 2010 at 01:21:39AM +0100, Raoul Bonnal wrote: > Pjotr: do you mean an IO access in a map/reduce style/way? No. Mapreduce only works when you can reduce data into independent packages of work. I find it does not generally work in biological research. Way overrated. Py. From pjotr.public14 at thebird.nl Tue Dec 14 07:32:28 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 14 Dec 2010 08:32:28 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> Message-ID: <20101214073228.GB6860@thebird.nl> On Mon, Dec 13, 2010 at 08:31:52PM -0600, Chris Fields wrote: > At some point the choice of a language will not matter as much, as > long as it is implemented in a VM (something Perl 5 cannot claim at > the moment, but Perl 6 does with the Parrot VM). In my opinion, multiple VMs will only serve the diaspora in bioinformatics. One VM would be optimal to make effective use of shared competencies. It is pretty clear (to me) that Perl is going it alone... Like the Erlang VM - it becomes a handicap, rather than an asset. > No idea. The problem with lack of C-based bindings is a bit of a > worry (Peter mentioned this in a previous post in this thread). It is no real problem to bind C to the JVM, using SWIG (for example). No worse than binding to Perl5. I am planning as much, since I don't want to drop the C projects from the equation. Nice point is that, a C/C++/FORTRAN lib once bound against the JVM, will work for all JVM languages. So, if, for example, the BioPython would resolve their math C bindings, and make Jython work, it would naturally translate to Java, Ruby, Scala, Clojure (which btw may be an R replacement), Groovy, and any future language on the JVM. Now, *that*, my friends, would be a major step forward in bioinformatics. Not only can we build on each others work, but also, each of us can work in a language of choice. Pj. From bonnalraoul at ingm.it Tue Dec 14 10:46:37 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 14 Dec 2010 11:46:37 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101213181659.GA4269@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> Message-ID: <5D135AF4-797E-4F45-948C-59E5CC433D2B@ingm.it> Il giorno 13/dic/2010, alle ore 19.16, Pjotr Prins ha scritto: > With all current implementations IO runs, then code, the IO, etc. > While we are IO constrained, we are actually doing worse. > > What I want is an IO thread going at maximum throughput. Every item > should get parcelled out for further parsing and processing, in > parallel to the IO thread. This is not clear to me, could you make a real example ? -- Ra From pjotr.public14 at thebird.nl Tue Dec 14 12:54:31 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 14 Dec 2010 13:54:31 +0100 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <5D135AF4-797E-4F45-948C-59E5CC433D2B@ingm.it> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <5D135AF4-797E-4F45-948C-59E5CC433D2B@ingm.it> Message-ID: <20101214125431.GC9257@thebird.nl> On Tue, Dec 14, 2010 at 11:46:37AM +0100, Raoul Bonnal wrote: > This is not clear to me, could you make a real example ? With FASTA, GFF3 etc. part of the job is reading, part is parsing. These can be on multiple threads. With GFF3 there are dependencies. So if it is not all in memory you get multiple readers and multiple parsers. I'll do an example, at some point. Pj. From cjfields at illinois.edu Tue Dec 14 13:05:31 2010 From: cjfields at illinois.edu (Chris Fields) Date: Tue, 14 Dec 2010 07:05:31 -0600 Subject: [BioRuby] Bringing the fun back to programming! (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101214073228.GB6860@thebird.nl> References: <20101211094654.GA19598@thebird.nl> <20101213114238.GA32702@thebird.nl> <7B5C3890-86F5-4939-B204-C6717214A26B@ingm.it> <20101213181659.GA4269@thebird.nl> <3BA6A7C6-9B00-4A0D-8EB1-392D71808D55@illinois.edu> <277D91B1-B1CD-4CAB-BE4B-F9312F551B9A@ingm.it> <20101214073228.GB6860@thebird.nl> Message-ID: <9C9CE549-854B-4EEF-9E00-B0D31C8E2D65@illinois.edu> On Dec 14, 2010, at 1:32 AM, Pjotr Prins wrote: > On Mon, Dec 13, 2010 at 08:31:52PM -0600, Chris Fields wrote: >> At some point the choice of a language will not matter as much, as >> long as it is implemented in a VM (something Perl 5 cannot claim at >> the moment, but Perl 6 does with the Parrot VM). > > In my opinion, multiple VMs will only serve the diaspora in > bioinformatics. One VM would be optimal to make effective use of > shared competencies. It is pretty clear (to me) that Perl is going > it alone... Like the Erlang VM - it becomes a handicap, rather than an > asset. Much of this is due to the complexities involved in parsing the language. It's long been said that only perl can parse perl, but that's changing with perl 6. >> No idea. The problem with lack of C-based bindings is a bit of a >> worry (Peter mentioned this in a previous post in this thread). > > It is no real problem to bind C to the JVM, using SWIG (for example). > No worse than binding to Perl5. I am planning as much, since I don't > want to drop the C projects from the equation. Nice point is that, a > C/C++/FORTRAN lib once bound against the JVM, will work for all JVM > languages. > > So, if, for example, the BioPython would resolve their math C > bindings, and make Jython work, it would naturally translate to Java, > Ruby, Scala, Clojure (which btw may be an R replacement), Groovy, and > any future language on the JVM. Now, *that*, my friends, would be a > major step forward in bioinformatics. Not only can we build on each > others work, but also, each of us can work in a language of choice. > > Pj. The same exists for Parrot (the NCI interface) and likely must exist for any successful VM. The preliminary DBI for Perl 6 uses it. chris From john.woods at marcottelab.org Wed Dec 15 23:02:27 2010 From: john.woods at marcottelab.org (John Woods) Date: Wed, 15 Dec 2010 17:02:27 -0600 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) Message-ID: Hi everyone, I wanted to put in my two cents about this (not this specific email, but the thread). I come from a CS background, and so when I began working in bioinformatics, I shifted from C++ to Python, Perl, back to Python, and finally to Ruby (and Rails!). Working in a bioinformatics lab, my experience has been that most people go with Python over Ruby because of SciPy and NumPy (and matplotlib). At least for us, it's not because of speed at all. If I need something speedy, I code it in C++ and use Rice to build a Ruby adapter. Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and with it, good numerical support). To that end, we've been working a bit to put together a spec for how a SciRuby with plotting and numeric capabilities would work. I'm not quite ready to say anything specific about it publicly just yet, but if you're interested in being involved, please write me back. Best, John O. Woods NSF Fellow The University of Texas at Austin On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields wrote: > On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > > > On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > >>> Part of how we try to handle big data files in Biopython is using > >>> Python iterators, whereby the file is loaded record by record (how > >>> depends on the file format - for BLAST we do this query by query), > >>> not all into memory in one go. I think BioPerl does something very > >>> similar in their parsers, I'm not so familiar with BioJava. > > > > BioJava uses a visitor pattern. In effect an iterator. > > > > With all current implementations IO runs, then code, the IO, etc. > > While we are IO constrained, we are actually doing worse. > > > > What I want is an IO thread going at maximum throughput. Every item > > should get parcelled out for further parsing and processing, in > > parallel to the IO thread. > > > > We should do better, and make it a generalization. I think we can do > > it by using Scala and the standard BioJava iterators. With Scala it > > can be turned in a parallelized iterator. That is a fun project. > > > >> From my point of view Python guys are doing a very good job on all > fields. > >> Unfortunately I'm in love with ruby :-) > > > > All you need is love :) > > > > Pj. > > At some point the choice of a language will not matter as much, as long as > it is implemented in a VM (something Perl 5 cannot claim at the moment, but > Perl 6 does with the Parrot VM). > > chris > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From bonnalraoul at ingm.it Thu Dec 16 00:29:44 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 16 Dec 2010 01:29:44 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: Message-ID: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> +1 for me. Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there is also an optimized version. John, in the web era a way to easily publish plots on the web not just as static images would be great. Probably I could not attend the first meeting, work trip. Btw if someone is around these day I'll try to be in the channel. Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: > Hi everyone, > > I wanted to put in my two cents about this (not this specific email, but the > thread). > > I come from a CS background, and so when I began working in bioinformatics, > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby (and > Rails!). > > Working in a bioinformatics lab, my experience has been that most people go > with Python over Ruby because of SciPy and NumPy (and matplotlib). At least > for us, it's not because of speed at all. If I need something speedy, I code > it in C++ and use Rice to build a Ruby adapter. > > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and > with it, good numerical support). > > To that end, we've been working a bit to put together a spec for how a > SciRuby with plotting and numeric capabilities would work. > > I'm not quite ready to say anything specific about it publicly just yet, but > if you're interested in being involved, please write me back. > > Best, > John O. Woods > NSF Fellow > The University of Texas at Austin > > > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields wrote: > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: >> >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: >>>>> Part of how we try to handle big data files in Biopython is using >>>>> Python iterators, whereby the file is loaded record by record (how >>>>> depends on the file format - for BLAST we do this query by query), >>>>> not all into memory in one go. I think BioPerl does something very >>>>> similar in their parsers, I'm not so familiar with BioJava. >>> >>> BioJava uses a visitor pattern. In effect an iterator. >>> >>> With all current implementations IO runs, then code, the IO, etc. >>> While we are IO constrained, we are actually doing worse. >>> >>> What I want is an IO thread going at maximum throughput. Every item >>> should get parcelled out for further parsing and processing, in >>> parallel to the IO thread. >>> >>> We should do better, and make it a generalization. I think we can do >>> it by using Scala and the standard BioJava iterators. With Scala it >>> can be turned in a parallelized iterator. That is a fun project. >>> >>>> From my point of view Python guys are doing a very good job on all >> fields. >>>> Unfortunately I'm in love with ruby :-) >>> >>> All you need is love :) >>> >>> Pj. >> >> At some point the choice of a language will not matter as much, as long as >> it is implemented in a VM (something Perl 5 cannot claim at the moment, but >> Perl 6 does with the Parrot VM). >> >> chris >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From john.woods at marcottelab.org Thu Dec 16 00:50:54 2010 From: john.woods at marcottelab.org (John Woods) Date: Wed, 15 Dec 2010 18:50:54 -0600 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> Message-ID: I saw ruby-statsample, and it looks like one of many things we'd (ideally) like to integrate. I'm glad you brought up publishing to the web. We spent a not-insignificant amount of time debating graphics platform, and ultimately we decided that HTML5 canvas, and SVG-type graphics, were probably the ideal. For the moment, we're looking at protovis. Interestingly, Claudio Bustos has been working on a Ruby protovis port (he also wrote ruby-statsample, I believe). The other big piece is NArray. Masahiro Tanaka is apparently rewriting it with a goal of getting it incorporated into Ruby core. John Prince has been working on documentation for the existing NArray, in part to see if it's got everything SciRuby would need. But ultimately, the NArray rewrite/core integration is the key. It's years from being finished, supposedly -- but perhaps if there are some eager volunteers we could speed that up. =) So it seems like most of the pieces are present -- it's just a matter of tying them together. And I just said I wasn't going to get into detail -- but there, you just got me going. =) John On Wed, Dec 15, 2010 at 6:29 PM, Raoul Bonnal wrote: > +1 for me. > Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there > is also an optimized version. > > John, in the web era a way to easily publish plots on the web not just as > static images would be great. > > Probably I could not attend the first meeting, work trip. Btw if someone is > around these day I'll try to be in the channel. > > Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: > > > Hi everyone, > > > > I wanted to put in my two cents about this (not this specific email, but > the > > thread). > > > > I come from a CS background, and so when I began working in > bioinformatics, > > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby > (and > > Rails!). > > > > Working in a bioinformatics lab, my experience has been that most people > go > > with Python over Ruby because of SciPy and NumPy (and matplotlib). At > least > > for us, it's not because of speed at all. If I need something speedy, I > code > > it in C++ and use Rice to build a Ruby adapter. > > > > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have > been > > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby > (and > > with it, good numerical support). > > > > To that end, we've been working a bit to put together a spec for how a > > SciRuby with plotting and numeric capabilities would work. > > > > I'm not quite ready to say anything specific about it publicly just yet, > but > > if you're interested in being involved, please write me back. > > > > Best, > > John O. Woods > > NSF Fellow > > The University of Texas at Austin > > > > > > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields > wrote: > > > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > >> > >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > >>>>> Part of how we try to handle big data files in Biopython is using > >>>>> Python iterators, whereby the file is loaded record by record (how > >>>>> depends on the file format - for BLAST we do this query by query), > >>>>> not all into memory in one go. I think BioPerl does something very > >>>>> similar in their parsers, I'm not so familiar with BioJava. > >>> > >>> BioJava uses a visitor pattern. In effect an iterator. > >>> > >>> With all current implementations IO runs, then code, the IO, etc. > >>> While we are IO constrained, we are actually doing worse. > >>> > >>> What I want is an IO thread going at maximum throughput. Every item > >>> should get parcelled out for further parsing and processing, in > >>> parallel to the IO thread. > >>> > >>> We should do better, and make it a generalization. I think we can do > >>> it by using Scala and the standard BioJava iterators. With Scala it > >>> can be turned in a parallelized iterator. That is a fun project. > >>> > >>>> From my point of view Python guys are doing a very good job on all > >> fields. > >>>> Unfortunately I'm in love with ruby :-) > >>> > >>> All you need is love :) > >>> > >>> Pj. > >> > >> At some point the choice of a language will not matter as much, as long > as > >> it is implemented in a VM (something Perl 5 cannot claim at the moment, > but > >> Perl 6 does with the Parrot VM). > >> > >> chris > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > From pjotr.public14 at thebird.nl Thu Dec 16 13:26:58 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 16 Dec 2010 14:26:58 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: Message-ID: <20101216132658.GA21601@thebird.nl> On Wed, Dec 15, 2010 at 05:02:27PM -0600, John Woods wrote: > Working in a bioinformatics lab, my experience has been that most people go > with Python over Ruby because of SciPy and NumPy (and matplotlib). At least > for us, it's not because of speed at all. If I need something speedy, I code > it in C++ and use Rice to build a Ruby adapter. That is what I always did. Reason to switch to functional languages are that they are stronger with parallelization abstractions (and shared memory). C++ can do parallelization, but it is harder work and more dangerous. > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and > with it, good numerical support). Great! Be good if it supports JRuby! > To that end, we've been working a bit to put together a spec for how a > SciRuby with plotting and numeric capabilities would work. I did a GSL port to Ruby https://github.com/pjotrp/ascilib using SWIG. It may not interest you, but it does support Perl too, and probably soon the JVM. Using SWIG is not that hard, and the advantage is a wider target. > I'm not quite ready to say anything specific about it publicly just yet, but > if you're interested in being involved, please write me back. It is a good thing, thanks, looking forward to it. Pj. From bonnalraoul at ingm.it Thu Dec 16 17:20:53 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 16 Dec 2010 18:20:53 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: Message-ID: <20101216172053.6ae641ea@mail.ingm.it> Hi Claudio, I like your work on statsample. As others pointed out iterators are the keys so probably we need to invest some time in designg. Could we organize this math/stat package like a plugin for bioruby, what do you think? Then about graphics attaching the visualization concept to some object would be great not only static but also for HTML5 +1 for all these topis from me. Rserve can be used efficiently for parallelization, it's ability to have session and user authentication. Distributed Programming with Ruby it's a must. _____ From: Claudio Bustos [mailto:clbustos at gmail.com] To: John Woods [mailto:john.woods at marcottelab.org] Cc: Raoul Bonnal [mailto:bonnalraoul at ingm.it], Chris Fields [mailto:cjfields at illinois.edu], bioruby at lists.open-bio.org Sent: Thu, 16 Dec 2010 04:59:51 +0100 Subject: Re: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) Rubyvis fellows: I'm Claudio, developer of statsample, rubyvis and rserve-client for ruby. I will be very happy if I can help in any way in bioruby project. Statsample: I'll start a Master on Statistics next year, so the library will be actively developed. I'm very interested in implementation of iterator based datasets, to allow statsample process big datasets. Rubyvis: I will finish port of all Layout on January, I think. What about creating a builder for rubyvis to create protovis scripts using the same set of instructions to create static images?. We can have out cake and eat it :) Regards -- Claudio Bustos Mg. en Psicolog?a clbustos at gmail.com From pjotr.public14 at thebird.nl Thu Dec 16 18:22:04 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 16 Dec 2010 19:22:04 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101216172053.6ae641ea@mail.ingm.it> References: <20101216172053.6ae641ea@mail.ingm.it> Message-ID: <20101216182204.GB24908@thebird.nl> On Thu, Dec 16, 2010 at 06:20:53PM +0100, Raoul Bonnal wrote: > +1 for all these topis from me. Topi is a hat http://www.thefreedictionary.com/topi (also in Bengali) I hope everyone can make the first world-wide BioRuby IRC conference on Sunday! I am looking forward to it, and hope to have a 10min screencast on JRuby ready on Saturday. Pj. From bonnalraoul at ingm.it Thu Dec 16 18:47:42 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 16 Dec 2010 19:47:42 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <20101216182204.GB24908@thebird.nl> References: <20101216172053.6ae641ea@mail.ingm.it> <20101216182204.GB24908@thebird.nl> Message-ID: <5F9A7F53-A292-4E09-8774-2269213AC3EC@ingm.it> On 16/dic/2010, at 19.22, Pjotr Prins wrote: > On Thu, Dec 16, 2010 at 06:20:53PM +0100, Raoul Bonnal wrote: >> +1 for all these topis from me. > > Topi is a hat > > http://www.thefreedictionary.com/topi > > (also in Bengali) Thanks Pjotr. http://vimeo.com/14435288 -- Ra From andrew.j.grimm at gmail.com Sat Dec 18 06:04:37 2010 From: andrew.j.grimm at gmail.com (Andrew Grimm) Date: Sat, 18 Dec 2010 17:04:37 +1100 Subject: [BioRuby] IRC conference archives Message-ID: Will there be archives of the IRC conference? The conference appears to be at 1am Monday Sydney time. If not, I guess I could leave my computer on all night... Andrew From francesco.strozzi at gmail.com Sat Dec 18 07:13:10 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Sat, 18 Dec 2010 08:13:10 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: Message-ID: +1 for this. I will try to connect to the correct time but I may miss the first part. It will be nice to have an archive with the IRC conference and the materials presented (screencasts etc.). Thanks Francesco On Sat, Dec 18, 2010 at 07:04, Andrew Grimm wrote: > Will there be archives of the IRC conference? The conference appears > to be at 1am Monday Sydney time. > > If not, I guess I could leave my computer on all night... > > Andrew > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From pjotr.public14 at thebird.nl Sat Dec 18 12:10:24 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 18 Dec 2010 13:10:24 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: Message-ID: <20101218121024.GA456@thebird.nl> We will set up logging. No worries. On Sat, Dec 18, 2010 at 05:04:37PM +1100, Andrew Grimm wrote: > Will there be archives of the IRC conference? The conference appears > to be at 1am Monday Sydney time. > > If not, I guess I could leave my computer on all night... > > Andrew > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Sat Dec 18 15:59:38 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 18 Dec 2010 16:59:38 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <5F9A7F53-A292-4E09-8774-2269213AC3EC@ingm.it> References: <20101216172053.6ae641ea@mail.ingm.it> <20101216182204.GB24908@thebird.nl> <5F9A7F53-A292-4E09-8774-2269213AC3EC@ingm.it> Message-ID: <20101218155938.GA1756@thebird.nl> Below is the screencast on JRuby Raoul sent: > http://vimeo.com/14435288 It is pretty good, as it shows the steps of running JRuby, and how to leverage JVM tools. Note the following, JRuby has 1. Full threads support 2. Compiles Ruby to bytecode, which is optimized to native code, when run often enough (the exception being 'eval' statements). 3. You get the performance and debugging toolset that come with the JVM, including a profiler that is not much slower than native 4. You can ship Ruby code with all (gem) dependencies included in a JAR, which will run on ANY JVM. Well, Dalvik excepted ;). The main message is that you get Ruby with all Java/JVM goodies. Pj. From pjotr.public14 at thebird.nl Sun Dec 19 12:53:13 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sun, 19 Dec 2010 13:53:13 +0100 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: References: Message-ID: <20101219125313.GA6033@thebird.nl> Today is the day!! I use the bitchx client, with logging. Pj. On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: > Dear BioRuby developers, > > Here we announce the first BioRuby world wide IRC conference! > > The topic is: "Current and future BioRuby development" > > The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), > which is 15pm CET (Central European Time) and 9am EST (New York) and > 6am PST (California time). > > You are kindly invited to join! > > Server: irc.freenode.org > Channel: #bioruby > > Toshiaki Katayama on behalf of the BioRuby staff. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From ktym at hgc.jp Sun Dec 19 18:49:36 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Mon, 20 Dec 2010 03:49:36 +0900 Subject: [BioRuby] The first BioRuby IRC conference on Dec 19th In-Reply-To: <20101219125313.GA6033@thebird.nl> References: <20101219125313.GA6033@thebird.nl> Message-ID: <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> Hi all, Today, we had >10 participants and discussed about * Current and future of BioRuby plugin system * BioRuby with JVM for high performance computing * Regularly continue the IRC meeting? etc. Details will be reported from Pjotr. We agreed to continue this IRC meeting on every Thursday as a "core time", and we may also keep the #bioruby channel open for everyone as much as possible, I just setup an IRC bot to log discussions. >From now on, the log of this channel will be made public at http://bioruby.org/irc/ so please join us (and please pay attention to privacy as a netiquette as well ;-). Toshiaki On 2010/12/19, at 21:53, Pjotr Prins wrote: > Today is the day!! > > I use the bitchx client, with logging. > > Pj. > > On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >> Dear BioRuby developers, >> >> Here we announce the first BioRuby world wide IRC conference! >> >> The topic is: "Current and future BioRuby development" >> >> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >> which is 15pm CET (Central European Time) and 9am EST (New York) and >> 6am PST (California time). >> >> You are kindly invited to join! >> >> Server: irc.freenode.org >> Channel: #bioruby >> >> Toshiaki Katayama on behalf of the BioRuby staff. >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Tue Dec 21 09:18:37 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 21 Dec 2010 10:18:37 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101218121024.GA456@thebird.nl> References: <20101218121024.GA456@thebird.nl> Message-ID: <20101221091837.GA21147@thebird.nl> Below the first IRC conference in a long time. We are keeping the #bioruby room open on freenode. Every Thursday 2pm (GMT) we'll try to be online. This Thursday we can look at developing new plugins. For anyone interested in some Ruby programming over the holidays. Pj. IRC log started Sun Dec 19 13:51:39 2010 good "morning"... hi there Hello everyone! the first world wide BioRuby IRC conference :) Hi hi Hi all. Would it be useful to have very quick introductions from everyone? Less than or equal to 10 words? tktym should open first Hmm, I'm Toshiaki Katayama, one of founders of the BioRuby project. Nice to see you. Pjotr Prins: 50% BioRuby, 50% other - next year lots of NGS Jan Aerts. Ruby + beginnings of Clojure. Lots of NGS now + visualization. I'm Naohisa Goto, a core developer, and a release manager of BioRuby My work -- comparative and evolutionary functional genomics, software and algorithm development mostly in Ruby and Java (www.cmzmasek.net) I'm Hiro Mishima, using Ruby and Bioruby for human genetics, genomics, and NGS analysis Mitsuteru Nakao: a BioRuby founder, TogoWS, next year lots of SemanticWeb :-) George Githinji, PhD student, exploring sequence diversity and immunoinformatics next year hello hello, we've just finished self introductions from each other arg -sorry I was stuck with something. Is there a log already? I'm not sure. Pjotr? I am logging my miniintro: I'm mostly a bioruby user. I'm an evolutionary biologist trying to understand the genetic basis of social life in ant colonies.... after getting fed up with perl's weirdness I decided to use bioruby for most of my scripting ant colonies! sounds interesting OK, how do we start discussion? Is Raoul coming on the call? Would be useful for the plugin stuff. Probably, he is in Spain for his vacation. As for the plug-in system, we (ngoto, mn and me) tried to setup bio-geek and started to make the first plugin, last week. How did that go? And what plugin was that? Installation procedure was fine. I thought we should use a name "biogem" instead of the "biogeek". ruby <-> gem ; bioruby <-> biogem for symmetry I'd vote for that as well. and bioruby plugins are actually RubyGems. OK, I'll talk with Raoul on this. Then, biogem (biogeek) command created a template plugin directory structure. It is working very well, but we may need more templates with a lot of comments instead of empty files. the plugin system is looking good from a simple version (ruby only, 1 file) to a complex version (many files and may contain C code) The version we tried is the former, simple one, which was kind of Bio::ColorScheme extension. more info: http://bioruby.open-bio.org/wiki/Plugins but I'm not sure these descriptions are the latest version ;-) It seems not up-to-date obsolete OK, I'll contribute my installation log there. Anyway, my overall feeling about the plugin system was good and it seems a way to go. Additionally, I'd like to discuss about the inclusion of workflows (with data) as plugins. workflows sounds interesting... how do you see that? there are a lot of methods described in journal articles, and probably some of them can be coded in (Bio)Ruby if the procedure can be distributed as a plugin, (with example data) user can try the workflow with ease and with his own data as well That'd be really cool. Would fit in nicely with the new journal Open Research Computation. (disclaimer: am an editor) That'd be really cood -- agreed! integrating git for history? One of the issues will be that many workflows rely an very specific site-specific setups. I think this kind of plugins are something like R's package, which is one of the strength of R and BioConductor. yes, which also allows people to submit to journalx Jan, yes, but we can start with generic ones. I think what is required now is a few examples NGS-related would be nice :-) But that is at the moment not the strongest field in bioruby... Something phylo? gff3 yes, NGS with stats and/or visualization would be a candidate. GFF3 too. what NGS formats are you using? products of Google Summer of Code 2010: NeXML, gene duplocation algorithm fastq -> sam/bam -> pileup or VCF. @pjotr: biolib? biolib has sam/bam (samtools) biojava too biolib version is not complete What was in my mind : analysis methods (>200?) implemented in G-language http://www.g-language.org/wiki/documents picard = java library on top of samtools All: I have to rush off, unfortunately. @pjotr: will you send us the logs? yes - both biolib and picard are samtools I have made screencasts for JRuby but I have trouble getting the sound track right silent movie now anyway, coming soon can we watch the silent version? heh maybe not that good, it needs info OK I could not spare time to watch the movie, Raoul sent us a link before. http://vimeo.com/14435288 Hmm, "Rails on the JVM with JRuby" it is mostly about JRuby and tools worth watching? it seems the length of the movie > 1h! depends it gives an idea of JRuby and the JVM tools I think the JVM is a good way forward for BioRuby for BioRuby developers or users? both developers benefit from existing libraries and tools picard a point in case users benefit from packaged jars you can package everything together, including Is a ruby program packaged by using jar? jruby, scala, biojava, bioscala and gems (!) I'm still not sure because I suppose that most of BioRuby users are not that geeks ;) yes - that you can see in the video, all in one tktym: it is easy for the users, even one-click installs on most platforms OK, potentially it sounds very interesting. only thing that causes problems are gems with C libs but, I think we can avoid that for most use cases also the JVM is very fast these days there is no reason to program in C any more, except a few border line cases so, what kind of help do you need? I am definately heading this way for big data no help, per se I'm still not sure because I suppose that most of BioRuby users are not that geeks -- I would very about too... I am interested in finding people who are interested ChristianZmasek: what are you saying? Anyway, we can try both with plugins. Some plugins for high-end users can depend on JVM to BioScala but we can still use BioRuby as it is now. sure we can take it case by case right ruby 1.9 is pretty fast jruby will win with parallel programming that is my main focus pjotr: to me it sounds as if these recent proposals (also on the mailinglist) are moving away from what many like about Ruby: simplicity and elegance. ChristianZmasek: I think bioruby on its own is a lost cause we just don't have the momentum biopython is going to be successful bioperl may live for some time biojava is doing really well so where are we? wouldn't worry that by spreading out the few developers over more "projects" would make it even worse? ultimately bioruby is interpreter agnostic don't you think it is possible to speed up many/some things from within bioruby (more effiecient coding) than relying on numerous "outside" tools? a lot needs to happen with bioruby, to make it suitable for big data not only that I remember somebody writing about some module for numeric methonds in ruby -- that sound interesting... ruby/python/perl are not fast enough for churning 1TB (it is not only an IO problem) rbnum would be useful, though we have quite a bit of that already ruby/python/perl are not fast enough for 1TB -- agreed! Wouldn't even think about using scripting languages for that amount of data... how about Java/JVM? Java/Scala are strongly typed that means they can be highly optimized for speed The JVM used to be bad with memory but that has mostly gone now but I predict that people will go by use cases I feel the amount of data itself is not a problem of coding. We can use DBs and indexing systems optimized for that. If so, all we need inside bioruby is binding for those external data store? if I write a fast GFF3 Scala lib, that you can use from bioruby are you going to reimplement that? tktym: that requires DBs etc. do you really think a JVM dependency is harder for users? I always use DBs or TokyoCabinet or other indexing systems for speed. Most machines have a JVM now. 1. is it user friendly? anyway I wrote: Difficulty of depending JVM is something equivalent to have external DBs for ordinary users in this context. It depends, and it would be a requirement for some cases. sorry, pressed the wrong key ;) I think JVM is one way to go, but I also suppose plugins and workflow applications will attract people if we can focus our efforts to making really useful ones for daily research. yes Anyway, we can continue both approach. i disagree that JVM makes it harder. It makes it even easier that what we have now you'll see. OK, I'm looking forward your movie :) well, they are dev focussed yes I'll need to make another one (#4) ;) We spent some time on the topic of JVM, I'd like to take from others. yes Just for statistics and to plan the next IRC meeting, can I ask where are you connecting from? tktym, kozo2, ngoto, mn are from Japan. mishima as well? I remember someone from Australia wanted to join this meeting if the time is sane. :) (As I joined conferences in Japan for past two weeks, I could not reply the thread on the mailing list but I just finished to read them yesterday.) We have 12 members (+2 raoul and jan) on this channel. i'm in jakarta indonesia COOL 30 degrees celcius actually :) I am near Amsterdam, Netherlands. hi welcome :) I am late, sorry : ( Today, we talked about BioRuby plugin system (current status and future) and BioRuby with JVM for high performance computing. Probably, Pjotr will put a log on the web? I am in California tktym: yes ok Chris, what time in your time zone? ChristianZmasek: early bird I can be productive today! yannickwurm: what are your bioruby needs? pjotr: work with ants is a bit different because you need to do everything yourself (eg: there is no established GO annotation) are you sequencing yannickwurm: that's the same in livestock animals for me so I've been parsing a lot of stuff (blasts...) and creating small pipelines to stick different tools together, and sometimes run things parallel (using only ruby threads for now... maybe the rqueue in the future) we sequenced and assembled our genome (will be published in PNAS in january :)) we all seem to be doing pipelines so have been dealing more and more with Illumina sequence (fastq), 4 sequence (SFF), their assembly, their mapping (for RNAseq stuff) I am in "Pacific Time Zone" yannickwurm: from Indonesia? I'm usually at the borderline between "shell script" and "pipeline" (most are one-off) I'm a postdoc in switzerland but in september my partner and I moved to indonesia ah, holiday too quick ;) no - well be here for a year or 2 I'm still using the Swiss cluster though (www.vital-it.ch) :) yannickwurm: I am Swiss as well gruetzi :) I have lived in Bangladesh, Italy and UK Hoi! piotr: n Italy also? Where? (I'm italian) Roma My wife was working with UN FAO May happen again I see Where are you in Italy? Milano Near Raoul near Raoul :-) Should meet up at ISMB Vienna I am planning to take the motor bike - Guzzi in july? Is with ECCB2011 if I remember yannickwurm: where in Indonesia it is kinda big jakarta crazy yannickwurm: I have an Indonesian friend there biorubyist? :) I'll hire him :p heh, no programmer we'll take this offline :) It is getting late for some of us (Japan) Yep. 1AM now Maybe we should do this more often? Shall we regularly continue this kind of meeting on IRC? yes, I have got to get going too yannickwurm: quick question do you know about the Hymenoptera Anatomy Ontology? (looking at your solenopsis now) who would not join in next time? mjy: heard about it, but I'm not a taxonomist so haven't pushed much to find out more the thing is yannickwurm: we should chat sometime, it's much than taxonomy the 'core' team does meet up occasionaly online erm much *more* we feel it is useful to broaden the meetings but the timezones are difficult pjotr: with fixed agenda? or as 'bioruby outreach'? I think of it as a meeting of minds so, it is more outreach ok will you send a copy of the log through the mailing list ? already it is useful francesco_strozz: sure already it is useful -- yes one thing we could do is weeklies keep IRC going anyway, but meet up, say one day a week tktym: what is a good time for you, to be logged in? ngoto: same probably not Friday night :) ! whoops maybe Thursday? We are basically online and we can setup a IRC bot to always keep this channel open and to make the log on web (that we had been doing before the Skype era), so all we need to set "core time" - Thursday maybe fine. 11pm JST, 3pm CET, 6am PST every Thursday, does not have to be active conversation, just representation of BioRuby Time to discuss developments, frustrations, and encourage new developments? ChristianZmasek: OK? and "I'm a n00b user why doesnt XXX work?" ? exactly anything :) that's sounds good :-) OK -- Th morning (for me)! that would be good - and important to advertise on the website the key point is, the conversation log will be made public on the web because the first "im going to try something" is when people either stay or leave we need to recruit new contributors, and find ways of leveraging existing code and projects I hope everyone on this channel will help answering those newbie questions :) like rtfm rtfc is more likely with BioRuby ;) Students may have been exposed to RoR so, yes. Hehe - anyway I really gotta run. I would like to just say that I am thankful for all the efforts that have been and continue to be put into bioruby. While many aspects clearly have potential for improvement, I differ with Pjotr's view that bioruby may be a lost cause. My vision is limited, but I have the feeling that bioruby has a solid base and hope that positive energy will lead to more contributions & improvement in the code, as well as m visibility thanks & good night! \quit() eh ok i'm going to control-c piotr2: sorry if this was already discussed but, why don't put (maybe on the website), the "open issues", so libraries that need improvements or new libraries to be written so people can see what is needed and maybe propose to help or write parts of bioruby ? and important to advertise on the website -- agreed francesco: https://github.com/bioruby/bioruby/issues it would be good, yes, but experience tells me it is hard for people to (start) contributing existing issues can be found on the above page, tktym: didn't know???.thanks :-) assigning "new libraries to be written" for others is a hard task There is more to it: I have quite a few branches of bioruby myself, which never make it into mainline we need to mentor them, we need to know their ability and motivation etc. that is one reason to have plugins some such chances were Google SoC so we can have 'experimental' code with bioruby tktym: yes I know, can be a hard task I think development is driven by the 'itch' that needs scratching pj: yes, I believe plugin system will casually invite more people to contribute their code to bioruby it is easy to fork bioruby or plugins but it does not necessarily translate into active and coordinated contribution I still need to write to promised phylogenetics related tutorials... :x ChristianZmasek: I would like that :) but let's continue with these IRC conferences, at least to encourage contribution we are a social species... like ant... :) Is there a "use-case" list for desired bioruby features? I still need to do some cleanup on the Google Summer of Code - "duplications" project and then it needs to be merged... :x mjy: probably, no merging is the hard bit (mad laughter) anyway... mjy: http://bioruby.open-bio.org/wiki/Plugins#Candidate_Whish_List let me ask, who would like to contribute to BioRuby itself here? me me I would at some point- don't know if I have the skills yet :) me (if I meet the required skills, otherwise I will improve skills and then come back to say "me") heh, that is quite a few who have not commited yet I don't think skills should be that critical we all learn as we try plugins can help - even to replace core code maybe each of us should start thinking in terms of creating a plugin yes pjotrp: am I correct in surmising that plugins are just code with a Bioruby namespace? mjy: these items in the plugin wish list are mainly libraries for file formats and external programs, but I would like to propose researchers to provide "workflow" plugins to reproduce the scientific procedure. yes just read the "What is a plugin" it can be really loosely connected, or tight bioruby I wish to encourage you to come up with an idea for a plugin we will support you pjotrp: ok- so I have several gems that I used jeweler for- it would be nice there was a "jeweler" for bioruby plugins if you konw what I mean tktym: I think workflows could be a great plus???.if you think about all the pipelines we need to write to run, for example, sequencing data That will come. francesco: yes, that will be a one of the goals :) tktym: is the idea gridengine or TORQUE? to cluster compute stuff? tktym: I think workflows could be a great plus??? yes! tktym: a workflow system design could also be a paper anyway, let's continue Thursday? Currently, real usages of the bioruby library are buried in the test codes, but we can bring it to front as a plugin. I'm fine with Thursday Besides, I'll setup "nadoka" http://www.atdot.net/nadoka/ as before. bye! And thanks! which is a IRC bot written in Ruby :) by the core developer of Ruby-1.9, Sasada-san !! I'll get the audio on the screen casts done bye, see on the next irc call bye for now bye and thanks! hello bioruby hello again (testing) OK, I think http://bioruby.org/irc/ is ready for you :) Updated http://bioruby.open-bio.org/wiki/BioRuby_IRC_conference Updated http://bioruby.open-bio.org/wiki/Plugins#Develop_a_plugin and added http://bioruby.open-bio.org/wiki/BiogemInstallation test IRC log ended Sun Dec 19 22:50:34 2010 From bonnalraoul at ingm.it Tue Dec 21 10:13:40 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 11:13:40 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> Message-ID: <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> Dear John, do you have an idea, plans on how to arrange all this stuffs ? During this holiday I'll work on the plugin system, code name bio-gem. Googling around I found this framework http://apotomo.de/ which seems to be very promising for building rich web applications. Why using widget ? Suppose to associate a widget to a biodata you can publish is immediately or interact with it easily, just an idea. I think there is another widget for rails but I can't remember the name. On 16/dic/2010, at 01.50, John Woods wrote: > I saw ruby-statsample, and it looks like one of many things we'd (ideally) like to integrate. > > I'm glad you brought up publishing to the web. We spent a not-insignificant amount of time debating graphics platform, and ultimately we decided that HTML5 canvas, and SVG-type graphics, were probably the ideal. For the moment, we're looking at protovis. Interestingly, Claudio Bustos has been working on a Ruby protovis port (he also wrote ruby-statsample, I believe). > > The other big piece is NArray. Masahiro Tanaka is apparently rewriting it with a goal of getting it incorporated into Ruby core. John Prince has been working on documentation for the existing NArray, in part to see if it's got everything SciRuby would need. But ultimately, the NArray rewrite/core integration is the key. It's years from being finished, supposedly -- but perhaps if there are some eager volunteers we could speed that up. =) > > So it seems like most of the pieces are present -- it's just a matter of tying them together. > > And I just said I wasn't going to get into detail -- but there, you just got me going. =) > > John > > On Wed, Dec 15, 2010 at 6:29 PM, Raoul Bonnal wrote: > +1 for me. > Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there is also an optimized version. > > John, in the web era a way to easily publish plots on the web not just as static images would be great. > > Probably I could not attend the first meeting, work trip. Btw if someone is around these day I'll try to be in the channel. > > Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: > > > Hi everyone, > > > > I wanted to put in my two cents about this (not this specific email, but the > > thread). > > > > I come from a CS background, and so when I began working in bioinformatics, > > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby (and > > Rails!). > > > > Working in a bioinformatics lab, my experience has been that most people go > > with Python over Ruby because of SciPy and NumPy (and matplotlib). At least > > for us, it's not because of speed at all. If I need something speedy, I code > > it in C++ and use Rice to build a Ruby adapter. > > > > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have been > > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby (and > > with it, good numerical support). > > > > To that end, we've been working a bit to put together a spec for how a > > SciRuby with plotting and numeric capabilities would work. > > > > I'm not quite ready to say anything specific about it publicly just yet, but > > if you're interested in being involved, please write me back. > > > > Best, > > John O. Woods > > NSF Fellow > > The University of Texas at Austin > > > > > > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields wrote: > > > >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: > >> > >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: > >>>>> Part of how we try to handle big data files in Biopython is using > >>>>> Python iterators, whereby the file is loaded record by record (how > >>>>> depends on the file format - for BLAST we do this query by query), > >>>>> not all into memory in one go. I think BioPerl does something very > >>>>> similar in their parsers, I'm not so familiar with BioJava. > >>> > >>> BioJava uses a visitor pattern. In effect an iterator. > >>> > >>> With all current implementations IO runs, then code, the IO, etc. > >>> While we are IO constrained, we are actually doing worse. > >>> > >>> What I want is an IO thread going at maximum throughput. Every item > >>> should get parcelled out for further parsing and processing, in > >>> parallel to the IO thread. > >>> > >>> We should do better, and make it a generalization. I think we can do > >>> it by using Scala and the standard BioJava iterators. With Scala it > >>> can be turned in a parallelized iterator. That is a fun project. > >>> > >>>> From my point of view Python guys are doing a very good job on all > >> fields. > >>>> Unfortunately I'm in love with ruby :-) > >>> > >>> All you need is love :) > >>> > >>> Pj. > >> > >> At some point the choice of a language will not matter as much, as long as > >> it is implemented in a VM (something Perl 5 cannot claim at the moment, but > >> Perl 6 does with the Parrot VM). > >> > >> chris > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > From biopython at maubp.freeserve.co.uk Tue Dec 21 11:15:34 2010 From: biopython at maubp.freeserve.co.uk (Peter) Date: Tue, 21 Dec 2010 11:15:34 +0000 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101221091837.GA21147@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: On Tue, Dec 21, 2010 at 9:18 AM, Pjotr Prins wrote: > > Below the first IRC conference in a long time. We are keeping the > #bioruby room open on freenode. Every Thursday 2pm (GMT) we'll try to > be online. > > This Thursday we can look at developing new plugins. For anyone > interested in some Ruby programming over the holidays. > > Pj. > > IRC log started Sun Dec 19 13:51:39 2010 > good "morning"... > ... > Additionally, I'd like to discuss about the inclusion of workflows (with data) as plugins. > workflows sounds interesting... > how do you see that? > there are a lot of methods described in journal articles, and probably some of them can be coded in (Bio)Ruby > if the procedure can be distributed as a plugin, > (with example data) > user can try the workflow with ease and with his own data as well > ... Hi all, I just read over the IRC log, and one of the topics that was discussed was workflows (in the form of BioRuby plugins). Looking at workflows a different way, perhaps there are some existing BioRuby scripts which might work really well as in workflow tool in a framework like Taverna or Galaxy? http://www.taverna.org.uk/ http://usegalaxy.org/ We're using Galaxy where I work, and it comes with tool wrappers covering Python, Perl, and R scripts, plus compiled code. I'm sure one could call (Bio)Ruby scripts too. Peter From benjamin.q.almeida at gmail.com Tue Dec 21 11:38:38 2010 From: benjamin.q.almeida at gmail.com (benjamin Almeida) Date: Tue, 21 Dec 2010 12:38:38 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: Hi all, thanks for the log, I couldn't make it, but I'll try thursday as of workflows I kind a like wrapping things in objects and just ask my way through delegate dependencies, but I would join forces on a nice workflow tool in ruby. galaxy is used here (Vienna) too ben From bonnalraoul at ingm.it Tue Dec 21 12:33:18 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 13:33:18 +0100 Subject: [BioRuby] NGS, transcriptome, workflows Message-ID: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> Hi all, I read in the irc's log that a lot of the memebers are or will start working on ngs data. I'll re-start ( I worked on 454) with the Illumina platform, in few week, do you have some consolidate workflow to follow for transcriptome analysis ? papers, blogs, etc.. everything is ok :-) I'm also interested about the workflow discussion but... are the workflow intended to let the not bioinformaticians analyze complex datasets or automate some tedious and repetitive task that we (bioinformaticians) must do every time ? In both cases our life will be probably better :-) -- Ra From missy at be.to Tue Dec 21 14:29:02 2010 From: missy at be.to (MISHIMA, Hiroyuki) Date: Tue, 21 Dec 2010 23:29:02 +0900 Subject: [BioRuby] NGS, transcriptome, workflows In-Reply-To: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> Message-ID: <4D10B9AE.2010206@be.to> Hi all, I would like to say something about workflow automation in Ruby. Recently I am interested in using Parallel Workflow extention for Rake (Pwrake) for NextGen sequencer data processing. As you may know, Rake is Ruby Make, a build tool. Pwrake is developped by Masahiro Tanaka at University of Tsukuba. He is also the author of "NArray", very fast matrix calculation engine for Ruby. Although Pwrake and regular Rake are compatible in syntax, Pwrake automatically detects workflow steps that can be run in parallel. Pwrake's parallelization model is "process based". Because I am just a *user* of bioinformatics packages (like BWA/GATK/DINDEL etc..), it is what I need. Pwrake invokes processes via ssh and supports the Gfarm large-scale distributed filesystem. Of course, it works well on a multi-processor Linux box. Although Pwrake is developed for astronomy science, its goal is also common in bioinformatics. I think that some helper methods may simplify Rakefiles for bioinformatics, and such helper methods are good for a BioRuby plugin. FYI, Pwrake on github: https://github.com/masa16/Pwrake/ Presentation at RubyConfX: http://www.slideshare.net/masa16tanaka/ruby-conftanaka16 Presentation at PRAGMA18: http://goc.pragma-grid.net/pragma-doc/pragma18/Cool_Things/pwrake.pptx Thanks, Hiro. Raoul Bonnal wrote(2010/12/21 21:33): > Hi all, I read in the irc's log that a lot of the memebers are or > will start working on ngs data. I'll re-start ( I worked on 454) with > the Illumina platform, in few week, do you have some consolidate > workflow to follow for transcriptome analysis ? > > papers, blogs, etc.. everything is ok :-) > > I'm also interested about the workflow discussion but... are the > workflow intended to let the not bioinformaticians analyze complex > datasets or automate some tedious and repetitive task that we > (bioinformaticians) must do every time ? In both cases our life will > be probably better :-) > > -- Ra -- MISHIMA, Hiroyuki, DDS, Ph.D. COE Research Fellow Department of Human Genetics Nagasaki University Graduate School of Biomedical Sciences From bonnalraoul at ingm.it Tue Dec 21 15:44:03 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 16:44:03 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: <52CB9CE6-ABE6-4FA9-8983-1F4ECBC6FD49@ingm.it> Hello Guys, something that has not been mentioned during the conference is the possibility to create meta plugins/gems specific for a particular field or release. Pjotr had the original idea because he was worried about compatibility between the plugins and the difference releases of bioruby. He invented the term blessed and we are thinking to mark as blessed the most important plugins, "certified" for working with a specific version of bioruby. Probably use have already discussed this but it's also possible to create meta packages for NGS, for workflows, for imaging, for parallel computation, for JVM (some plugin installable only if a JVM in available) I think that these meta packages, at least at the beginning will be supervised by core developers, just to start and organize them but in the future other people could takeover them. Another important thing is to avoid the proliferation of namespaces, try to look at the core bioruby and find if the your namespace fits in some place, then it would be more easy for the people find the code, classes and modules logically. -- Ra : ml Helius : irc helios : githup ilpuccio : skype On 21/dic/2010, at 12.38, benjamin Almeida wrote: > Hi all, > thanks for the log, I couldn't make it, but I'll try thursday > as of workflows I kind a like wrapping things in objects and just ask my way > through delegate dependencies, but I would join forces on a nice workflow > tool in ruby. > galaxy is used here (Vienna) too > > ben > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From john.woods at marcottelab.org Tue Dec 21 16:46:08 2010 From: john.woods at marcottelab.org (John Woods) Date: Tue, 21 Dec 2010 11:46:08 -0500 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> Message-ID: Apotomo looks like it could be very useful. I think widgets are definitely the way to go. One thing I want to make clear is that although many of us are biologists, SciRuby should be a standalone project. This is not simply intended as a bioinformatics tool. It should be something general purpose, that any scientist/statistician would feel comfortable using. With that said, I think it's appropriate to write plugins that bridge BioRuby and SciRuby. Can you be more specific with your question? I'm planning on taking a look at what we already have later today and updating the spec. (Unfortunately, our lab website is being upgraded, so I can't link you to the current spec right now.) Incidentally, I just set up a Google Group for SciRuby development. It's public. http://groups.google.com/group/sciruby-dev Best, John On Tue, Dec 21, 2010 at 5:13 AM, Raoul Bonnal wrote: > Dear John, > do you have an idea, plans on how to arrange all this stuffs ? > During this holiday I'll work on the plugin system, code name bio-gem. > > Googling around I found this framework http://apotomo.de/ which seems to > be very promising for building rich web applications. > Why using widget ? > Suppose to associate a widget to a biodata you can publish is immediately > or interact with it easily, just an idea. > I think there is another widget for rails but I can't remember the name. > > On 16/dic/2010, at 01.50, John Woods wrote: > > I saw ruby-statsample, and it looks like one of many things we'd (ideally) > like to integrate. > > I'm glad you brought up publishing to the web. We spent a not-insignificant > amount of time debating graphics platform, and ultimately we decided that > HTML5 canvas, and SVG-type graphics, were probably the ideal. For the > moment, we're looking at protovis. Interestingly, Claudio Bustos has been > working on a Ruby protovis port (he also > wrote ruby-statsample, I believe). > > The other big piece is NArray. Masahiro Tanaka is apparently rewriting it > with a goal of getting it incorporated into Ruby core. John Prince has been > working on documentation for the existing NArray, in part to see if it's got > everything SciRuby would need. But ultimately, the NArray rewrite/core > integration is the key. It's years from being finished, supposedly -- but > perhaps if there are some eager volunteers we could speed that up. =) > > So it seems like most of the pieces are present -- it's just a matter of > tying them together. > > And I just said I wasn't going to get into detail -- but there, you just > got me going. =) > > John > > On Wed, Dec 15, 2010 at 6:29 PM, Raoul Bonnal wrote: > >> +1 for me. >> Did you take a look at http://ruby-statsample.rubyforge.org/ ? and there >> is also an optimized version. >> >> John, in the web era a way to easily publish plots on the web not just as >> static images would be great. >> >> Probably I could not attend the first meeting, work trip. Btw if someone >> is around these day I'll try to be in the channel. >> >> Il giorno 16/dic/2010, alle ore 00.02, John Woods ha scritto: >> >> > Hi everyone, >> > >> > I wanted to put in my two cents about this (not this specific email, but >> the >> > thread). >> > >> > I come from a CS background, and so when I began working in >> bioinformatics, >> > I shifted from C++ to Python, Perl, back to Python, and finally to Ruby >> (and >> > Rails!). >> > >> > Working in a bioinformatics lab, my experience has been that most people >> go >> > with Python over Ruby because of SciPy and NumPy (and matplotlib). At >> least >> > for us, it's not because of speed at all. If I need something speedy, I >> code >> > it in C++ and use Rice to build a Ruby adapter. >> > >> > Anyway, a couple of us (me, John Prince, Ara Howard, a few others) have >> been >> > brainstorming a bit. We think the #1 thing Ruby is missing is SciRuby >> (and >> > with it, good numerical support). >> > >> > To that end, we've been working a bit to put together a spec for how a >> > SciRuby with plotting and numeric capabilities would work. >> > >> > I'm not quite ready to say anything specific about it publicly just yet, >> but >> > if you're interested in being involved, please write me back. >> > >> > Best, >> > John O. Woods >> > NSF Fellow >> > The University of Texas at Austin >> > >> > >> > On Mon, Dec 13, 2010 at 3:48 PM, Chris Fields >> wrote: >> > >> >> On Dec 13, 2010, at 12:16 PM, Pjotr Prins wrote: >> >> >> >>> On Mon, Dec 13, 2010 at 06:57:39PM +0100, Raoul Bonnal wrote: >> >>>>> Part of how we try to handle big data files in Biopython is using >> >>>>> Python iterators, whereby the file is loaded record by record (how >> >>>>> depends on the file format - for BLAST we do this query by query), >> >>>>> not all into memory in one go. I think BioPerl does something very >> >>>>> similar in their parsers, I'm not so familiar with BioJava. >> >>> >> >>> BioJava uses a visitor pattern. In effect an iterator. >> >>> >> >>> With all current implementations IO runs, then code, the IO, etc. >> >>> While we are IO constrained, we are actually doing worse. >> >>> >> >>> What I want is an IO thread going at maximum throughput. Every item >> >>> should get parcelled out for further parsing and processing, in >> >>> parallel to the IO thread. >> >>> >> >>> We should do better, and make it a generalization. I think we can do >> >>> it by using Scala and the standard BioJava iterators. With Scala it >> >>> can be turned in a parallelized iterator. That is a fun project. >> >>> >> >>>> From my point of view Python guys are doing a very good job on all >> >> fields. >> >>>> Unfortunately I'm in love with ruby :-) >> >>> >> >>> All you need is love :) >> >>> >> >>> Pj. >> >> >> >> At some point the choice of a language will not matter as much, as long >> as >> >> it is implemented in a VM (something Perl 5 cannot claim at the moment, >> but >> >> Perl 6 does with the Parrot VM). >> >> >> >> chris >> >> _______________________________________________ >> >> BioRuby Project - http://www.bioruby.org/ >> >> BioRuby mailing list >> >> BioRuby at lists.open-bio.org >> >> http://lists.open-bio.org/mailman/listinfo/bioruby >> >> >> > _______________________________________________ >> > BioRuby Project - http://www.bioruby.org/ >> > BioRuby mailing list >> > BioRuby at lists.open-bio.org >> > http://lists.open-bio.org/mailman/listinfo/bioruby >> >> >> > > From bonnalraoul at ingm.it Tue Dec 21 17:02:06 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 18:02:06 +0100 Subject: [BioRuby] SciRuby (The first BioRuby IRC conference on Dec 19th) In-Reply-To: References: <9C5785F3-84B3-40B5-B491-C2D135726160@ingm.it> <4A0E01BA-F3B5-4048-8344-F6FF6260B5AF@ingm.it> Message-ID: <3B4E0689-EAB1-4079-9137-A21D560BF953@ingm.it> Dear John, On 21/dic/2010, at 17.46, John Woods wrote: > Apotomo looks like it could be very useful. I think widgets are definitely the way to go. > > One thing I want to make clear is that although many of us are biologists, SciRuby should be a standalone project. This is not simply intended as a bioinformatics tool. It should be something general purpose, that any scientist/statistician would feel comfortable using. With that said, I think it's appropriate to write plugins that bridge BioRuby and SciRuby. It's fine and logical, i agree with you. > Can you be more specific with your question? I'm planning on taking a look at what we already have later today and updating the spec. (Unfortunately, our lab website is being upgraded, so I can't link you to the current spec right now.) I mean a road map, but I'll wait for the new web site. > > Incidentally, I just set up a Google Group for SciRuby development. It's public. > http://groups.google.com/group/sciruby-dev Good to know. -- Ra From pjotr.public14 at thebird.nl Tue Dec 21 17:04:51 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 21 Dec 2010 18:04:51 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101221091837.GA21147@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> Message-ID: <20101221170451.GA25645@thebird.nl> On Tue, Dec 21, 2010 at 10:18:37AM +0100, Pjotr Prins wrote: > This Thursday we can look at developing new plugins. For anyone > interested in some Ruby programming over the holidays. Some of the list members are interested in contributing new bioruby code. I would like to ask here for candidates for new plugins. My vote would be for example: - bio-blast: BLAST parser plugin based on libxml that does not load everything in RAM - bio-samtools: Samtools plugin (based on biolib?) Who would be interested in writing that? I can help with the implementation and design, if required. Any other ideas here? Pj. From bonnalraoul at ingm.it Tue Dec 21 17:50:24 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 21 Dec 2010 18:50:24 +0100 Subject: [BioRuby] Workflows and Parallelization In-Reply-To: <4D10B9AE.2010206@be.to> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> <4D10B9AE.2010206@be.to> Message-ID: <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> My original request was on analysis approach on NGS and transcriptome. btw Very interesting Hiro, searching on github there is also this project: https://github.com/grosser/parallel Run any code in parallel Processes(> use all CPUs) or Threads(> speedup blocking operations). Best suited for map-reduce or e.g. parallel downloads/uploads. Processes Speedup through multiple CPUs Speedup for blocking operations Protects global data Extra memory used ( very low on REE through copy_on_write_friendly ) Child processes are killed when your main process is killed through Ctrl+c or kill -2 Threads Speedup for blocking operations Global data can be modified No extra memory used Processes/Threads are workers, they grab the next piece of work when they finish On 21/dic/2010, at 15.29, MISHIMA, Hiroyuki wrote: > Hi all, > > I would like to say something about workflow automation in Ruby. > > Recently I am interested in using Parallel Workflow extention for Rake > (Pwrake) for NextGen sequencer data processing. As you may know, Rake is Ruby Make, a build tool. > > Pwrake is developped by Masahiro Tanaka at University of Tsukuba. He is > also the author of "NArray", very fast matrix calculation engine for > Ruby. Although Pwrake and regular Rake are compatible in syntax, Pwrake > automatically detects workflow steps that can be run in parallel. > > Pwrake's parallelization model is "process based". Because I am just a > *user* of bioinformatics packages (like BWA/GATK/DINDEL etc..), it is > what I need. > > Pwrake invokes processes via ssh and supports the Gfarm large-scale > distributed filesystem. Of course, it works well on a multi-processor > Linux box. > > Although Pwrake is developed for astronomy science, its goal is also > common in bioinformatics. > > I think that some helper methods may simplify Rakefiles for bioinformatics, and such helper methods are good for a BioRuby plugin. > > FYI, > Pwrake on github: > https://github.com/masa16/Pwrake/ > > Presentation at RubyConfX: > http://www.slideshare.net/masa16tanaka/ruby-conftanaka16 > > Presentation at PRAGMA18: > http://goc.pragma-grid.net/pragma-doc/pragma18/Cool_Things/pwrake.pptx > > Thanks, > Hiro. > > Raoul Bonnal wrote(2010/12/21 21:33): >> Hi all, I read in the irc's log that a lot of the memebers are or >> will start working on ngs data. I'll re-start ( I worked on 454) with >> the Illumina platform, in few week, do you have some consolidate >> workflow to follow for transcriptome analysis ? >> >> papers, blogs, etc.. everything is ok :-) >> >> I'm also interested about the workflow discussion but... are the >> workflow intended to let the not bioinformaticians analyze complex >> datasets or automate some tedious and repetitive task that we >> (bioinformaticians) must do every time ? In both cases our life will >> be probably better :-) >> >> -- Ra > -- > MISHIMA, Hiroyuki, DDS, Ph.D. > COE Research Fellow > Department of Human Genetics > Nagasaki University Graduate School of Biomedical Sciences > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Wed Dec 22 10:13:44 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 22 Dec 2010 11:13:44 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101221170451.GA25645@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> Message-ID: <20101222101344.GB29739@thebird.nl> Here my first 4 min. screencast on BioRuby on the JVM: http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm (temporary download link) I'll try to put a few more together in the coming month(s): 2) Calling JRuby from Scala (8 min) 3) Calling BioJava from JRuby and Jython (8 min) 4) JRuby parallelization using Actors (BioRuby performance) And, maybe later (partly depending on demand) 5) One-click install of JRuby applications on non-Ruby machines 6) JRuby BioRuby plugins 7) Using C modules from JRuby 8) Using JVM profiling and debugging tools with JRuby 9) Clustering BioRuby with Terracotta Now, if someone could clone me... Pj. From yannick.wurm at unil.ch Wed Dec 22 10:06:20 2010 From: yannick.wurm at unil.ch (Yannick Wurm) Date: Wed, 22 Dec 2010 17:06:20 +0700 Subject: [BioRuby] rake Message-ID: <6CFECABA-E9B4-40CE-9B3B-F1AFBAA38031@unil.ch> Hiro-san's comment about Pwrake for workflows convinced me to finally have a look at Rake. So I was trying to make a Rakefile that gives me a tree from the gblocks run on the codon alignment from the backtranslated protein alignment from the protein sequence from nucleotide squences. But I think I quickly bumped into a killer limitation with rules: that they apparently only do one level of inference And I quote from http://onestepback.org/articles/buildingwithrake/rulelimitations.html > > rule ".c" => [".y"] do |t| yacc(t.source) end > rule ".o" => [".c"] do |t| compile_c(t.source) end > If lex.y exists ? Rake will not Build lex.c from lex.y and lex.o from lex.c. Is this limitation still true? Or is something else wrong with my code? Do you have a workaround? And question two: lets say I want a generic rule that cleans up fasta sequences (so that if I require 'aasdflkjsalkfjasdlkj.fasta', the file aasdflkjsalkfjasdlkj should be run through Emboss' seqret. Is that possible? The following hasn't been working: > rule ".fasta" => "" do |task| > sh "seqret -sequence #{task.prerequisites.join} -outseq #{task.name}" > end Thanks & kind regards, Yannick From missy at be.to Thu Dec 23 01:11:42 2010 From: missy at be.to (MISHIMA, Hiroyuki) Date: Thu, 23 Dec 2010 10:11:42 +0900 Subject: [BioRuby] Workflows and Parallelization In-Reply-To: <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> <4D10B9AE.2010206@be.to> <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> Message-ID: <4D12A1CE.4040702@be.to> Hi Raoul and all, Unfortunately I have not had experience in analyzing transcriptome using NGS. Recently I am working on exome mutation analysis and micro RNA analysis using NGS. I will let you know if I find good material for NGS + transcriptome analysis. Thank you for information about the Parallel library. It can be foundations for the workflow management. And I would like to add Raoul's comment about merits of workflow automation. I think it has three merits. First and second are as Rauol mentioned. It is good for wet-side researchers to simplify the complex tasks, and it is good for dry-side researchers to simplify the repetitive tasks. Third is that it is good for workflow sharing. My ideas are like the followings: Rspec (Ruby's test workframe) is the executable specification. It is better than paper specification. Rakefile (or some scripts) can be executable experiment note. It must be better than paper experiment note. And it is good to share protocols in "experiment notes". By the way, I afraid that talking about Rake/Pwrake and workflow automation in the BioRuby mailing list may be kind of out-of-focus. Is it OK? Thanks, Hiro. Raoul Bonnal wrote (2010/12/22 2:50): > My original request was on analysis approach on NGS and transcriptome. > > btw > > Very interesting Hiro, > > searching on github there is also this project: > https://github.com/grosser/parallel -- MISHIMA, Hiroyuki, DDS, Ph.D. COE Research Fellow Department of Human Genetics Nagasaki University Graduate School of Biomedical Sciences From missy at be.to Thu Dec 23 01:13:16 2010 From: missy at be.to (MISHIMA, Hiroyuki) Date: Thu, 23 Dec 2010 10:13:16 +0900 Subject: [BioRuby] rake In-Reply-To: <6CFECABA-E9B4-40CE-9B3B-F1AFBAA38031@unil.ch> References: <6CFECABA-E9B4-40CE-9B3B-F1AFBAA38031@unil.ch> Message-ID: <4D12A22C.5010602@be.to> Hi Yannick and all, I will try to answer your questions.... Yannick Wurm wrote (2010/12/22 19:06): > Hiro-san's comment about Pwrake for workflows convinced me to > finally have a look at Rake. > > So I was trying to make a Rakefile that gives me a tree from the > gblocks run on the cordon alignment from the backtranslated protein > alignment from the protein sequence from nucleotide squences. But I > think I quickly bumped into a killer limitation with rules: that > they apparently only do one level of inference > > And I quote from > http://onestepback.org/articles/buildingwithrake/rulelimitations.html > >> rule ".c" => [".y"] do |t| yacc(t.source) end >> rule ".o" => [".c"] do |t| compile_c(t.source) end >> If lex.y exists >> ? Rake will not Build lex.c from lex.y and lex.o from lex.c. > > Is this limitation still true? Or is something else wrong with my > code? Do you have a workaround? Rakefile: rule ".c" => [".y"] do |t| puts "run yacc(#{t.source})" touch t.source.ext('c') end rule ".o" => [".c"] do |t| puts "run compile_c(#{t.source})" touch t.source.ext('o') end $ touch lex.y $ rake lex.o run yacc(lex.y) touch lex.c run compile_c(lex.c) touch lex.o Hmm.. it seems to work well.. Do I make misconception about the slide? > And question two: lets say I want a generic rule that cleans up fasta > sequences (so that if I require 'aasdflkjsalkfjasdlkj.fasta', the > file aasdflkjsalkfjasdlkj should be run through Emboss' seqret. Is > that possible? The following hasn't been working: >> rule ".fasta" => "" do |task| >> sh "seqret -sequence >> #{task.prerequisites.join} -outseq #{task.name}" >> end Try dynamic definition using the "file" method. Rekefile (put all the files to be cleaned into the source directory): SOURCES = FileList["source/*"] SOURCES.each do |src| file "#{src}.fasta" => src do |t| sh ["segret", "-sequence #{t.prerequisites.join(" ")}", "-outseq #{t.name}", ].join(" ") end end task :default => SOURCES Sometimes the "rule" methods can be complicated because it basically assumes that all files have one extension (not ".tar.gz" but ".tgz"). Recently I prefer to use "file" instead of "rule". Because Rakefiles are Ruby codes themselves, dynamic definition using the file methods are more flexible. However, I think some hack introducing some new methods (Rake DSL syntax) can make the description simpler... Thanks, Hiro -- MISHIMA, Hiroyuki, DDS, Ph.D. COE Research Fellow Department of Human Genetics Nagasaki University Graduate School of Biomedical Sciences From bonnalraoul at ingm.it Thu Dec 23 09:47:47 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 23 Dec 2010 10:47:47 +0100 Subject: [BioRuby] Workflows and Parallelization In-Reply-To: <4D12A1CE.4040702@be.to> References: <2D62D256-4CBD-4793-97F1-37A908C734F6@ingm.it> <4D10B9AE.2010206@be.to> <5EB99E14-47EF-4267-99F5-216C16A93426@ingm.it> <4D12A1CE.4040702@be.to> Message-ID: <0B6FAF54-370F-4CB3-8752-1E5F80DAC569@ingm.it> Hi Hiro, On 23/dic/2010, at 02.11, MISHIMA, Hiroyuki wrote: > Unfortunately I have not had experience in analyzing transcriptome using > NGS. Recently I am working on exome mutation analysis and micro RNA > analysis using NGS. I will let you know if I find good material for NGS > + transcriptome analysis. Actually the focus of my institute is mainly on mirna, so I'm also interested on techniques for analyzing NGS(illumina) and microRNA. Btw. > By the way, I afraid that talking about Rake/Pwrake and workflow > automation in the BioRuby mailing list may be kind of out-of-focus. Is it OK? Don't be afraid, was just a matter of topic (subject) :-) During the irc meeting they talked about workflows, so this is the right place -- R.J.P.B. From bonnalraoul at ingm.it Thu Dec 23 10:06:39 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Thu, 23 Dec 2010 11:06:39 +0100 Subject: [BioRuby] Wiki Message-ID: Hello, Wiki updated * Plugins: removed old doc & updated http://bioruby.open-bio.org/wiki/Plugins * Added section BioRuby::Ideas::Workflows http://bioruby.open-bio.org/wiki/Main_Page#Ideas http://bioruby.open-bio.org/wiki/Workflows please contribute to this page. Hiro, I reported a piece of your mail, please check if everything is ok. -- R.J.P.B. From ktym at hgc.jp Thu Dec 23 12:09:47 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Thu, 23 Dec 2010 21:09:47 +0900 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> Message-ID: Hi all, The 2nd IRC conf will be started in less than 2 hours from now. Toshiaki On 2010/12/20, at 3:49, Toshiaki Katayama wrote: > Hi all, > > Today, we had >10 participants and discussed about > > * Current and future of BioRuby plugin system > * BioRuby with JVM for high performance computing > * Regularly continue the IRC meeting? > > etc. Details will be reported from Pjotr. > > We agreed to continue this IRC meeting on every Thursday as a "core time", > and we may also keep the #bioruby channel open for everyone > as much as possible, I just setup an IRC bot to log discussions. > > From now on, the log of this channel will be made public at > > http://bioruby.org/irc/ > > so please join us (and please pay attention to privacy > as a netiquette as well ;-). > > Toshiaki > > > On 2010/12/19, at 21:53, Pjotr Prins wrote: > >> Today is the day!! >> >> I use the bitchx client, with logging. >> >> Pj. >> >> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>> Dear BioRuby developers, >>> >>> Here we announce the first BioRuby world wide IRC conference! >>> >>> The topic is: "Current and future BioRuby development" >>> >>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>> 6am PST (California time). >>> >>> You are kindly invited to join! >>> >>> Server: irc.freenode.org >>> Channel: #bioruby >>> >>> Toshiaki Katayama on behalf of the BioRuby staff. >>> _______________________________________________ >>> BioRuby Project - http://www.bioruby.org/ >>> BioRuby mailing list >>> BioRuby at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/bioruby > From pjotr.public14 at thebird.nl Thu Dec 23 20:53:10 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 23 Dec 2010 21:53:10 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101222101344.GB29739@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> Message-ID: <20101223205310.GA9623@thebird.nl> Interesting post on parallel programming, "High Level Concurrency with JRuby and Akka Actors" http://metaphysicaldeveloper.wordpress.com/2010/12/16/high-level-concurrency-with-jruby-and-akka-actors/ Something we'll use soon. Note that Akka is written in Scala, by one of the brightest people: Jonas Bon?r, Pj. From ktym at hgc.jp Fri Dec 24 00:21:29 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Fri, 24 Dec 2010 09:21:29 +0900 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> Message-ID: <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> Hi, Last night, conversation was not so active but we talked about * The actual usage of the BioGem, newly created BioRuby's plugin development system * NGS support including samtools The log is available at http://bioruby.org/irc/?date=2010-12 and I also wrote a step-by-step documentation on the use of BioGem at http://bioruby.open-bio.org/wiki/BiogemDevelopment Enjoy. Toshiaki On 2010/12/23, at 21:09, Toshiaki Katayama wrote: > Hi all, > > The 2nd IRC conf will be started in less than 2 hours from now. > > Toshiaki > > On 2010/12/20, at 3:49, Toshiaki Katayama wrote: > >> Hi all, >> >> Today, we had >10 participants and discussed about >> >> * Current and future of BioRuby plugin system >> * BioRuby with JVM for high performance computing >> * Regularly continue the IRC meeting? >> >> etc. Details will be reported from Pjotr. >> >> We agreed to continue this IRC meeting on every Thursday as a "core time", >> and we may also keep the #bioruby channel open for everyone >> as much as possible, I just setup an IRC bot to log discussions. >> >> From now on, the log of this channel will be made public at >> >> http://bioruby.org/irc/ >> >> so please join us (and please pay attention to privacy >> as a netiquette as well ;-). >> >> Toshiaki >> >> >> On 2010/12/19, at 21:53, Pjotr Prins wrote: >> >>> Today is the day!! >>> >>> I use the bitchx client, with logging. >>> >>> Pj. >>> >>> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>>> Dear BioRuby developers, >>>> >>>> Here we announce the first BioRuby world wide IRC conference! >>>> >>>> The topic is: "Current and future BioRuby development" >>>> >>>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>>> 6am PST (California time). >>>> >>>> You are kindly invited to join! >>>> >>>> Server: irc.freenode.org >>>> Channel: #bioruby >>>> >>>> Toshiaki Katayama on behalf of the BioRuby staff. >>>> _______________________________________________ >>>> BioRuby Project - http://www.bioruby.org/ >>>> BioRuby mailing list >>>> BioRuby at lists.open-bio.org >>>> http://lists.open-bio.org/mailman/listinfo/bioruby >> > From pjotr.public14 at thebird.nl Fri Dec 24 07:34:36 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 24 Dec 2010 08:34:36 +0100 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> Message-ID: <20101224073436.GA12220@thebird.nl> The plugin system appears to be functional - great stuff Raoul. Another thing is that the IRC channel will be attended to the coming weeks by a number of developers. If you feel like little coding and/or experimentation during the holidays, we are there to help. Pj. On Fri, Dec 24, 2010 at 09:21:29AM +0900, Toshiaki Katayama wrote: > Hi, > > Last night, conversation was not so active but we talked about > > * The actual usage of the BioGem, newly created BioRuby's plugin development system > * NGS support including samtools > > The log is available at > > http://bioruby.org/irc/?date=2010-12 > > and I also wrote a step-by-step documentation on the use of BioGem at > > http://bioruby.open-bio.org/wiki/BiogemDevelopment > > Enjoy. > > Toshiaki > > > On 2010/12/23, at 21:09, Toshiaki Katayama wrote: > > > Hi all, > > > > The 2nd IRC conf will be started in less than 2 hours from now. > > > > Toshiaki > > > > On 2010/12/20, at 3:49, Toshiaki Katayama wrote: > > > >> Hi all, > >> > >> Today, we had >10 participants and discussed about > >> > >> * Current and future of BioRuby plugin system > >> * BioRuby with JVM for high performance computing > >> * Regularly continue the IRC meeting? > >> > >> etc. Details will be reported from Pjotr. > >> > >> We agreed to continue this IRC meeting on every Thursday as a "core time", > >> and we may also keep the #bioruby channel open for everyone > >> as much as possible, I just setup an IRC bot to log discussions. > >> > >> From now on, the log of this channel will be made public at > >> > >> http://bioruby.org/irc/ > >> > >> so please join us (and please pay attention to privacy > >> as a netiquette as well ;-). > >> > >> Toshiaki > >> > >> > >> On 2010/12/19, at 21:53, Pjotr Prins wrote: > >> > >>> Today is the day!! > >>> > >>> I use the bitchx client, with logging. > >>> > >>> Pj. > >>> > >>> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: > >>>> Dear BioRuby developers, > >>>> > >>>> Here we announce the first BioRuby world wide IRC conference! > >>>> > >>>> The topic is: "Current and future BioRuby development" > >>>> > >>>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), > >>>> which is 15pm CET (Central European Time) and 9am EST (New York) and > >>>> 6am PST (California time). > >>>> > >>>> You are kindly invited to join! > >>>> > >>>> Server: irc.freenode.org > >>>> Channel: #bioruby > >>>> > >>>> Toshiaki Katayama on behalf of the BioRuby staff. > >>>> _______________________________________________ > >>>> BioRuby Project - http://www.bioruby.org/ > >>>> BioRuby mailing list > >>>> BioRuby at lists.open-bio.org > >>>> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From bonnalraoul at ingm.it Fri Dec 24 11:08:04 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Fri, 24 Dec 2010 12:08:04 +0100 Subject: [BioRuby] Plugins, Biogem and Christmas 2010 Message-ID: <3F1D10FB-B121-4819-A519-87A6637DD8F6@ingm.it> Hello guys, BioRuby plugin system was firstly announced at [BOSC 2010] and will be implemented by the Christmas 2010. Hopefully. :) -- Yes, we made it! Check out the BiogemInstallation and BiogemDevelopment sections. This is our present for you, today is December 24th, 2010. BioGem helps Bioinformaticians start developing plugins/modules for BioRuby creating a scaffold/template and a gem package. We are very happy to release it officially to all our community, as promised. Thanks to everyone for the support and the effort. You can follow the project here https://github.com/helios/bioruby-gem http://bioruby.open-bio.org/wiki/Plugins Start using it ? gem install bio-gem http://bioruby.open-bio.org/wiki/BiogemInstallation http://bioruby.open-bio.org/wiki/BiogemDevelopment Happy Christmas from BioRuby Developer Team -- R.J.P.B. From francesco.strozzi at gmail.com Mon Dec 27 11:25:26 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 12:25:26 +0100 Subject: [BioRuby] Bio Assembly Plugin Message-ID: Hi Bioruby coders, I want to start a Bioruby Assembly Plugin, where I'd like to put parsers for common used formats like ACE, CAF and maybe MAF that is used by the Mira assembler. For what I have seen the Bioruby Core does not provide this type of parsers yet. If someone else is doing something similar please let me know, so we can avoid duplicate the same code... Cheers -- Francesco From francesco.strozzi at gmail.com Mon Dec 27 11:54:01 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 12:54:01 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Thanks! I will have a look. On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO wrote: > Hi, > > http://github.com/chmille4/bioruby > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > On Mon, 27 Dec 2010 12:25:26 +0100 > Francesco Strozzi wrote: > > > Hi Bioruby coders, > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put parsers > for > > common used formats like ACE, CAF and maybe MAF that is used by the Mira > > assembler. For what I have seen the Bioruby Core does not provide this > type > > of parsers yet. If someone else is doing something similar please let me > > know, so we can avoid duplicate the same code... > > > > Cheers > > > > -- > > > > Francesco > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > Naohisa Goto > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > -- Francesco From ngoto at gen-info.osaka-u.ac.jp Mon Dec 27 11:39:06 2010 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Mon, 27 Dec 2010 20:39:06 +0900 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: Message-ID: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Hi, http://github.com/chmille4/bioruby http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html On Mon, 27 Dec 2010 12:25:26 +0100 Francesco Strozzi wrote: > Hi Bioruby coders, > > I want to start a Bioruby Assembly Plugin, where I'd like to put parsers for > common used formats like ACE, CAF and maybe MAF that is used by the Mira > assembler. For what I have seen the Bioruby Core does not provide this type > of parsers yet. If someone else is doing something similar please let me > know, so we can avoid duplicate the same code... > > Cheers > > -- > > Francesco > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org From chmille4 at gmail.com Mon Dec 27 13:11:45 2010 From: chmille4 at gmail.com (Chase Miller) Date: Mon, 27 Dec 2010 08:11:45 -0500 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Hi Francesco Currently, Bio::Assembly only has an ace parser, but I it should be straightforward to add a CAF ( or whatever ) parser. Take a look at the code (https://github.com/chmille4/bioruby) and let me know if you have any questions. I'm in the process of turning this into a plugin. I was thinking of turning it into a standalone ace parser plugin, but maybe it makes more sense to keep it as a bio-assembly plugin, especially if we can add another parser :) . Chase On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < francesco.strozzi at gmail.com> wrote: > Thanks! I will have a look. > > > On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO >wrote: > > > Hi, > > > > http://github.com/chmille4/bioruby > > > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > > > On Mon, 27 Dec 2010 12:25:26 +0100 > > Francesco Strozzi wrote: > > > > > Hi Bioruby coders, > > > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put > parsers > > for > > > common used formats like ACE, CAF and maybe MAF that is used by the > Mira > > > assembler. For what I have seen the Bioruby Core does not provide this > > type > > > of parsers yet. If someone else is doing something similar please let > me > > > know, so we can avoid duplicate the same code... > > > > > > Cheers > > > > > > -- > > > > > > Francesco > > > _______________________________________________ > > > BioRuby Project - http://www.bioruby.org/ > > > BioRuby mailing list > > > BioRuby at lists.open-bio.org > > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > Naohisa Goto > > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > > > > > -- > > Francesco > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From francesco.strozzi at gmail.com Mon Dec 27 13:28:25 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 14:28:25 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Hi Chase, I will have a look for sure. Good to know you will create a plugin for this, so it will be easy to share your code with BioRuby and add new features. A Bio::Assembly module does not exist in BioRuby at the moment, so it will make more sense also for me to keep the plugin as a general Bio::Assembly "container" where you or me or everybody else can add new parsers for different file formats. Cheers P.S. It seems the plugin system is getting ground very fast! :-) On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: > Hi Francesco > > Currently, Bio::Assembly only has an ace parser, but I it should be > straightforward to add a CAF ( or whatever ) parser. Take a look at the > code (https://github.com/chmille4/bioruby) and let me know if you have any > questions. > > I'm in the process of turning this into a plugin. I was thinking of turning > it into a standalone ace parser plugin, but maybe it makes more sense to > keep it as a bio-assembly plugin, especially if we can add another parser :) > . > > Chase > > > > On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > francesco.strozzi at gmail.com> wrote: > >> Thanks! I will have a look. >> >> >> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO > >wrote: >> >> > Hi, >> > >> > http://github.com/chmille4/bioruby >> > >> > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html >> > >> > On Mon, 27 Dec 2010 12:25:26 +0100 >> > Francesco Strozzi wrote: >> > >> > > Hi Bioruby coders, >> > > >> > > I want to start a Bioruby Assembly Plugin, where I'd like to put >> parsers >> > for >> > > common used formats like ACE, CAF and maybe MAF that is used by the >> Mira >> > > assembler. For what I have seen the Bioruby Core does not provide this >> > type >> > > of parsers yet. If someone else is doing something similar please let >> me >> > > know, so we can avoid duplicate the same code... >> > > >> > > Cheers >> > > >> > > -- >> > > >> > > Francesco >> > > _______________________________________________ >> > > BioRuby Project - http://www.bioruby.org/ >> > > BioRuby mailing list >> > > BioRuby at lists.open-bio.org >> > > http://lists.open-bio.org/mailman/listinfo/bioruby >> > >> > Naohisa Goto >> > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org >> > >> >> >> >> -- >> >> Francesco >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > > -- Francesco From chmille4 at gmail.com Mon Dec 27 17:00:44 2010 From: chmille4 at gmail.com (Chase Miller) Date: Mon, 27 Dec 2010 12:00:44 -0500 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Francesco, Looking over the code, it looks like I never got around to generalizing bio::assembly to work with multiple parsers. This is long overdue and I'm going to work on it today. Chase On Dec 27, 2010, at 8:28 AM, Francesco Strozzi wrote: > Hi Chase, > I will have a look for sure. Good to know you will create a plugin for this, so it will be easy to share your code with BioRuby and add new features. A Bio::Assembly module does not exist in BioRuby at the moment, so it will make more sense also for me to keep the plugin as a general Bio::Assembly "container" where you or me or everybody else can add new parsers for different file formats. > > Cheers > > P.S. It seems the plugin system is getting ground very fast! :-) > > On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: > Hi Francesco > > Currently, Bio::Assembly only has an ace parser, but I it should be straightforward to add a CAF ( or whatever ) parser. Take a look at the code (https://github.com/chmille4/bioruby) and let me know if you have any questions. > > I'm in the process of turning this into a plugin. I was thinking of turning it into a standalone ace parser plugin, but maybe it makes more sense to keep it as a bio-assembly plugin, especially if we can add another parser :) . > > Chase > > > > On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi wrote: > Thanks! I will have a look. > > > On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO wrote: > > > Hi, > > > > http://github.com/chmille4/bioruby > > > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > > > On Mon, 27 Dec 2010 12:25:26 +0100 > > Francesco Strozzi wrote: > > > > > Hi Bioruby coders, > > > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put parsers > > for > > > common used formats like ACE, CAF and maybe MAF that is used by the Mira > > > assembler. For what I have seen the Bioruby Core does not provide this > > type > > > of parsers yet. If someone else is doing something similar please let me > > > know, so we can avoid duplicate the same code... > > > > > > Cheers > > > > > > -- > > > > > > Francesco > > > _______________________________________________ > > > BioRuby Project - http://www.bioruby.org/ > > > BioRuby mailing list > > > BioRuby at lists.open-bio.org > > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > Naohisa Goto > > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > > > > > -- > > Francesco > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > -- > > Francesco > From pjotr.public14 at thebird.nl Mon Dec 27 17:13:57 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 27 Dec 2010 18:13:57 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101222101344.GB29739@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> Message-ID: <20101227171357.GA32230@thebird.nl> Here the second 8 minute screencast, where I call into BioRuby from Scala. A similar procedure would work for Java and Clojure, calling into (Bio)Ruby libraries. It just works (tm). http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm (temporary download link) Note how Scala resembles Ruby. Just to be clear: for me Scala does not replace Ruby. I use both languages. Scala is for performance, mostly. Essentially a replacement for C, in my book. It is a much nicer language than C, C++, or Java (shudder). On Wed, Dec 22, 2010 at 11:13:44AM +0100, Pjotr Prins wrote: > Here my first 4 min. screencast on BioRuby on the JVM: > > http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm > > (temporary download link) > > I'll try to put a few more together in the coming month(s): > > 3) Calling BioJava from JRuby and Jython (8 min) > 4) JRuby parallelization using Actors (BioRuby performance) > > And, maybe later (partly depending on demand) > > 5) One-click install of JRuby applications on non-Ruby machines > 6) JRuby BioRuby plugins > 7) Using C modules from JRuby > 8) Using JVM profiling and debugging tools with JRuby > 9) Clustering BioRuby with Terracotta > > Now, if someone could clone me... > > Pj. From georgkam at gmail.com Mon Dec 27 18:11:46 2010 From: georgkam at gmail.com (George Githinji) Date: Mon, 27 Dec 2010 21:11:46 +0300 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: Hi, This definitely is intresting as sometime back I was looking for an CAF parser. I have some local dirty CAF parser. It would be interesting to develop a community CAF parser. Thanks George On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller wrote: > Francesco, > > Looking over the code, it looks like I never got around to generalizing > bio::assembly to work with multiple parsers. This is long overdue and I'm > going to work on it today. > > Chase > > On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < > francesco.strozzi at gmail.com> wrote: > > > Hi Chase, > > I will have a look for sure. Good to know you will create a plugin for > this, so it will be easy to share your code with BioRuby and add new > features. A Bio::Assembly module does not exist in BioRuby at the moment, so > it will make more sense also for me to keep the plugin as a general > Bio::Assembly "container" where you or me or everybody else can add new > parsers for different file formats. > > > > Cheers > > > > P.S. It seems the plugin system is getting ground very fast! :-) > > > > On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: > > Hi Francesco > > > > Currently, Bio::Assembly only has an ace parser, but I it should be > straightforward to add a CAF ( or whatever ) parser. Take a look at the > code (https://github.com/chmille4/bioruby) and let me know if you have any > questions. > > > > I'm in the process of turning this into a plugin. I was thinking of > turning it into a standalone ace parser plugin, but maybe it makes more > sense to keep it as a bio-assembly plugin, especially if we can add another > parser :) . > > > > Chase > > > > > > > > On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > francesco.strozzi at gmail.com> wrote: > > Thanks! I will have a look. > > > > > > On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < > ngoto at gen-info.osaka-u.ac.jp>wrote: > > > > > Hi, > > > > > > http://github.com/chmille4/bioruby > > > > > > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > > > > > > On Mon, 27 Dec 2010 12:25:26 +0100 > > > Francesco Strozzi wrote: > > > > > > > Hi Bioruby coders, > > > > > > > > I want to start a Bioruby Assembly Plugin, where I'd like to put > parsers > > > for > > > > common used formats like ACE, CAF and maybe MAF that is used by the > Mira > > > > assembler. For what I have seen the Bioruby Core does not provide > this > > > type > > > > of parsers yet. If someone else is doing something similar please let > me > > > > know, so we can avoid duplicate the same code... > > > > > > > > Cheers > > > > > > > > -- > > > > > > > > Francesco > > > > _______________________________________________ > > > > BioRuby Project - http://www.bioruby.org/ > > > > BioRuby mailing list > > > > BioRuby at lists.open-bio.org > > > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > > Naohisa Goto > > > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > > > > > > > > > > -- > > > > Francesco > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > > > > > > -- > > > > Francesco > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- --------------- Sincerely George KEMRI/Wellcome-Trust Research Program Skype: george_g2 Blog: http://biorelated.wordpress.com/ From francesco.strozzi at gmail.com Mon Dec 27 19:55:27 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Mon, 27 Dec 2010 20:55:27 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: <20101227171357.GA32230@thebird.nl> References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> <20101227171357.GA32230@thebird.nl> Message-ID: Hi Pjotr, I have some problem with the screencast audio. It seems broken and I don't hear anything but bad noises. Is anyone having the same issue? I'm on a Mac using VLC and the previous screencast was fine. Thanks On Mon, Dec 27, 2010 at 18:13, Pjotr Prins wrote: > Here the second 8 minute screencast, where I call into BioRuby from > Scala. A similar procedure would work for Java and Clojure, calling > into (Bio)Ruby libraries. It just works (tm). > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > (temporary download link) > > Note how Scala resembles Ruby. > > Just to be clear: for me Scala does not replace Ruby. I use both > languages. Scala is for performance, mostly. Essentially a > replacement for C, in my book. It is a much nicer language than C, > C++, or Java (shudder). > > On Wed, Dec 22, 2010 at 11:13:44AM +0100, Pjotr Prins wrote: > > Here my first 4 min. screencast on BioRuby on the JVM: > > > > http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm > > > > (temporary download link) > > > > I'll try to put a few more together in the coming month(s): > > > > 3) Calling BioJava from JRuby and Jython (8 min) > > 4) JRuby parallelization using Actors (BioRuby performance) > > > > And, maybe later (partly depending on demand) > > > > 5) One-click install of JRuby applications on non-Ruby machines > > 6) JRuby BioRuby plugins > > 7) Using C modules from JRuby > > 8) Using JVM profiling and debugging tools with JRuby > > 9) Clustering BioRuby with Terracotta > > > > Now, if someone could clone me... > > > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- Francesco From pjotr.public14 at thebird.nl Mon Dec 27 20:13:56 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Mon, 27 Dec 2010 21:13:56 +0100 Subject: [BioRuby] IRC conference archives In-Reply-To: References: <20101218121024.GA456@thebird.nl> <20101221091837.GA21147@thebird.nl> <20101221170451.GA25645@thebird.nl> <20101222101344.GB29739@thebird.nl> <20101227171357.GA32230@thebird.nl> Message-ID: <20101227201356.GA658@thebird.nl> Works for me. md5: > md5sum JRuby_from_Scala_or_Java.ogm a28646e48527c268a26469c4ef271238 JRuby_from_Scala_or_Java.ogm if it does not work can make one again... Pj. On Mon, Dec 27, 2010 at 08:55:27PM +0100, Francesco Strozzi wrote: > Hi Pjotr, > > I have some problem with the screencast audio. It seems broken and I don't > hear anything but bad noises. Is anyone having the same issue? I'm on a Mac > using VLC and the previous screencast was fine. > > Thanks > > On Mon, Dec 27, 2010 at 18:13, Pjotr Prins wrote: > > > Here the second 8 minute screencast, where I call into BioRuby from > > Scala. A similar procedure would work for Java and Clojure, calling > > into (Bio)Ruby libraries. It just works (tm). > > > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > > > (temporary download link) > > > > Note how Scala resembles Ruby. > > > > Just to be clear: for me Scala does not replace Ruby. I use both > > languages. Scala is for performance, mostly. Essentially a > > replacement for C, in my book. It is a much nicer language than C, > > C++, or Java (shudder). > > > > On Wed, Dec 22, 2010 at 11:13:44AM +0100, Pjotr Prins wrote: > > > Here my first 4 min. screencast on BioRuby on the JVM: > > > > > > http://bio4.dnsalias.net/download2/JRuby_and_BioRuby.ogm > > > > > > (temporary download link) > > > > > > I'll try to put a few more together in the coming month(s): > > > > > > 3) Calling BioJava from JRuby and Jython (8 min) > > > 4) JRuby parallelization using Actors (BioRuby performance) > > > > > > And, maybe later (partly depending on demand) > > > > > > 5) One-click install of JRuby applications on non-Ruby machines > > > 6) JRuby BioRuby plugins > > > 7) Using C modules from JRuby > > > 8) Using JVM profiling and debugging tools with JRuby > > > 9) Clustering BioRuby with Terracotta > > > > > > Now, if someone could clone me... > > > > > > Pj. > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > > > > > -- > > Francesco From pjotr.public14 at thebird.nl Tue Dec 28 09:29:23 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 28 Dec 2010 10:29:23 +0100 Subject: [BioRuby] BioRuby Screencast Message-ID: <20101228092923.GA2089@thebird.nl> Ok, try again. Now it is ogg proper - the file is smaller too. 204d140c57f00ff16a19343742712f4c JRuby_from_Scala_or_Java.ogm On Mon, Dec 27, 2010 at 06:13:57PM +0100, Pjotr Prins wrote: > Here the second 8 minute screencast, where I call into BioRuby from > Scala. A similar procedure would work for Java and Clojure, calling > into (Bio)Ruby libraries. It just works (tm). > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > (temporary download link) > > Note how Scala resembles Ruby. > > Just to be clear: for me Scala does not replace Ruby. I use both > languages. Scala is for performance, mostly. Essentially a > replacement for C, in my book. It is a much nicer language than C, > C++, or Java (shudder). From francesco.strozzi at gmail.com Tue Dec 28 09:35:57 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Tue, 28 Dec 2010 10:35:57 +0100 Subject: [BioRuby] BioRuby Screencast In-Reply-To: <20101228092923.GA2089@thebird.nl> References: <20101228092923.GA2089@thebird.nl> Message-ID: Works perfectly now! Thanks Cheers On Tue, Dec 28, 2010 at 10:29, Pjotr Prins wrote: > Ok, try again. Now it is ogg proper - the file is smaller too. > > 204d140c57f00ff16a19343742712f4c JRuby_from_Scala_or_Java.ogm > > On Mon, Dec 27, 2010 at 06:13:57PM +0100, Pjotr Prins wrote: > > Here the second 8 minute screencast, where I call into BioRuby from > > Scala. A similar procedure would work for Java and Clojure, calling > > into (Bio)Ruby libraries. It just works (tm). > > > > http://bio4.dnsalias.net/download2/JRuby_from_Scala_or_Java.ogm > > > > (temporary download link) > > > > Note how Scala resembles Ruby. > > > > Just to be clear: for me Scala does not replace Ruby. I use both > > languages. Scala is for performance, mostly. Essentially a > > replacement for C, in my book. It is a much nicer language than C, > > C++, or Java (shudder). > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- Francesco From bonnalraoul at ingm.it Tue Dec 28 10:18:25 2010 From: bonnalraoul at ingm.it (Raoul Bonnal) Date: Tue, 28 Dec 2010 11:18:25 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> On 27/dic/2010, at 19.11, George Githinji wrote: > Hi, > This definitely is intresting as sometime back I was looking for an CAF > parser. > I have some local dirty CAF parser. It would be interesting to develop a > community CAF parser. I think so. I agree with Francesco, I suggest to start designing a general Assembler and use adapters like Ace, CAF an others in the way we did with Bio::Sequence. Bio::Assembler Bio::Assembler::Ace Bio::Assembler::CAF and we need to keep in mind that we can not store everything in memory :-) > Thanks > George > > > On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller wrote: > >> Francesco, >> >> Looking over the code, it looks like I never got around to generalizing >> bio::assembly to work with multiple parsers. This is long overdue and I'm >> going to work on it today. >> >> Chase >> >> On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < >> francesco.strozzi at gmail.com> wrote: >> >>> Hi Chase, >>> I will have a look for sure. Good to know you will create a plugin for >> this, so it will be easy to share your code with BioRuby and add new >> features. A Bio::Assembly module does not exist in BioRuby at the moment, so >> it will make more sense also for me to keep the plugin as a general >> Bio::Assembly "container" where you or me or everybody else can add new >> parsers for different file formats. >>> >>> Cheers >>> >>> P.S. It seems the plugin system is getting ground very fast! :-) >>> >>> On Mon, Dec 27, 2010 at 14:11, Chase Miller wrote: >>> Hi Francesco >>> >>> Currently, Bio::Assembly only has an ace parser, but I it should be >> straightforward to add a CAF ( or whatever ) parser. Take a look at the >> code (https://github.com/chmille4/bioruby) and let me know if you have any >> questions. >>> >>> I'm in the process of turning this into a plugin. I was thinking of >> turning it into a standalone ace parser plugin, but maybe it makes more >> sense to keep it as a bio-assembly plugin, especially if we can add another >> parser :) . >>> >>> Chase >>> >>> >>> >>> On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < >> francesco.strozzi at gmail.com> wrote: >>> Thanks! I will have a look. >>> >>> >>> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < >> ngoto at gen-info.osaka-u.ac.jp>wrote: >>> >>>> Hi, >>>> >>>> http://github.com/chmille4/bioruby >>>> >>>> http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html >>>> >>>> On Mon, 27 Dec 2010 12:25:26 +0100 >>>> Francesco Strozzi wrote: >>>> >>>>> Hi Bioruby coders, >>>>> >>>>> I want to start a Bioruby Assembly Plugin, where I'd like to put >> parsers >>>> for >>>>> common used formats like ACE, CAF and maybe MAF that is used by the >> Mira >>>>> assembler. For what I have seen the Bioruby Core does not provide >> this >>>> type >>>>> of parsers yet. If someone else is doing something similar please let >> me >>>>> know, so we can avoid duplicate the same code... >>>>> >>>>> Cheers >>>>> >>>>> -- >>>>> >>>>> Francesco >>>>> _______________________________________________ >>>>> BioRuby Project - http://www.bioruby.org/ >>>>> BioRuby mailing list >>>>> BioRuby at lists.open-bio.org >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby >>>> >>>> Naohisa Goto >>>> ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org >>>> >>> >>> >>> >>> -- >>> >>> Francesco >>> _______________________________________________ >>> BioRuby Project - http://www.bioruby.org/ >>> BioRuby mailing list >>> BioRuby at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/bioruby >>> >>> >>> >>> >>> -- >>> >>> Francesco >>> >> >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > > > > -- > --------------- > Sincerely > George > KEMRI/Wellcome-Trust Research Program > Skype: george_g2 > Blog: http://biorelated.wordpress.com/ > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby -- R.J.P.B. From francesco.strozzi at gmail.com Tue Dec 28 13:56:41 2010 From: francesco.strozzi at gmail.com (Francesco Strozzi) Date: Tue, 28 Dec 2010 14:56:41 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> Message-ID: Iteretors will be the way, as discussed in another thread :-) Ciao Francesco On Tue, Dec 28, 2010 at 11:18, Raoul Bonnal wrote: > > On 27/dic/2010, at 19.11, George Githinji wrote: > > > Hi, > > This definitely is intresting as sometime back I was looking for an CAF > > parser. > > I have some local dirty CAF parser. It would be interesting to develop a > > community CAF parser. > > I think so. > I agree with Francesco, I suggest to start designing a general Assembler > and use adapters like Ace, CAF an others in the way we did with > Bio::Sequence. > > Bio::Assembler > Bio::Assembler::Ace > Bio::Assembler::CAF > > and we need to keep in mind that we can not store everything in memory :-) > > > > > Thanks > > George > > > > > > On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller > wrote: > > > >> Francesco, > >> > >> Looking over the code, it looks like I never got around to generalizing > >> bio::assembly to work with multiple parsers. This is long overdue and > I'm > >> going to work on it today. > >> > >> Chase > >> > >> On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >> > >>> Hi Chase, > >>> I will have a look for sure. Good to know you will create a plugin for > >> this, so it will be easy to share your code with BioRuby and add new > >> features. A Bio::Assembly module does not exist in BioRuby at the > moment, so > >> it will make more sense also for me to keep the plugin as a general > >> Bio::Assembly "container" where you or me or everybody else can add new > >> parsers for different file formats. > >>> > >>> Cheers > >>> > >>> P.S. It seems the plugin system is getting ground very fast! :-) > >>> > >>> On Mon, Dec 27, 2010 at 14:11, Chase Miller > wrote: > >>> Hi Francesco > >>> > >>> Currently, Bio::Assembly only has an ace parser, but I it should be > >> straightforward to add a CAF ( or whatever ) parser. Take a look at the > >> code (https://github.com/chmille4/bioruby) and let me know if you have > any > >> questions. > >>> > >>> I'm in the process of turning this into a plugin. I was thinking of > >> turning it into a standalone ace parser plugin, but maybe it makes more > >> sense to keep it as a bio-assembly plugin, especially if we can add > another > >> parser :) . > >>> > >>> Chase > >>> > >>> > >>> > >>> On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >>> Thanks! I will have a look. > >>> > >>> > >>> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < > >> ngoto at gen-info.osaka-u.ac.jp>wrote: > >>> > >>>> Hi, > >>>> > >>>> http://github.com/chmille4/bioruby > >>>> > >>>> > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > >>>> > >>>> On Mon, 27 Dec 2010 12:25:26 +0100 > >>>> Francesco Strozzi wrote: > >>>> > >>>>> Hi Bioruby coders, > >>>>> > >>>>> I want to start a Bioruby Assembly Plugin, where I'd like to put > >> parsers > >>>> for > >>>>> common used formats like ACE, CAF and maybe MAF that is used by the > >> Mira > >>>>> assembler. For what I have seen the Bioruby Core does not provide > >> this > >>>> type > >>>>> of parsers yet. If someone else is doing something similar please let > >> me > >>>>> know, so we can avoid duplicate the same code... > >>>>> > >>>>> Cheers > >>>>> > >>>>> -- > >>>>> > >>>>> Francesco > >>>>> _______________________________________________ > >>>>> BioRuby Project - http://www.bioruby.org/ > >>>>> BioRuby mailing list > >>>>> BioRuby at lists.open-bio.org > >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>>> > >>>> Naohisa Goto > >>>> ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > >>>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> _______________________________________________ > >>> BioRuby Project - http://www.bioruby.org/ > >>> BioRuby mailing list > >>> BioRuby at lists.open-bio.org > >>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> > >> > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > > > > > > > -- > > --------------- > > Sincerely > > George > > KEMRI/Wellcome-Trust Research Program > > Skype: george_g2 > > Blog: http://biorelated.wordpress.com/ > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > -- > R.J.P.B. > > > > > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > -- Francesco From chmille4 at gmail.com Tue Dec 28 14:16:08 2010 From: chmille4 at gmail.com (Chase Miller) Date: Tue, 28 Dec 2010 09:16:08 -0500 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> <01A465D5-C0B7-43E2-9625-1DB029C0735B@ingm.it> Message-ID: > I agree with Francesco, I suggest to start designing a general Assembler > and use adapters like Ace, CAF an others in the way we did with > Bio::Sequence. > > Bio::Assembler > Bio::Assembler::Ace > Bio::Assembler::CAF > My code currently does this, but not in the way Bio::Sequence does. I based my solution off this stackoverflow example ( http://stackoverflow.com/questions/746207/ruby-design-pattern-how-to-make-an-extensible-factory-class ). I'll take look at Bio::Sequence and see if it's easy to refactor. Here's what I have so far (suggestions very welcome!): code: https://github.com/chmille4/bioruby-assembly usage example: https://gist.github.com/615296 > and we need to keep in mind that we can not store everything in memory :-) > By this do you mean streaming, where the entire file is eventually read into memory if all the data is accessed? Or are you talking about something else, perhaps creating an index file similar to a project like https://github.com/ekg/fastahack. Cheers, Chase > > > > > Thanks > > George > > > > > > On Mon, Dec 27, 2010 at 8:00 PM, Chase Miller > wrote: > > > >> Francesco, > >> > >> Looking over the code, it looks like I never got around to generalizing > >> bio::assembly to work with multiple parsers. This is long overdue and > I'm > >> going to work on it today. > >> > >> Chase > >> > >> On Dec 27, 2010, at 8:28 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >> > >>> Hi Chase, > >>> I will have a look for sure. Good to know you will create a plugin for > >> this, so it will be easy to share your code with BioRuby and add new > >> features. A Bio::Assembly module does not exist in BioRuby at the > moment, so > >> it will make more sense also for me to keep the plugin as a general > >> Bio::Assembly "container" where you or me or everybody else can add new > >> parsers for different file formats. > >>> > >>> Cheers > >>> > >>> P.S. It seems the plugin system is getting ground very fast! :-) > >>> > >>> On Mon, Dec 27, 2010 at 14:11, Chase Miller > wrote: > >>> Hi Francesco > >>> > >>> Currently, Bio::Assembly only has an ace parser, but I it should be > >> straightforward to add a CAF ( or whatever ) parser. Take a look at the > >> code (https://github.com/chmille4/bioruby) and let me know if you have > any > >> questions. > >>> > >>> I'm in the process of turning this into a plugin. I was thinking of > >> turning it into a standalone ace parser plugin, but maybe it makes more > >> sense to keep it as a bio-assembly plugin, especially if we can add > another > >> parser :) . > >>> > >>> Chase > >>> > >>> > >>> > >>> On Mon, Dec 27, 2010 at 6:54 AM, Francesco Strozzi < > >> francesco.strozzi at gmail.com> wrote: > >>> Thanks! I will have a look. > >>> > >>> > >>> On Mon, Dec 27, 2010 at 12:39, Naohisa GOTO < > >> ngoto at gen-info.osaka-u.ac.jp>wrote: > >>> > >>>> Hi, > >>>> > >>>> http://github.com/chmille4/bioruby > >>>> > >>>> > http://lists.open-bio.org/pipermail/bioruby/2010-September/001480.html > >>>> > >>>> On Mon, 27 Dec 2010 12:25:26 +0100 > >>>> Francesco Strozzi wrote: > >>>> > >>>>> Hi Bioruby coders, > >>>>> > >>>>> I want to start a Bioruby Assembly Plugin, where I'd like to put > >> parsers > >>>> for > >>>>> common used formats like ACE, CAF and maybe MAF that is used by the > >> Mira > >>>>> assembler. For what I have seen the Bioruby Core does not provide > >> this > >>>> type > >>>>> of parsers yet. If someone else is doing something similar please let > >> me > >>>>> know, so we can avoid duplicate the same code... > >>>>> > >>>>> Cheers > >>>>> > >>>>> -- > >>>>> > >>>>> Francesco > >>>>> _______________________________________________ > >>>>> BioRuby Project - http://www.bioruby.org/ > >>>>> BioRuby mailing list > >>>>> BioRuby at lists.open-bio.org > >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>>> > >>>> Naohisa Goto > >>>> ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > >>>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> _______________________________________________ > >>> BioRuby Project - http://www.bioruby.org/ > >>> BioRuby mailing list > >>> BioRuby at lists.open-bio.org > >>> http://lists.open-bio.org/mailman/listinfo/bioruby > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> Francesco > >>> > >> > >> _______________________________________________ > >> BioRuby Project - http://www.bioruby.org/ > >> BioRuby mailing list > >> BioRuby at lists.open-bio.org > >> http://lists.open-bio.org/mailman/listinfo/bioruby > >> > > > > > > > > -- > > --------------- > > Sincerely > > George > > KEMRI/Wellcome-Trust Research Program > > Skype: george_g2 > > Blog: http://biorelated.wordpress.com/ > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > -- > R.J.P.B. > > > > > From pjotr.public14 at thebird.nl Tue Dec 28 17:33:45 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 28 Dec 2010 18:33:45 +0100 Subject: [BioRuby] Bio Assembly Plugin In-Reply-To: References: <20101227113906.4AC111CBC3E0@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <20101228173345.GA4446@thebird.nl> On Mon, Dec 27, 2010 at 02:28:25PM +0100, Francesco Strozzi wrote: > P.S. It seems the plugin system is getting ground very fast! :-) I just cut my first bioruby-gff3 plugin gem using the tutorials :). Raoul has done a great job. The next step is to make 'blessed' gems available from bioruby itself. I should be online for the Thursday IRC session. Pj. From ktym at hgc.jp Fri Dec 31 04:44:31 2010 From: ktym at hgc.jp (Toshiaki Katayama) Date: Fri, 31 Dec 2010 13:44:31 +0900 Subject: [BioRuby] The third BioRuby IRC conference on Dec 30th In-Reply-To: <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> Message-ID: <6487068C-FB06-4F17-8308-A70A3324F819@hgc.jp> Hi, Last night, we discussed about the "blessed" plugins, which will be a set of recommended plugins. You can read the IRC log at the following page as always: http://bioruby.org/irc/?date=2010-12 We agreed to maintain two "meta" packages of plugins * "bio-full" --> not reviewd but includes all plugins * "bio-core" --> reviewed/selected/recommended plugins and Raoul provided the first example of the "bio-full" meta package at https://github.com/helios/bioruby-full Although, the name of these meta packages and the system to maintain those in detail are still under discussion. Additionally, I created a toy plugin "bio-hello" for fun (and hopefully a working example of the BioRuby plugin). % gem install bio-hello % biohello.rb -x -c "A happy new year!" gcataacatgcacctccttattaaaatgaatggtaatatgaagcaaga gc c--g a---t t----a a----t a---t c--g at at c--g g---c t----a g----c g---c a--t gc cg t--a t---a a----t t----a t---a a--t at ta t--a a---t c----g t----a t---a a--t cg gc t--a a---t a----t t----a a---t t--a gc ta t--a c---g g----c t----a t---a c--g ta % ruby -rubygems -r bio -e 'p Bio::Sequence::NA.new("gcataacatgcacctccttattaaaatgaatggtaatatgaagcaaga").translate' ==> "A*HAPPY*NEW*YEAR" Please see https://github.com/ktym/bioruby-hello/blob/master/lib/bio-hello.rb and try % biohello.rb --help for more details. I hope you enjoy! Toshiaki On 2010/12/24, at 9:21, Toshiaki Katayama wrote: > Hi, > > Last night, conversation was not so active but we talked about > > * The actual usage of the BioGem, newly created BioRuby's plugin development system > * NGS support including samtools > > The log is available at > > http://bioruby.org/irc/?date=2010-12 > > and I also wrote a step-by-step documentation on the use of BioGem at > > http://bioruby.open-bio.org/wiki/BiogemDevelopment > > Enjoy. > > Toshiaki > > > On 2010/12/23, at 21:09, Toshiaki Katayama wrote: > >> Hi all, >> >> The 2nd IRC conf will be started in less than 2 hours from now. >> >> Toshiaki >> >> On 2010/12/20, at 3:49, Toshiaki Katayama wrote: >> >>> Hi all, >>> >>> Today, we had >10 participants and discussed about >>> >>> * Current and future of BioRuby plugin system >>> * BioRuby with JVM for high performance computing >>> * Regularly continue the IRC meeting? >>> >>> etc. Details will be reported from Pjotr. >>> >>> We agreed to continue this IRC meeting on every Thursday as a "core time", >>> and we may also keep the #bioruby channel open for everyone >>> as much as possible, I just setup an IRC bot to log discussions. >>> >>> From now on, the log of this channel will be made public at >>> >>> http://bioruby.org/irc/ >>> >>> so please join us (and please pay attention to privacy >>> as a netiquette as well ;-). >>> >>> Toshiaki >>> >>> >>> On 2010/12/19, at 21:53, Pjotr Prins wrote: >>> >>>> Today is the day!! >>>> >>>> I use the bitchx client, with logging. >>>> >>>> Pj. >>>> >>>> On Tue, Dec 07, 2010 at 08:51:32PM +0900, Toshiaki Katayama wrote: >>>>> Dear BioRuby developers, >>>>> >>>>> Here we announce the first BioRuby world wide IRC conference! >>>>> >>>>> The topic is: "Current and future BioRuby development" >>>>> >>>>> The meeting is Sunday, Dec. 19, 2010, at 23pm JST (Japanese Standard Time), >>>>> which is 15pm CET (Central European Time) and 9am EST (New York) and >>>>> 6am PST (California time). >>>>> >>>>> You are kindly invited to join! >>>>> >>>>> Server: irc.freenode.org >>>>> Channel: #bioruby >>>>> >>>>> Toshiaki Katayama on behalf of the BioRuby staff. >>>>> _______________________________________________ >>>>> BioRuby Project - http://www.bioruby.org/ >>>>> BioRuby mailing list >>>>> BioRuby at lists.open-bio.org >>>>> http://lists.open-bio.org/mailman/listinfo/bioruby >>> >> > From pjotr.public14 at thebird.nl Fri Dec 31 14:40:04 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 31 Dec 2010 15:40:04 +0100 Subject: [BioRuby] The second BioRuby IRC conference on Dec 23rd In-Reply-To: <20101224073436.GA12220@thebird.nl> References: <20101219125313.GA6033@thebird.nl> <909DEA17-D3A0-49AC-A3ED-C080B4DF396F@hgc.jp> <3B8431EF-C2D5-4482-8A93-23397D049B62@hgc.jp> <20101224073436.GA12220@thebird.nl> Message-ID: <20101231144004.GB16875@thebird.nl> Today I fixed a few bug in the GFF3 parser. The problem was the use of phase in CDSs. By default it is not used, and now everything appears to pass. It is available as a bio-gff3 bioruby plugin on rubygems. https://rubygems.org/gems/bio-gff3 or my github repo https://github.com/pjotrp/bioruby-gff3-plugin which has the README. For example, fetch mRNA and CDS information from GFF3 files and output to FASTA: gff3-fetch mrna test/data/gff/test.gff3 gff3-fetch cds test/data/gff/test.gff3 On Fri, Dec 24, 2010 at 08:34:36AM +0100, Pjotr Prins wrote: > The plugin system appears to be functional - great stuff Raoul.