From p.j.a.cock at googlemail.com Thu Sep 15 15:17:57 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Thu, 15 Sep 2011 20:17:57 +0100 Subject: [BioRuby] Running BioRuby tests offline Message-ID: Hi all, I recently tried running the BioRuby test suite while offline (on a train), $ ruby runner.rb Loaded suite . Started ...............F.........EE.....EEEEEEEEFFEFEEEFEE...EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFFFFFE..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FF............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Finished in 100.74387 seconds. 1) Failure: test_read_uri(Bio::FuncTestCommandNet) [./functional/bio/test_command.rb:337]: Exception raised: Class: Message: <"getaddrinfo: nodename nor servname provided, or not known"> ---Backtrace--- ... 73) Failure: test_libxml(Bio::TestPhyloXML_Check_LibXML) [./unit/bio/db/test_phyloxml.rb:29]: Error: libxml-ruby library is not present. Please install libxml-ruby library. It is needed for Bio::PhyloXML module. Unit test for PhyloXML will not be performed. is not true. 3640 tests, 20824 assertions, 26 failures, 47 errors By eye, all 73 failures or errors were down to missing the XML library, or lacking network access. Is there a way to run the test suite skipping the tests requiring network access? If BioRuby starts running nightly tests (I was talking to Raoul about this at the BioHackathon last month), an offline switch would seem a good idea. We do this in Biopython to avoid hitting any online servers too hard, and because network issues would otherwise get flagged as potential regressions (false positives). Regards, Peter From mphoeppner at gmail.com Mon Sep 19 04:57:14 2011 From: mphoeppner at gmail.com (Marc P. Hoeppner) Date: Mon, 19 Sep 2011 10:57:14 +0200 Subject: [BioRuby] Bio::Fetch issue - biofetch.rb not found Message-ID: <4E7703EA.100@gmail.com> Hi, when trying to fetch entries from remote databases via Bio::Fetch, I receive the following error: /usr/lib/ruby/1.8/open-uri.rb:277:in `open_http': 404 Not Found (OpenURI::HTTPError) It appears the default entry for Fetch.new searches for http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at that location? Am I doing anything wrong? Using gem version 1.4.2 and ruby 1.8.7 on Ubuntu 11.04 /Marc From pjotr.public14 at thebird.nl Tue Sep 20 12:56:01 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 20 Sep 2011 18:56:01 +0200 Subject: [BioRuby] biogems.info updated Message-ID: <20110920165601.GA8933@thebird.nl> I have updated Biogems to show the number of outstanding issues, for each plugin, as listed on github. Also a number of additional plugins are listed now as core and stable. If you feel your plugin should be listed as such, please bring that up here. See http://biogems.info/ or http://gems.bioruby.org/ Enjoy. Pj. From p.j.a.cock at googlemail.com Wed Sep 21 11:44:03 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 16:44:03 +0100 Subject: [BioRuby] Broken link on wiki installation page Message-ID: Hello all, I was looking at the Installation page on the Wiki (to see what the minimum version of Ruby required was): http://bioruby.open-bio.org/wiki/Installation The first line links to a README file, but this URL doesn't work: http://bioruby.open-bio.org/rdoc/files/README_rdoc.html Should this now be the following?: http://bioruby.open-bio.org/rdoc/README_rdoc.html Regards, Peter From p.j.a.cock at googlemail.com Wed Sep 21 12:04:35 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 17:04:35 +0100 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> Message-ID: On Wed, Sep 21, 2011 at 4:56 PM, Naohisa GOTO wrote: > Hi Peter, > > Thank you. I've changed the link. > The URL is changed due to the RDoc renderer's version up. > > Naohisa Goto Great. I see you require currently require Ruby 1.8.6 or later (except Ruby 1.9.0) I'm thinking about potential build slaves for nightly testing (something Raoul and I chatted about the BioHackathon). That means my Mac OS X 10.6.8 Snow Leopard with Ruby 1.8.7 should be fine. On the other hand, one of our CentOS servers which use as a Biopython buildslave only has Ruby 1.8.5, so that won't work without me installing a more recent version of ruby under the user account. Thanks, Peter From pjotr.public14 at thebird.nl Wed Sep 21 12:16:05 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 21 Sep 2011 18:16:05 +0200 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <20110921161605.GA18787@thebird.nl> Hi Peter, You should look into rvm. I tried avoiding using rvm - but now I do. I gave in. rvm should work in user land. Are you also planning to add testing for other biogems? At least bio-core and bio-core-ext (which include the 'core' gems). It would be really nice to have broader coverage. Pj. On Wed, Sep 21, 2011 at 05:04:35PM +0100, Peter Cock wrote: > On Wed, Sep 21, 2011 at 4:56 PM, Naohisa GOTO > wrote: > > Hi Peter, > > > > Thank you. I've changed the link. > > The URL is changed due to the RDoc renderer's version up. > > > > Naohisa Goto > > Great. > > I see you require currently require Ruby 1.8.6 or later (except Ruby 1.9.0) > > I'm thinking about potential build slaves for nightly testing > (something Raoul and I chatted about the BioHackathon). > > That means my Mac OS X 10.6.8 Snow Leopard with Ruby 1.8.7 should be fine. > > On the other hand, one of our CentOS servers which use as a Biopython > buildslave only has Ruby 1.8.5, so that won't work without me > installing a more recent version of ruby under the user account. > > Thanks, > > Peter > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From ngoto at gen-info.osaka-u.ac.jp Wed Sep 21 11:56:33 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 00:56:33 +0900 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: References: Message-ID: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> Hi Peter, Thank you. I've changed the link. The URL is changed due to the RDoc renderer's version up. Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Wed, 21 Sep 2011 16:44:03 +0100 Peter Cock wrote: > Hello all, > > I was looking at the Installation page on the Wiki (to see > what the minimum version of Ruby required was): > http://bioruby.open-bio.org/wiki/Installation > > The first line links to a README file, but this URL doesn't work: > http://bioruby.open-bio.org/rdoc/files/README_rdoc.html > > Should this now be the following?: > http://bioruby.open-bio.org/rdoc/README_rdoc.html > > Regards, > > Peter > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From cjfields at illinois.edu Wed Sep 21 12:20:50 2011 From: cjfields at illinois.edu (Fields, Christopher J) Date: Wed, 21 Sep 2011 16:20:50 +0000 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: <20110921161605.GA18787@thebird.nl> References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> <20110921161605.GA18787@thebird.nl> Message-ID: <0D660C10-EAD3-43EE-8BC2-4767DBAB78AD@illinois.edu> Ah, see the buildbot system is up? Methinks we need to set up bioperl... chris On Sep 21, 2011, at 11:16 AM, Pjotr Prins wrote: > Hi Peter, > > You should look into rvm. I tried avoiding using rvm - but now I do. > I gave in. rvm should work in user land. > > Are you also planning to add testing for other biogems? At least > bio-core and bio-core-ext (which include the 'core' gems). It would be > really nice to have broader coverage. > > Pj. > > On Wed, Sep 21, 2011 at 05:04:35PM +0100, Peter Cock wrote: >> On Wed, Sep 21, 2011 at 4:56 PM, Naohisa GOTO >> wrote: >>> Hi Peter, >>> >>> Thank you. I've changed the link. >>> The URL is changed due to the RDoc renderer's version up. >>> >>> Naohisa Goto >> >> Great. >> >> I see you require currently require Ruby 1.8.6 or later (except Ruby 1.9.0) >> >> I'm thinking about potential build slaves for nightly testing >> (something Raoul and I chatted about the BioHackathon). >> >> That means my Mac OS X 10.6.8 Snow Leopard with Ruby 1.8.7 should be fine. >> >> On the other hand, one of our CentOS servers which use as a Biopython >> buildslave only has Ruby 1.8.5, so that won't work without me >> installing a more recent version of ruby under the user account. >> >> Thanks, >> >> Peter >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From p.j.a.cock at googlemail.com Wed Sep 21 16:02:01 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 21:02:01 +0100 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: <20110921161605.GA18787@thebird.nl> References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> <20110921161605.GA18787@thebird.nl> Message-ID: On Wednesday, September 21, 2011, Pjotr Prins wrote: > Hi Peter, > > You should look into rvm. I tried avoiding using rvm - but now I do. > I gave in. rvm should work in user land. > > Are you also planning to add testing for other biogems? At least > bio-core and bio-core-ext (which include the 'core' gems). It would be > really nice to have broader coverage. > > Pj. Initially I was thinking just the core BioRuby tests, but if it is simple to install/update all the gems and run their tests too that's possible too. You may need to do something to ensure missing dependencies (eg 3rd party binaries) and handled gracefully - skipped not fails. And I would recommend disabling the online tests (see other thread). Peter From pjotr.public14 at thebird.nl Wed Sep 21 16:07:11 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 21 Sep 2011 22:07:11 +0200 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> <20110921161605.GA18787@thebird.nl> Message-ID: <20110921200711.GA20122@thebird.nl> Normally it is just running 'rake test' in the base dir. I think gems can test also at install time. Anyway, I am sure some of us are interested in helping out, including myself. On Wed, Sep 21, 2011 at 09:02:01PM +0100, Peter Cock wrote: > On Wednesday, September 21, 2011, Pjotr Prins > <[1]pjotr.public14 at thebird.nl> wrote: > > Hi Peter, > > > > You should look into rvm. I tried avoiding using rvm - but now I do. > > I gave in. rvm should work in user land. > > > > Are you also planning to add testing for other biogems? At least > > bio-core and bio-core-ext (which include the 'core' gems). It would be > > really nice to have broader coverage. > > > > Pj. > > Initially I was thinking just the core BioRuby tests, but > if it is simple to install/update all the gems and run their > tests too that's possible too. > > You may need to do something to ensure missing > dependencies (eg 3rd party binaries) and handled > gracefully - skipped not fails. > > And I would recommend disabling the online tests > (see other thread). > > Peter > > References > > Visible links > 1. mailto:pjotr.public14 at thebird.nl From p.j.a.cock at googlemail.com Wed Sep 21 17:17:46 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 22:17:46 +0100 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page Message-ID: Hello all, Note I've retitled the thread since the topic has change. Apologies for the long email - I was hoping to talk though most of this first with Raoul, Goto-san and Toshiaki-san while in Japan, but there was plenty of other things going on. Sorry. On Wed, Sep 21, 2011 at 9:07 PM, Pjotr Prins wrote: > Normally it is just running 'rake test' in the base dir. I think > gems can test also at install time. Actually I may have been over enthusiastic regarding testing the BioRuby gems: http://lists.open-bio.org/pipermail/bioruby/2011-September/001896.html One of the key points of the buildbot software (which we are using for the Biopython nightly testing) is it is linked to a repository, and you can see the test results versus the revisions. This includes git support which is good. You can browse this here: http://testing.open-bio.org/biopython/ Personally I like the transposed grid on my widescreen monitor at work, where each row is a revision: http://testing.open-bio.org/biopython/tgrid When the row turns red, you have a major cross platform regression. When a subgroup turns red, you have a platform specific failure (e.g. just Windows, or a specific version of Python). We also have failure emails sent out, and this can be followed by an RSS feed too. There is also a time based view, the "waterfall": http://testing.open-bio.org/biopython/waterfall If all the tests are triggered automatically on a nightly basis, the chronological order of the tests will match the changes made to the repository. However, via the web interface (with a password) you can request a buildslave run the tests using a particular branch/revision. This is handy if a buildslave was offline for some reason and missed a nightly test. Now, in the typical setup you have one software suite with one repository and one test suite. That's what we have for Biopython, and what I was picturing initially for BioRuby. In the case of the BioRuby gems, we could set this up against the core BioRuby repository, and have the core BioRuby tests run (against the known revision from that repository) plus the BioGem tests run with the latest Gem code. The downside is that wouldn't track changes to the Gems themselves. Is that likely to be a problem? I've not looked into the details of possible multi-repository setups - perhaps Tiago knows more (he setup the buildbot for Biopython, CC'd). Thinking out loud, one solution would be a meta repository - I think you can have a git repository with sub projects... maybe that could be done for the BioRuby core plus (important) Gems? Alternatively, perhaps a different test server infrastructure might support a composite project with multiple sub projects, each with their own repository? You don't have to use buildbot - Chris Fields was talking about something called smoke test for BioPerl (CC'd). > Anyway, I am sure some of us are > interested in helping out, including myself. > Excellent. You (BioRuby) will need several volunteer machines as buildslaves which are are either on all the time, or at least online regularly. I'm willing to offer one. You (BioRuby) will also need to decide which platforms you'll be testing - by which I mean what combinations of operating system and version of Ruby. You will need at least one buildslave for each platform. I plan to offer at least one of my machines (it will make testing an initial setup much easier). For Biopython we try to cover Mac, Windows, Linux, both 32 bit and 64 bit, and a range of versions of the standard Python implementation (in C) plus also Jython (Python on the Java Virtual Machine), and in the future perhaps PyPy and IronPython. For each platform we will need the (short) steps to build/compile BioRuby from a fresh git checkout, and run the tests. e.g. For Biopython under Windows on Python 2.5, C:\Python25\python.exe setup.py build C:\Python25\python.exe setup.py test While on Linux on Python 2.7 it would be: python2.7 setup.py build python2.7 setup,py test We try and have each buildslave cover a couple of different versions of Python - thus we use explicit and slightly different commands in each case (rather than just using the default installation of Python). We can go though a lot of this kind of detail off the mailing list if you prefer. I hope I haven't bored most of the mailing list subscribers. There isn't a separate list for BioRuby development is there? Regards, Peter From pjotr.public14 at thebird.nl Thu Sep 22 01:21:48 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 07:21:48 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: Message-ID: <20110922052148.GA22535@thebird.nl> Hi Peter, It is for reason of limited installs we have meta-packages. Meta-packages provide a sense of stability, and is what most users will install. I suggest testing for: - bio - bio-core, which contains core pure ruby biogems - bio-core-ext, which contains non-pure ruby biogems (extensions) anything worth testing will be in there. We can help create the test scripts for those packages. Also, it looks like the testing environment should be hosted in a ready made VM image. I can create a CloudBioLinux flavour for that. Once we know what to install. I will also want tests for biolib_hpc, when that gets production ready (btw). It is a long standing wish on my end to get this sorted. Pj. On Wed, Sep 21, 2011 at 10:17:46PM +0100, Peter Cock wrote: > Hello all, > > Note I've retitled the thread since the topic has change. > > Apologies for the long email - I was hoping to talk > though most of this first with Raoul, Goto-san and > Toshiaki-san while in Japan, but there was plenty > of other things going on. Sorry. > > On Wed, Sep 21, 2011 at 9:07 PM, Pjotr Prins wrote: > > Normally it is just running 'rake test' in the base dir. I think > > gems can test also at install time. > > Actually I may have been over enthusiastic regarding > testing the BioRuby gems: > http://lists.open-bio.org/pipermail/bioruby/2011-September/001896.html > > One of the key points of the buildbot software (which > we are using for the Biopython nightly testing) is it is > linked to a repository, and you can see the test results > versus the revisions. This includes git support which > is good. You can browse this here: > http://testing.open-bio.org/biopython/ > > Personally I like the transposed grid on my widescreen > monitor at work, where each row is a revision: > http://testing.open-bio.org/biopython/tgrid > > When the row turns red, you have a major cross > platform regression. When a subgroup turns red, > you have a platform specific failure (e.g. just > Windows, or a specific version of Python). > > We also have failure emails sent out, and this can > be followed by an RSS feed too. > > There is also a time based view, the "waterfall": > http://testing.open-bio.org/biopython/waterfall > > If all the tests are triggered automatically on a > nightly basis, the chronological order of the tests > will match the changes made to the repository. > However, via the web interface (with a password) > you can request a buildslave run the tests using > a particular branch/revision. This is handy if a > buildslave was offline for some reason and > missed a nightly test. > > Now, in the typical setup you have one software > suite with one repository and one test suite. > That's what we have for Biopython, and what > I was picturing initially for BioRuby. > > In the case of the BioRuby gems, we could set > this up against the core BioRuby repository, and > have the core BioRuby tests run (against the > known revision from that repository) plus the > BioGem tests run with the latest Gem code. > The downside is that wouldn't track changes > to the Gems themselves. Is that likely to be > a problem? > > I've not looked into the details of possible > multi-repository setups - perhaps Tiago > knows more (he setup the buildbot for > Biopython, CC'd). > > Thinking out loud, one solution would be a > meta repository - I think you can have a git > repository with sub projects... maybe that > could be done for the BioRuby core plus > (important) Gems? > > Alternatively, perhaps a different test server > infrastructure might support a composite > project with multiple sub projects, each > with their own repository? You don't have > to use buildbot - Chris Fields was talking > about something called smoke test for > BioPerl (CC'd). > > > Anyway, I am sure some of us are > > interested in helping out, including myself. > > > > Excellent. You (BioRuby) will need several volunteer > machines as buildslaves which are are either on all > the time, or at least online regularly. I'm willing to offer > one. > > You (BioRuby) will also need to decide which platforms > you'll be testing - by which I mean what combinations > of operating system and version of Ruby. You will need > at least one buildslave for each platform. I plan to > offer at least one of my machines (it will make testing > an initial setup much easier). > > For Biopython we try to cover Mac, Windows, Linux, > both 32 bit and 64 bit, and a range of versions of the > standard Python implementation (in C) plus also > Jython (Python on the Java Virtual Machine), and > in the future perhaps PyPy and IronPython. > > For each platform we will need the (short) steps > to build/compile BioRuby from a fresh git checkout, > and run the tests. > > e.g. For Biopython under Windows on Python 2.5, > > C:\Python25\python.exe setup.py build > C:\Python25\python.exe setup.py test > > While on Linux on Python 2.7 it would be: > > python2.7 setup.py build > python2.7 setup,py test > > We try and have each buildslave cover a couple of different > versions of Python - thus we use explicit and slightly different > commands in each case (rather than just using the default > installation of Python). > > We can go though a lot of this kind of detail off the > mailing list if you prefer. > > I hope I haven't bored most of the mailing list > subscribers. There isn't a separate list for BioRuby > development is there? > > Regards, > > Peter > From ngoto at gen-info.osaka-u.ac.jp Thu Sep 22 02:12:07 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 15:12:07 +0900 Subject: [BioRuby] Bio::Fetch issue - biofetch.rb not found In-Reply-To: <4E7703EA.100@gmail.com> References: <4E7703EA.100@gmail.com> Message-ID: <20110922061207.F0BD01CBC414@idnmail.gen-info.osaka-u.ac.jp> Hello Marc, Sorry for late response. > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > that location? The service is down for years, due to the server trouble, and it is currently difficult to restore the service because of the internal structure of bioruby.org. Please use other server, e.g. http://www.ebi.ac.uk/Tools/dbfetch/dbfetch bf =Bio::Fetch.new('http://www.ebi.ac.uk/Tools/dbfetch/dbfetch') str = bf.fetch('embl', 'AE014292') print str Alternatively, some other classes provide similar function. str = Bio::TogoWS::REST.entry('genbank', 'AF237819') print str Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Mon, 19 Sep 2011 10:57:14 +0200 "Marc P. Hoeppner" wrote: > Hi, > > when trying to fetch entries from remote databases via Bio::Fetch, I > receive the following error: > > /usr/lib/ruby/1.8/open-uri.rb:277:in `open_http': 404 Not Found > (OpenURI::HTTPError) > > It appears the default entry for Fetch.new searches for > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > that location? > > Am I doing anything wrong? > > Using gem version 1.4.2 and ruby 1.8.7 on Ubuntu 11.04 > > /Marc > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From p.j.a.cock at googlemail.com Thu Sep 22 08:03:11 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Thu, 22 Sep 2011 13:03:11 +0100 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: <20110922052148.GA22535@thebird.nl> References: <20110922052148.GA22535@thebird.nl> Message-ID: On Thu, Sep 22, 2011 at 6:21 AM, Pjotr Prins wrote: > Hi Peter, > > It is for reason of limited installs we have meta-packages. > Meta-packages provide a sense of stability, and is what most users > will install. > > I suggest testing for: > > ?- bio > ?- bio-core, which contains core pure ruby biogems > ?- bio-core-ext, which contains non-pure ruby biogems > ? ?(extensions) > > anything worth testing will be in there. We can help create the test > scripts for those packages. What I am trying to understand is do all those bits live in many different repositories, or is there one super repository? And if there isn't a single repository, can we create a meta-repository (e.g. using git submodules). > Also, it looks like the testing environment should be hosted in a > ready made VM image. I can create a CloudBioLinux flavour for that. > Once we know what to install. I will also want tests for biolib_hpc, > when that gets production ready (btw). It is a long standing wish on > my end to get this sorted. Perhaps I have misunderstood you, but if all the build slaves run the tests via the same VM image, there is no diversity. For Biopython we deliberately want inhomogeneous setups for our buildslaves to try and cover the variety of potential setups our end users will have. Peter From ngoto at gen-info.osaka-u.ac.jp Thu Sep 22 08:18:13 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 21:18:13 +0900 Subject: [BioRuby] Running BioRuby tests offline In-Reply-To: References: Message-ID: <20110922121813.C6EB01CBC414@idnmail.gen-info.osaka-u.ac.jp> Hi Peter, The proposal is marked as a Feature Request to BioRuby. https://redmine.open-bio.org/issues/3293 Thanks, Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Thu, 15 Sep 2011 20:17:57 +0100 Peter Cock wrote: > Hi all, > > I recently tried running the BioRuby test suite while offline (on a train), > > > $ ruby runner.rb > Loaded suite . > Started > ...............F.........EE.....EEEEEEEEFFEFEEEFEE...EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFFFFFE..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... . > ! > ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FF............................................................................................................................. .! > > ...................................................................... > ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ . > ! > ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ > Finished in 100.74387 seconds. > > 1) Failure: > test_read_uri(Bio::FuncTestCommandNet) [./functional/bio/test_command.rb:337]: > Exception raised: > Class: > Message: <"getaddrinfo: nodename nor servname provided, or not known"> > ---Backtrace--- > ... > 73) Failure: > test_libxml(Bio::TestPhyloXML_Check_LibXML) [./unit/bio/db/test_phyloxml.rb:29]: > Error: libxml-ruby library is not present. Please install libxml-ruby > library. It is needed for Bio::PhyloXML module. Unit test for PhyloXML > will not be performed. > is not true. > > 3640 tests, 20824 assertions, 26 failures, 47 errors > > By eye, all 73 failures or errors were down to missing the XML > library, or lacking network access. > > Is there a way to run the test suite skipping the tests requiring > network access? > > If BioRuby starts running nightly tests (I was talking to Raoul about > this at the BioHackathon last month), an offline switch would seem a > good idea. We do this in Biopython to avoid hitting any online servers > too hard, and because network issues would otherwise get flagged as > potential regressions (false positives). > > Regards, > > Peter > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Thu Sep 22 08:24:51 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 14:24:51 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: <20110922052148.GA22535@thebird.nl> Message-ID: <20110922122451.GA24244@thebird.nl> Hi, On Thu, Sep 22, 2011 at 01:03:11PM +0100, Peter Cock wrote: > What I am trying to understand is do all those bits live in many > different repositories, Yes. biogems are individual efforts - we actively encourage that. You have to understand that the combined biogems are arguably more important now than the core bioruby project. Though both have their roles. > or is there one super repository? And if there isn't a single > repository, can we create a meta-repository (e.g. using git > submodules). git submodules are not that great. I used them for biolib. They are nice and not nice ;) Like I wrote, we track repositories through biogems, bio-core and bio-core-ext. I think you aim to check out the master repositories (development branch) for testing, right? So we can write a script to do just that. I would prefer to track the meta-gems, as well as the regular biogems. Test results can be listed on biogems.info. Would it work to clone all repositories in one directory, and run both combined and individual tests? We can do the scripting of that. > > Also, it looks like the testing environment should be hosted in a > > ready made VM image. I can create a CloudBioLinux flavour for that. > > Once we know what to install. I will also want tests for biolib_hpc, > > when that gets production ready (btw). It is a long standing wish on > > my end to get this sorted. > > Perhaps I have misunderstood you, but if all the build slaves > run the tests via the same VM image, there is no diversity. > For Biopython we deliberately want inhomogeneous setups > for our buildslaves to try and cover the variety of potential > setups our end users will have. Different VMs can be created. For each target one. I am primarily interested in a Debian VM - as that covers >80% of users in bioinformatics. I am not saying it replaces your installations - but it would certainly add to it. Pj. From ngoto at gen-info.osaka-u.ac.jp Thu Sep 22 08:26:28 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 21:26:28 +0900 Subject: [BioRuby] Bio::Fetch issue - biofetch.rb not found In-Reply-To: <20110922061207.F0BD01CBC414@idnmail.gen-info.osaka-u.ac.jp> References: <4E7703EA.100@gmail.com> <20110922061207.F0BD01CBC414@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <20110922122629.537531CBC511@idnmail.gen-info.osaka-u.ac.jp> Hi, The issue is marked as a bug in the OBF Redmine. https://redmine.open-bio.org/issues/3294 Thanks, Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Thu, 22 Sep 2011 15:12:07 +0900 Naohisa GOTO wrote: > Hello Marc, > > Sorry for late response. > > > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > > that location? > > The service is down for years, due to the server trouble, > and it is currently difficult to restore the service > because of the internal structure of bioruby.org. > > Please use other server, e.g. > http://www.ebi.ac.uk/Tools/dbfetch/dbfetch > > bf =Bio::Fetch.new('http://www.ebi.ac.uk/Tools/dbfetch/dbfetch') > str = bf.fetch('embl', 'AE014292') > print str > > Alternatively, some other classes provide similar function. > > str = Bio::TogoWS::REST.entry('genbank', 'AF237819') > print str > > Naohisa Goto > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > On Mon, 19 Sep 2011 10:57:14 +0200 > "Marc P. Hoeppner" wrote: > > > Hi, > > > > when trying to fetch entries from remote databases via Bio::Fetch, I > > receive the following error: > > > > /usr/lib/ruby/1.8/open-uri.rb:277:in `open_http': 404 Not Found > > (OpenURI::HTTPError) > > > > It appears the default entry for Fetch.new searches for > > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > > that location? > > > > Am I doing anything wrong? > > > > Using gem version 1.4.2 and ruby 1.8.7 on Ubuntu 11.04 > > > > /Marc > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From p.j.a.cock at googlemail.com Thu Sep 22 09:31:19 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Thu, 22 Sep 2011 14:31:19 +0100 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: <20110922122451.GA24244@thebird.nl> References: <20110922052148.GA22535@thebird.nl> <20110922122451.GA24244@thebird.nl> Message-ID: On Thu, Sep 22, 2011 at 1:24 PM, Pjotr Prins wrote: > Hi, > > On Thu, Sep 22, 2011 at 01:03:11PM +0100, Peter Cock wrote: >> What I am trying to understand is do all those bits live in many >> different repositories, > > Yes. biogems are individual efforts - we actively encourage that. You > have to understand that the combined biogems are arguably more > important now than the core bioruby project. Though both have their > roles. OK. >> or is there one super repository? And if there isn't a single >> repository, can we create a meta-repository (e.g. using git >> submodules). > > git submodules are not that great. I used them for biolib. They are > nice and not nice ;) > > Like I wrote, we track repositories through biogems, bio-core and > bio-core-ext. I think you aim to check out the master repositories > (development branch) for testing, right? ?So we can write a script to > do just that. I would prefer to track the meta-gems, as well as > the regular biogems. Test results can be listed on biogems.info. > > Would it work to clone all repositories in one directory, and run > both combined and individual tests? We can do the scripting of that. It sounds like the buildbot software we're using for the Biopython testing will probably not cover this use case, from their FAQ: http://trac.buildbot.net/wiki/FAQ "However, a project that packages several independently-developed utilities into a single binary for users to download is more difficult, as Buildbot does not have a means to represent the multiple revisions from multiple repositories that must be used to make up the single binary. For simple cases (e.g,. always building the latest version), this can be made to work, but more complex cases are still a work in progress." It sounds like the best we can hope for using BuildBot is to track a core BioRuby repository, but for the BioRuby Gems we'd just be able to test the latest version. That would still be better than nothing, but you wouldn't be able to easily work out which revision of a Gem caused a problem for example. >> > Also, it looks like the testing environment should be hosted in a >> > ready made VM image. I can create a CloudBioLinux flavour for that. >> > Once we know what to install. I will also want tests for biolib_hpc, >> > when that gets production ready (btw). It is a long standing wish on >> > my end to get this sorted. >> >> Perhaps I have misunderstood you, but if all the build slaves >> run the tests via the same VM image, there is no diversity. >> For Biopython we deliberately want inhomogeneous setups >> for our buildslaves to try and cover the variety of potential >> setups our end users will have. > > Different VMs can be created. For each target one. I am primarily > interested in a Debian VM - as that covers >80% of users in > bioinformatics. I am not saying it replaces your installations - but > it would certainly add to it. OK, having a Debian VM as one of several BioRuby buildslaves covering a range of setups makes sense. Peter From pjotr.public14 at thebird.nl Thu Sep 22 09:48:03 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 15:48:03 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: <20110922052148.GA22535@thebird.nl> <20110922122451.GA24244@thebird.nl> Message-ID: <20110922134803.GB27229@thebird.nl> On Thu, Sep 22, 2011 at 02:31:19PM +0100, Peter Cock wrote: > "However, a project that packages several independently-developed > utilities into a single binary for users to download is more difficult, > as Buildbot does not have a means to represent the multiple > revisions from multiple repositories that must be used to make > up the single binary. For simple cases (e.g,. always building the > latest version), this can be made to work, but more complex cases > are still a work in progress." > > It sounds like the best we can hope for using BuildBot is to > track a core BioRuby repository, but for the BioRuby Gems > we'd just be able to test the latest version. That would still be > better than nothing, but you wouldn't be able to easily work > out which revision of a Gem caused a problem for example. bio-core and bio-core-ext are versioned and have *versioned* dependencies. I.e. each included gem has a fixed version. That is exactly what we want, so we can support users. You don't want every user to have different combinations of gems. No way we can troubleshoot the latter! This way we can also pull the consistent case for testing through buildbot! That is we test current releases of bioruby and two meta-packages - just need a script to unpack them and run the tests. For biogems individually we can test the last revision on the master branch of each github repo. Which is a good thing too, because that is what users resort to, if the meta-gem is not up-to-date enough. Obviously, we don't *support* that. It is a risk the user is taking. Any issues he'll have to resolve with the maintainer of the gem. Pj. From pjotr.public14 at thebird.nl Thu Sep 22 10:15:42 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 16:15:42 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: <20110922134803.GB27229@thebird.nl> References: <20110922052148.GA22535@thebird.nl> <20110922122451.GA24244@thebird.nl> <20110922134803.GB27229@thebird.nl> Message-ID: <20110922141542.GA28321@thebird.nl> A versioned gem can be unpacked in place with gem unpack bio-blastxmlparser Unpacked gem: '/home/export/izip/git/projects/www/biogems.info/bio-blastxmlparser-1.0.1' cd bio-blast* install the build dependencies bundle install run tests rake test or here (I should add this to Rake). rspec spec/* Finished in 0.426 seconds 9 examples, 0 failures The versioned gems are listed inside bio-core, see https://github.com/helios/bioruby-core/blob/master/Rakefile e.g. gem.add_runtime_dependency 'bio-gff3',["= 0.8.7"] Anyone installing the bio-core 0.0.1 will get bio-gff3 0.8.7. No other version. We can automate this. Pj. From cjfields at illinois.edu Thu Sep 22 11:27:20 2011 From: cjfields at illinois.edu (Fields, Christopher J) Date: Thu, 22 Sep 2011 15:27:20 +0000 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: <20110922052148.GA22535@thebird.nl> Message-ID: <2165313F-B505-4C5A-AA00-69FE07FE2548@illinois.edu> On Sep 22, 2011, at 7:03 AM, Peter Cock wrote: > On Thu, Sep 22, 2011 at 6:21 AM, Pjotr Prins wrote: >> Hi Peter, >> >> It is for reason of limited installs we have meta-packages. >> Meta-packages provide a sense of stability, and is what most users >> will install. >> >> I suggest testing for: >> >> - bio >> - bio-core, which contains core pure ruby biogems >> - bio-core-ext, which contains non-pure ruby biogems >> (extensions) >> >> anything worth testing will be in there. We can help create the test >> scripts for those packages. > > What I am trying to understand is do all those bits live in many > different repositories, or is there one super repository? And if > there isn't a single repository, can we create a meta-repository > (e.g. using git submodules). This is the plan for bioperl; namely tying together working code from various soon-to-be-split repos. One could then checkout the parent repo and the specific versions of the submodules, with the end game of tagging releases of the parent to specific commits/tags/branches of working submodules. Of course I haven't tested this extensively (just started looking into it recently), not even sure tagging of the parent is possible, though if it isn't it certainly diminishes any utility of 'git submodule'. And I'm not sure how maintainable that will be in the long run. >> Also, it looks like the testing environment should be hosted in a >> ready made VM image. I can create a CloudBioLinux flavour for that. >> Once we know what to install. I will also want tests for biolib_hpc, >> when that gets production ready (btw). It is a long standing wish on >> my end to get this sorted. > > Perhaps I have misunderstood you, but if all the build slaves > run the tests via the same VM image, there is no diversity. > For Biopython we deliberately want inhomogeneous setups > for our buildslaves to try and cover the variety of potential > setups our end users will have. > > Peter The way this is handled via perl is to either use smoke testing during the build (as Peter alluded to earlier) or to run tests during CPAN installation. Either one requires a little more overhead, but in re: to the latter we do get a nice spread of OS and perl versions to test against: http://matrix.cpantesters.org/?dist=BioPerl+1.6.901 (obviously from that link we need windows devs to help out, they seem to be scarce these days :) We don't have smoke set up for bioperl yet (this would allow testing the repo on a more regular basis instead of per-release), though installation of Smolder on testing.open-bio.org was mentioned at one point but fell through the cracks. chris From pjotr.public14 at thebird.nl Fri Sep 30 10:38:47 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 30 Sep 2011 16:38:47 +0200 Subject: [BioRuby] BioRuby developers worldwide Message-ID: <20110930143847.GA8366@thebird.nl> As if it is not enough, Raoul has combined biogems with the openstreetmap: http://biogems.info/usersmap.html South-America and Russia are conspicuously absent... Pj. From chmille4 at gmail.com Fri Sep 30 10:42:05 2011 From: chmille4 at gmail.com (Chase Miller) Date: Fri, 30 Sep 2011 10:42:05 -0400 Subject: [BioRuby] BioRuby developers worldwide In-Reply-To: <20110930143847.GA8366@thebird.nl> References: <20110930143847.GA8366@thebird.nl> Message-ID: Nice work! Really cool. (From the boston guy) Chase On Fri, Sep 30, 2011 at 10:38 AM, Pjotr Prins wrote: > As if it is not enough, Raoul has combined biogems with the > openstreetmap: > > http://biogems.info/usersmap.html > > South-America and Russia are conspicuously absent... > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From hlapp at drycafe.net Fri Sep 30 11:05:50 2011 From: hlapp at drycafe.net (Hilmar Lapp) Date: Fri, 30 Sep 2011 11:05:50 -0400 Subject: [BioRuby] BioRuby developers worldwide In-Reply-To: <20110930143847.GA8366@thebird.nl> References: <20110930143847.GA8366@thebird.nl> Message-ID: Antarctica, too, actually, which I find really disappointing. Let alone no kiwis. -hilmar On Sep 30, 2011, at 10:38 AM, Pjotr Prins wrote: > As if it is not enough, Raoul has combined biogems with the > openstreetmap: > > http://biogems.info/usersmap.html > > South-America and Russia are conspicuously absent... > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby -- =========================================================== : Hilmar Lapp -:- Durham, NC -:- hlapp at drycafe dot net : =========================================================== From p.j.a.cock at googlemail.com Thu Sep 15 19:17:57 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Thu, 15 Sep 2011 20:17:57 +0100 Subject: [BioRuby] Running BioRuby tests offline Message-ID: Hi all, I recently tried running the BioRuby test suite while offline (on a train), $ ruby runner.rb Loaded suite . Started ...............F.........EE.....EEEEEEEEFFEFEEEFEE...EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFFFFFE..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FF............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Finished in 100.74387 seconds. 1) Failure: test_read_uri(Bio::FuncTestCommandNet) [./functional/bio/test_command.rb:337]: Exception raised: Class: Message: <"getaddrinfo: nodename nor servname provided, or not known"> ---Backtrace--- ... 73) Failure: test_libxml(Bio::TestPhyloXML_Check_LibXML) [./unit/bio/db/test_phyloxml.rb:29]: Error: libxml-ruby library is not present. Please install libxml-ruby library. It is needed for Bio::PhyloXML module. Unit test for PhyloXML will not be performed. is not true. 3640 tests, 20824 assertions, 26 failures, 47 errors By eye, all 73 failures or errors were down to missing the XML library, or lacking network access. Is there a way to run the test suite skipping the tests requiring network access? If BioRuby starts running nightly tests (I was talking to Raoul about this at the BioHackathon last month), an offline switch would seem a good idea. We do this in Biopython to avoid hitting any online servers too hard, and because network issues would otherwise get flagged as potential regressions (false positives). Regards, Peter From mphoeppner at gmail.com Mon Sep 19 08:57:14 2011 From: mphoeppner at gmail.com (Marc P. Hoeppner) Date: Mon, 19 Sep 2011 10:57:14 +0200 Subject: [BioRuby] Bio::Fetch issue - biofetch.rb not found Message-ID: <4E7703EA.100@gmail.com> Hi, when trying to fetch entries from remote databases via Bio::Fetch, I receive the following error: /usr/lib/ruby/1.8/open-uri.rb:277:in `open_http': 404 Not Found (OpenURI::HTTPError) It appears the default entry for Fetch.new searches for http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at that location? Am I doing anything wrong? Using gem version 1.4.2 and ruby 1.8.7 on Ubuntu 11.04 /Marc From pjotr.public14 at thebird.nl Tue Sep 20 16:56:01 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 20 Sep 2011 18:56:01 +0200 Subject: [BioRuby] biogems.info updated Message-ID: <20110920165601.GA8933@thebird.nl> I have updated Biogems to show the number of outstanding issues, for each plugin, as listed on github. Also a number of additional plugins are listed now as core and stable. If you feel your plugin should be listed as such, please bring that up here. See http://biogems.info/ or http://gems.bioruby.org/ Enjoy. Pj. From p.j.a.cock at googlemail.com Wed Sep 21 15:44:03 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 16:44:03 +0100 Subject: [BioRuby] Broken link on wiki installation page Message-ID: Hello all, I was looking at the Installation page on the Wiki (to see what the minimum version of Ruby required was): http://bioruby.open-bio.org/wiki/Installation The first line links to a README file, but this URL doesn't work: http://bioruby.open-bio.org/rdoc/files/README_rdoc.html Should this now be the following?: http://bioruby.open-bio.org/rdoc/README_rdoc.html Regards, Peter From p.j.a.cock at googlemail.com Wed Sep 21 16:04:35 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 17:04:35 +0100 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> Message-ID: On Wed, Sep 21, 2011 at 4:56 PM, Naohisa GOTO wrote: > Hi Peter, > > Thank you. I've changed the link. > The URL is changed due to the RDoc renderer's version up. > > Naohisa Goto Great. I see you require currently require Ruby 1.8.6 or later (except Ruby 1.9.0) I'm thinking about potential build slaves for nightly testing (something Raoul and I chatted about the BioHackathon). That means my Mac OS X 10.6.8 Snow Leopard with Ruby 1.8.7 should be fine. On the other hand, one of our CentOS servers which use as a Biopython buildslave only has Ruby 1.8.5, so that won't work without me installing a more recent version of ruby under the user account. Thanks, Peter From pjotr.public14 at thebird.nl Wed Sep 21 16:16:05 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 21 Sep 2011 18:16:05 +0200 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <20110921161605.GA18787@thebird.nl> Hi Peter, You should look into rvm. I tried avoiding using rvm - but now I do. I gave in. rvm should work in user land. Are you also planning to add testing for other biogems? At least bio-core and bio-core-ext (which include the 'core' gems). It would be really nice to have broader coverage. Pj. On Wed, Sep 21, 2011 at 05:04:35PM +0100, Peter Cock wrote: > On Wed, Sep 21, 2011 at 4:56 PM, Naohisa GOTO > wrote: > > Hi Peter, > > > > Thank you. I've changed the link. > > The URL is changed due to the RDoc renderer's version up. > > > > Naohisa Goto > > Great. > > I see you require currently require Ruby 1.8.6 or later (except Ruby 1.9.0) > > I'm thinking about potential build slaves for nightly testing > (something Raoul and I chatted about the BioHackathon). > > That means my Mac OS X 10.6.8 Snow Leopard with Ruby 1.8.7 should be fine. > > On the other hand, one of our CentOS servers which use as a Biopython > buildslave only has Ruby 1.8.5, so that won't work without me > installing a more recent version of ruby under the user account. > > Thanks, > > Peter > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From ngoto at gen-info.osaka-u.ac.jp Wed Sep 21 15:56:33 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 00:56:33 +0900 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: References: Message-ID: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> Hi Peter, Thank you. I've changed the link. The URL is changed due to the RDoc renderer's version up. Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Wed, 21 Sep 2011 16:44:03 +0100 Peter Cock wrote: > Hello all, > > I was looking at the Installation page on the Wiki (to see > what the minimum version of Ruby required was): > http://bioruby.open-bio.org/wiki/Installation > > The first line links to a README file, but this URL doesn't work: > http://bioruby.open-bio.org/rdoc/files/README_rdoc.html > > Should this now be the following?: > http://bioruby.open-bio.org/rdoc/README_rdoc.html > > Regards, > > Peter > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From cjfields at illinois.edu Wed Sep 21 16:20:50 2011 From: cjfields at illinois.edu (Fields, Christopher J) Date: Wed, 21 Sep 2011 16:20:50 +0000 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: <20110921161605.GA18787@thebird.nl> References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> <20110921161605.GA18787@thebird.nl> Message-ID: <0D660C10-EAD3-43EE-8BC2-4767DBAB78AD@illinois.edu> Ah, see the buildbot system is up? Methinks we need to set up bioperl... chris On Sep 21, 2011, at 11:16 AM, Pjotr Prins wrote: > Hi Peter, > > You should look into rvm. I tried avoiding using rvm - but now I do. > I gave in. rvm should work in user land. > > Are you also planning to add testing for other biogems? At least > bio-core and bio-core-ext (which include the 'core' gems). It would be > really nice to have broader coverage. > > Pj. > > On Wed, Sep 21, 2011 at 05:04:35PM +0100, Peter Cock wrote: >> On Wed, Sep 21, 2011 at 4:56 PM, Naohisa GOTO >> wrote: >>> Hi Peter, >>> >>> Thank you. I've changed the link. >>> The URL is changed due to the RDoc renderer's version up. >>> >>> Naohisa Goto >> >> Great. >> >> I see you require currently require Ruby 1.8.6 or later (except Ruby 1.9.0) >> >> I'm thinking about potential build slaves for nightly testing >> (something Raoul and I chatted about the BioHackathon). >> >> That means my Mac OS X 10.6.8 Snow Leopard with Ruby 1.8.7 should be fine. >> >> On the other hand, one of our CentOS servers which use as a Biopython >> buildslave only has Ruby 1.8.5, so that won't work without me >> installing a more recent version of ruby under the user account. >> >> Thanks, >> >> Peter >> _______________________________________________ >> BioRuby Project - http://www.bioruby.org/ >> BioRuby mailing list >> BioRuby at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/bioruby >> > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From p.j.a.cock at googlemail.com Wed Sep 21 20:02:01 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 21:02:01 +0100 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: <20110921161605.GA18787@thebird.nl> References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> <20110921161605.GA18787@thebird.nl> Message-ID: On Wednesday, September 21, 2011, Pjotr Prins wrote: > Hi Peter, > > You should look into rvm. I tried avoiding using rvm - but now I do. > I gave in. rvm should work in user land. > > Are you also planning to add testing for other biogems? At least > bio-core and bio-core-ext (which include the 'core' gems). It would be > really nice to have broader coverage. > > Pj. Initially I was thinking just the core BioRuby tests, but if it is simple to install/update all the gems and run their tests too that's possible too. You may need to do something to ensure missing dependencies (eg 3rd party binaries) and handled gracefully - skipped not fails. And I would recommend disabling the online tests (see other thread). Peter From pjotr.public14 at thebird.nl Wed Sep 21 20:07:11 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 21 Sep 2011 22:07:11 +0200 Subject: [BioRuby] Broken link on wiki installation page In-Reply-To: References: <20110921155633.A5FCB1CBC599@idnmail.gen-info.osaka-u.ac.jp> <20110921161605.GA18787@thebird.nl> Message-ID: <20110921200711.GA20122@thebird.nl> Normally it is just running 'rake test' in the base dir. I think gems can test also at install time. Anyway, I am sure some of us are interested in helping out, including myself. On Wed, Sep 21, 2011 at 09:02:01PM +0100, Peter Cock wrote: > On Wednesday, September 21, 2011, Pjotr Prins > <[1]pjotr.public14 at thebird.nl> wrote: > > Hi Peter, > > > > You should look into rvm. I tried avoiding using rvm - but now I do. > > I gave in. rvm should work in user land. > > > > Are you also planning to add testing for other biogems? At least > > bio-core and bio-core-ext (which include the 'core' gems). It would be > > really nice to have broader coverage. > > > > Pj. > > Initially I was thinking just the core BioRuby tests, but > if it is simple to install/update all the gems and run their > tests too that's possible too. > > You may need to do something to ensure missing > dependencies (eg 3rd party binaries) and handled > gracefully - skipped not fails. > > And I would recommend disabling the online tests > (see other thread). > > Peter > > References > > Visible links > 1. mailto:pjotr.public14 at thebird.nl From p.j.a.cock at googlemail.com Wed Sep 21 21:17:46 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Wed, 21 Sep 2011 22:17:46 +0100 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page Message-ID: Hello all, Note I've retitled the thread since the topic has change. Apologies for the long email - I was hoping to talk though most of this first with Raoul, Goto-san and Toshiaki-san while in Japan, but there was plenty of other things going on. Sorry. On Wed, Sep 21, 2011 at 9:07 PM, Pjotr Prins wrote: > Normally it is just running 'rake test' in the base dir. I think > gems can test also at install time. Actually I may have been over enthusiastic regarding testing the BioRuby gems: http://lists.open-bio.org/pipermail/bioruby/2011-September/001896.html One of the key points of the buildbot software (which we are using for the Biopython nightly testing) is it is linked to a repository, and you can see the test results versus the revisions. This includes git support which is good. You can browse this here: http://testing.open-bio.org/biopython/ Personally I like the transposed grid on my widescreen monitor at work, where each row is a revision: http://testing.open-bio.org/biopython/tgrid When the row turns red, you have a major cross platform regression. When a subgroup turns red, you have a platform specific failure (e.g. just Windows, or a specific version of Python). We also have failure emails sent out, and this can be followed by an RSS feed too. There is also a time based view, the "waterfall": http://testing.open-bio.org/biopython/waterfall If all the tests are triggered automatically on a nightly basis, the chronological order of the tests will match the changes made to the repository. However, via the web interface (with a password) you can request a buildslave run the tests using a particular branch/revision. This is handy if a buildslave was offline for some reason and missed a nightly test. Now, in the typical setup you have one software suite with one repository and one test suite. That's what we have for Biopython, and what I was picturing initially for BioRuby. In the case of the BioRuby gems, we could set this up against the core BioRuby repository, and have the core BioRuby tests run (against the known revision from that repository) plus the BioGem tests run with the latest Gem code. The downside is that wouldn't track changes to the Gems themselves. Is that likely to be a problem? I've not looked into the details of possible multi-repository setups - perhaps Tiago knows more (he setup the buildbot for Biopython, CC'd). Thinking out loud, one solution would be a meta repository - I think you can have a git repository with sub projects... maybe that could be done for the BioRuby core plus (important) Gems? Alternatively, perhaps a different test server infrastructure might support a composite project with multiple sub projects, each with their own repository? You don't have to use buildbot - Chris Fields was talking about something called smoke test for BioPerl (CC'd). > Anyway, I am sure some of us are > interested in helping out, including myself. > Excellent. You (BioRuby) will need several volunteer machines as buildslaves which are are either on all the time, or at least online regularly. I'm willing to offer one. You (BioRuby) will also need to decide which platforms you'll be testing - by which I mean what combinations of operating system and version of Ruby. You will need at least one buildslave for each platform. I plan to offer at least one of my machines (it will make testing an initial setup much easier). For Biopython we try to cover Mac, Windows, Linux, both 32 bit and 64 bit, and a range of versions of the standard Python implementation (in C) plus also Jython (Python on the Java Virtual Machine), and in the future perhaps PyPy and IronPython. For each platform we will need the (short) steps to build/compile BioRuby from a fresh git checkout, and run the tests. e.g. For Biopython under Windows on Python 2.5, C:\Python25\python.exe setup.py build C:\Python25\python.exe setup.py test While on Linux on Python 2.7 it would be: python2.7 setup.py build python2.7 setup,py test We try and have each buildslave cover a couple of different versions of Python - thus we use explicit and slightly different commands in each case (rather than just using the default installation of Python). We can go though a lot of this kind of detail off the mailing list if you prefer. I hope I haven't bored most of the mailing list subscribers. There isn't a separate list for BioRuby development is there? Regards, Peter From pjotr.public14 at thebird.nl Thu Sep 22 05:21:48 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 07:21:48 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: Message-ID: <20110922052148.GA22535@thebird.nl> Hi Peter, It is for reason of limited installs we have meta-packages. Meta-packages provide a sense of stability, and is what most users will install. I suggest testing for: - bio - bio-core, which contains core pure ruby biogems - bio-core-ext, which contains non-pure ruby biogems (extensions) anything worth testing will be in there. We can help create the test scripts for those packages. Also, it looks like the testing environment should be hosted in a ready made VM image. I can create a CloudBioLinux flavour for that. Once we know what to install. I will also want tests for biolib_hpc, when that gets production ready (btw). It is a long standing wish on my end to get this sorted. Pj. On Wed, Sep 21, 2011 at 10:17:46PM +0100, Peter Cock wrote: > Hello all, > > Note I've retitled the thread since the topic has change. > > Apologies for the long email - I was hoping to talk > though most of this first with Raoul, Goto-san and > Toshiaki-san while in Japan, but there was plenty > of other things going on. Sorry. > > On Wed, Sep 21, 2011 at 9:07 PM, Pjotr Prins wrote: > > Normally it is just running 'rake test' in the base dir. I think > > gems can test also at install time. > > Actually I may have been over enthusiastic regarding > testing the BioRuby gems: > http://lists.open-bio.org/pipermail/bioruby/2011-September/001896.html > > One of the key points of the buildbot software (which > we are using for the Biopython nightly testing) is it is > linked to a repository, and you can see the test results > versus the revisions. This includes git support which > is good. You can browse this here: > http://testing.open-bio.org/biopython/ > > Personally I like the transposed grid on my widescreen > monitor at work, where each row is a revision: > http://testing.open-bio.org/biopython/tgrid > > When the row turns red, you have a major cross > platform regression. When a subgroup turns red, > you have a platform specific failure (e.g. just > Windows, or a specific version of Python). > > We also have failure emails sent out, and this can > be followed by an RSS feed too. > > There is also a time based view, the "waterfall": > http://testing.open-bio.org/biopython/waterfall > > If all the tests are triggered automatically on a > nightly basis, the chronological order of the tests > will match the changes made to the repository. > However, via the web interface (with a password) > you can request a buildslave run the tests using > a particular branch/revision. This is handy if a > buildslave was offline for some reason and > missed a nightly test. > > Now, in the typical setup you have one software > suite with one repository and one test suite. > That's what we have for Biopython, and what > I was picturing initially for BioRuby. > > In the case of the BioRuby gems, we could set > this up against the core BioRuby repository, and > have the core BioRuby tests run (against the > known revision from that repository) plus the > BioGem tests run with the latest Gem code. > The downside is that wouldn't track changes > to the Gems themselves. Is that likely to be > a problem? > > I've not looked into the details of possible > multi-repository setups - perhaps Tiago > knows more (he setup the buildbot for > Biopython, CC'd). > > Thinking out loud, one solution would be a > meta repository - I think you can have a git > repository with sub projects... maybe that > could be done for the BioRuby core plus > (important) Gems? > > Alternatively, perhaps a different test server > infrastructure might support a composite > project with multiple sub projects, each > with their own repository? You don't have > to use buildbot - Chris Fields was talking > about something called smoke test for > BioPerl (CC'd). > > > Anyway, I am sure some of us are > > interested in helping out, including myself. > > > > Excellent. You (BioRuby) will need several volunteer > machines as buildslaves which are are either on all > the time, or at least online regularly. I'm willing to offer > one. > > You (BioRuby) will also need to decide which platforms > you'll be testing - by which I mean what combinations > of operating system and version of Ruby. You will need > at least one buildslave for each platform. I plan to > offer at least one of my machines (it will make testing > an initial setup much easier). > > For Biopython we try to cover Mac, Windows, Linux, > both 32 bit and 64 bit, and a range of versions of the > standard Python implementation (in C) plus also > Jython (Python on the Java Virtual Machine), and > in the future perhaps PyPy and IronPython. > > For each platform we will need the (short) steps > to build/compile BioRuby from a fresh git checkout, > and run the tests. > > e.g. For Biopython under Windows on Python 2.5, > > C:\Python25\python.exe setup.py build > C:\Python25\python.exe setup.py test > > While on Linux on Python 2.7 it would be: > > python2.7 setup.py build > python2.7 setup,py test > > We try and have each buildslave cover a couple of different > versions of Python - thus we use explicit and slightly different > commands in each case (rather than just using the default > installation of Python). > > We can go though a lot of this kind of detail off the > mailing list if you prefer. > > I hope I haven't bored most of the mailing list > subscribers. There isn't a separate list for BioRuby > development is there? > > Regards, > > Peter > From ngoto at gen-info.osaka-u.ac.jp Thu Sep 22 06:12:07 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 15:12:07 +0900 Subject: [BioRuby] Bio::Fetch issue - biofetch.rb not found In-Reply-To: <4E7703EA.100@gmail.com> References: <4E7703EA.100@gmail.com> Message-ID: <20110922061207.F0BD01CBC414@idnmail.gen-info.osaka-u.ac.jp> Hello Marc, Sorry for late response. > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > that location? The service is down for years, due to the server trouble, and it is currently difficult to restore the service because of the internal structure of bioruby.org. Please use other server, e.g. http://www.ebi.ac.uk/Tools/dbfetch/dbfetch bf =Bio::Fetch.new('http://www.ebi.ac.uk/Tools/dbfetch/dbfetch') str = bf.fetch('embl', 'AE014292') print str Alternatively, some other classes provide similar function. str = Bio::TogoWS::REST.entry('genbank', 'AF237819') print str Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Mon, 19 Sep 2011 10:57:14 +0200 "Marc P. Hoeppner" wrote: > Hi, > > when trying to fetch entries from remote databases via Bio::Fetch, I > receive the following error: > > /usr/lib/ruby/1.8/open-uri.rb:277:in `open_http': 404 Not Found > (OpenURI::HTTPError) > > It appears the default entry for Fetch.new searches for > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > that location? > > Am I doing anything wrong? > > Using gem version 1.4.2 and ruby 1.8.7 on Ubuntu 11.04 > > /Marc > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From p.j.a.cock at googlemail.com Thu Sep 22 12:03:11 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Thu, 22 Sep 2011 13:03:11 +0100 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: <20110922052148.GA22535@thebird.nl> References: <20110922052148.GA22535@thebird.nl> Message-ID: On Thu, Sep 22, 2011 at 6:21 AM, Pjotr Prins wrote: > Hi Peter, > > It is for reason of limited installs we have meta-packages. > Meta-packages provide a sense of stability, and is what most users > will install. > > I suggest testing for: > > ?- bio > ?- bio-core, which contains core pure ruby biogems > ?- bio-core-ext, which contains non-pure ruby biogems > ? ?(extensions) > > anything worth testing will be in there. We can help create the test > scripts for those packages. What I am trying to understand is do all those bits live in many different repositories, or is there one super repository? And if there isn't a single repository, can we create a meta-repository (e.g. using git submodules). > Also, it looks like the testing environment should be hosted in a > ready made VM image. I can create a CloudBioLinux flavour for that. > Once we know what to install. I will also want tests for biolib_hpc, > when that gets production ready (btw). It is a long standing wish on > my end to get this sorted. Perhaps I have misunderstood you, but if all the build slaves run the tests via the same VM image, there is no diversity. For Biopython we deliberately want inhomogeneous setups for our buildslaves to try and cover the variety of potential setups our end users will have. Peter From ngoto at gen-info.osaka-u.ac.jp Thu Sep 22 12:18:13 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 21:18:13 +0900 Subject: [BioRuby] Running BioRuby tests offline In-Reply-To: References: Message-ID: <20110922121813.C6EB01CBC414@idnmail.gen-info.osaka-u.ac.jp> Hi Peter, The proposal is marked as a Feature Request to BioRuby. https://redmine.open-bio.org/issues/3293 Thanks, Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Thu, 15 Sep 2011 20:17:57 +0100 Peter Cock wrote: > Hi all, > > I recently tried running the BioRuby test suite while offline (on a train), > > > $ ruby runner.rb > Loaded suite . > Started > ...............F.........EE.....EEEEEEEEFFEFEEEFEE...EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFFFFFE..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... . > ! > ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................FF............................................................................................................................. .! > > ...................................................................... > ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ . > ! > ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ > Finished in 100.74387 seconds. > > 1) Failure: > test_read_uri(Bio::FuncTestCommandNet) [./functional/bio/test_command.rb:337]: > Exception raised: > Class: > Message: <"getaddrinfo: nodename nor servname provided, or not known"> > ---Backtrace--- > ... > 73) Failure: > test_libxml(Bio::TestPhyloXML_Check_LibXML) [./unit/bio/db/test_phyloxml.rb:29]: > Error: libxml-ruby library is not present. Please install libxml-ruby > library. It is needed for Bio::PhyloXML module. Unit test for PhyloXML > will not be performed. > is not true. > > 3640 tests, 20824 assertions, 26 failures, 47 errors > > By eye, all 73 failures or errors were down to missing the XML > library, or lacking network access. > > Is there a way to run the test suite skipping the tests requiring > network access? > > If BioRuby starts running nightly tests (I was talking to Raoul about > this at the BioHackathon last month), an offline switch would seem a > good idea. We do this in Biopython to avoid hitting any online servers > too hard, and because network issues would otherwise get flagged as > potential regressions (false positives). > > Regards, > > Peter > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From pjotr.public14 at thebird.nl Thu Sep 22 12:24:51 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 14:24:51 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: <20110922052148.GA22535@thebird.nl> Message-ID: <20110922122451.GA24244@thebird.nl> Hi, On Thu, Sep 22, 2011 at 01:03:11PM +0100, Peter Cock wrote: > What I am trying to understand is do all those bits live in many > different repositories, Yes. biogems are individual efforts - we actively encourage that. You have to understand that the combined biogems are arguably more important now than the core bioruby project. Though both have their roles. > or is there one super repository? And if there isn't a single > repository, can we create a meta-repository (e.g. using git > submodules). git submodules are not that great. I used them for biolib. They are nice and not nice ;) Like I wrote, we track repositories through biogems, bio-core and bio-core-ext. I think you aim to check out the master repositories (development branch) for testing, right? So we can write a script to do just that. I would prefer to track the meta-gems, as well as the regular biogems. Test results can be listed on biogems.info. Would it work to clone all repositories in one directory, and run both combined and individual tests? We can do the scripting of that. > > Also, it looks like the testing environment should be hosted in a > > ready made VM image. I can create a CloudBioLinux flavour for that. > > Once we know what to install. I will also want tests for biolib_hpc, > > when that gets production ready (btw). It is a long standing wish on > > my end to get this sorted. > > Perhaps I have misunderstood you, but if all the build slaves > run the tests via the same VM image, there is no diversity. > For Biopython we deliberately want inhomogeneous setups > for our buildslaves to try and cover the variety of potential > setups our end users will have. Different VMs can be created. For each target one. I am primarily interested in a Debian VM - as that covers >80% of users in bioinformatics. I am not saying it replaces your installations - but it would certainly add to it. Pj. From ngoto at gen-info.osaka-u.ac.jp Thu Sep 22 12:26:28 2011 From: ngoto at gen-info.osaka-u.ac.jp (Naohisa GOTO) Date: Thu, 22 Sep 2011 21:26:28 +0900 Subject: [BioRuby] Bio::Fetch issue - biofetch.rb not found In-Reply-To: <20110922061207.F0BD01CBC414@idnmail.gen-info.osaka-u.ac.jp> References: <4E7703EA.100@gmail.com> <20110922061207.F0BD01CBC414@idnmail.gen-info.osaka-u.ac.jp> Message-ID: <20110922122629.537531CBC511@idnmail.gen-info.osaka-u.ac.jp> Hi, The issue is marked as a bug in the OBF Redmine. https://redmine.open-bio.org/issues/3294 Thanks, Naohisa Goto ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org On Thu, 22 Sep 2011 15:12:07 +0900 Naohisa GOTO wrote: > Hello Marc, > > Sorry for late response. > > > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > > that location? > > The service is down for years, due to the server trouble, > and it is currently difficult to restore the service > because of the internal structure of bioruby.org. > > Please use other server, e.g. > http://www.ebi.ac.uk/Tools/dbfetch/dbfetch > > bf =Bio::Fetch.new('http://www.ebi.ac.uk/Tools/dbfetch/dbfetch') > str = bf.fetch('embl', 'AE014292') > print str > > Alternatively, some other classes provide similar function. > > str = Bio::TogoWS::REST.entry('genbank', 'AF237819') > print str > > Naohisa Goto > ngoto at gen-info.osaka-u.ac.jp / ng at bioruby.org > > On Mon, 19 Sep 2011 10:57:14 +0200 > "Marc P. Hoeppner" wrote: > > > Hi, > > > > when trying to fetch entries from remote databases via Bio::Fetch, I > > receive the following error: > > > > /usr/lib/ruby/1.8/open-uri.rb:277:in `open_http': 404 Not Found > > (OpenURI::HTTPError) > > > > It appears the default entry for Fetch.new searches for > > http://bioruby.org/cgi-bin/biofetch.rb - which does not seem to exist at > > that location? > > > > Am I doing anything wrong? > > > > Using gem version 1.4.2 and ruby 1.8.7 on Ubuntu 11.04 > > > > /Marc > > _______________________________________________ > > BioRuby Project - http://www.bioruby.org/ > > BioRuby mailing list > > BioRuby at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/bioruby > > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby From p.j.a.cock at googlemail.com Thu Sep 22 13:31:19 2011 From: p.j.a.cock at googlemail.com (Peter Cock) Date: Thu, 22 Sep 2011 14:31:19 +0100 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: <20110922122451.GA24244@thebird.nl> References: <20110922052148.GA22535@thebird.nl> <20110922122451.GA24244@thebird.nl> Message-ID: On Thu, Sep 22, 2011 at 1:24 PM, Pjotr Prins wrote: > Hi, > > On Thu, Sep 22, 2011 at 01:03:11PM +0100, Peter Cock wrote: >> What I am trying to understand is do all those bits live in many >> different repositories, > > Yes. biogems are individual efforts - we actively encourage that. You > have to understand that the combined biogems are arguably more > important now than the core bioruby project. Though both have their > roles. OK. >> or is there one super repository? And if there isn't a single >> repository, can we create a meta-repository (e.g. using git >> submodules). > > git submodules are not that great. I used them for biolib. They are > nice and not nice ;) > > Like I wrote, we track repositories through biogems, bio-core and > bio-core-ext. I think you aim to check out the master repositories > (development branch) for testing, right? ?So we can write a script to > do just that. I would prefer to track the meta-gems, as well as > the regular biogems. Test results can be listed on biogems.info. > > Would it work to clone all repositories in one directory, and run > both combined and individual tests? We can do the scripting of that. It sounds like the buildbot software we're using for the Biopython testing will probably not cover this use case, from their FAQ: http://trac.buildbot.net/wiki/FAQ "However, a project that packages several independently-developed utilities into a single binary for users to download is more difficult, as Buildbot does not have a means to represent the multiple revisions from multiple repositories that must be used to make up the single binary. For simple cases (e.g,. always building the latest version), this can be made to work, but more complex cases are still a work in progress." It sounds like the best we can hope for using BuildBot is to track a core BioRuby repository, but for the BioRuby Gems we'd just be able to test the latest version. That would still be better than nothing, but you wouldn't be able to easily work out which revision of a Gem caused a problem for example. >> > Also, it looks like the testing environment should be hosted in a >> > ready made VM image. I can create a CloudBioLinux flavour for that. >> > Once we know what to install. I will also want tests for biolib_hpc, >> > when that gets production ready (btw). It is a long standing wish on >> > my end to get this sorted. >> >> Perhaps I have misunderstood you, but if all the build slaves >> run the tests via the same VM image, there is no diversity. >> For Biopython we deliberately want inhomogeneous setups >> for our buildslaves to try and cover the variety of potential >> setups our end users will have. > > Different VMs can be created. For each target one. I am primarily > interested in a Debian VM - as that covers >80% of users in > bioinformatics. I am not saying it replaces your installations - but > it would certainly add to it. OK, having a Debian VM as one of several BioRuby buildslaves covering a range of setups makes sense. Peter From pjotr.public14 at thebird.nl Thu Sep 22 13:48:03 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 15:48:03 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: <20110922052148.GA22535@thebird.nl> <20110922122451.GA24244@thebird.nl> Message-ID: <20110922134803.GB27229@thebird.nl> On Thu, Sep 22, 2011 at 02:31:19PM +0100, Peter Cock wrote: > "However, a project that packages several independently-developed > utilities into a single binary for users to download is more difficult, > as Buildbot does not have a means to represent the multiple > revisions from multiple repositories that must be used to make > up the single binary. For simple cases (e.g,. always building the > latest version), this can be made to work, but more complex cases > are still a work in progress." > > It sounds like the best we can hope for using BuildBot is to > track a core BioRuby repository, but for the BioRuby Gems > we'd just be able to test the latest version. That would still be > better than nothing, but you wouldn't be able to easily work > out which revision of a Gem caused a problem for example. bio-core and bio-core-ext are versioned and have *versioned* dependencies. I.e. each included gem has a fixed version. That is exactly what we want, so we can support users. You don't want every user to have different combinations of gems. No way we can troubleshoot the latter! This way we can also pull the consistent case for testing through buildbot! That is we test current releases of bioruby and two meta-packages - just need a script to unpack them and run the tests. For biogems individually we can test the last revision on the master branch of each github repo. Which is a good thing too, because that is what users resort to, if the meta-gem is not up-to-date enough. Obviously, we don't *support* that. It is a risk the user is taking. Any issues he'll have to resolve with the maintainer of the gem. Pj. From pjotr.public14 at thebird.nl Thu Sep 22 14:15:42 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 22 Sep 2011 16:15:42 +0200 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: <20110922134803.GB27229@thebird.nl> References: <20110922052148.GA22535@thebird.nl> <20110922122451.GA24244@thebird.nl> <20110922134803.GB27229@thebird.nl> Message-ID: <20110922141542.GA28321@thebird.nl> A versioned gem can be unpacked in place with gem unpack bio-blastxmlparser Unpacked gem: '/home/export/izip/git/projects/www/biogems.info/bio-blastxmlparser-1.0.1' cd bio-blast* install the build dependencies bundle install run tests rake test or here (I should add this to Rake). rspec spec/* Finished in 0.426 seconds 9 examples, 0 failures The versioned gems are listed inside bio-core, see https://github.com/helios/bioruby-core/blob/master/Rakefile e.g. gem.add_runtime_dependency 'bio-gff3',["= 0.8.7"] Anyone installing the bio-core 0.0.1 will get bio-gff3 0.8.7. No other version. We can automate this. Pj. From cjfields at illinois.edu Thu Sep 22 15:27:20 2011 From: cjfields at illinois.edu (Fields, Christopher J) Date: Thu, 22 Sep 2011 15:27:20 +0000 Subject: [BioRuby] Nightly build testing, was: Broken link on wiki installation page In-Reply-To: References: <20110922052148.GA22535@thebird.nl> Message-ID: <2165313F-B505-4C5A-AA00-69FE07FE2548@illinois.edu> On Sep 22, 2011, at 7:03 AM, Peter Cock wrote: > On Thu, Sep 22, 2011 at 6:21 AM, Pjotr Prins wrote: >> Hi Peter, >> >> It is for reason of limited installs we have meta-packages. >> Meta-packages provide a sense of stability, and is what most users >> will install. >> >> I suggest testing for: >> >> - bio >> - bio-core, which contains core pure ruby biogems >> - bio-core-ext, which contains non-pure ruby biogems >> (extensions) >> >> anything worth testing will be in there. We can help create the test >> scripts for those packages. > > What I am trying to understand is do all those bits live in many > different repositories, or is there one super repository? And if > there isn't a single repository, can we create a meta-repository > (e.g. using git submodules). This is the plan for bioperl; namely tying together working code from various soon-to-be-split repos. One could then checkout the parent repo and the specific versions of the submodules, with the end game of tagging releases of the parent to specific commits/tags/branches of working submodules. Of course I haven't tested this extensively (just started looking into it recently), not even sure tagging of the parent is possible, though if it isn't it certainly diminishes any utility of 'git submodule'. And I'm not sure how maintainable that will be in the long run. >> Also, it looks like the testing environment should be hosted in a >> ready made VM image. I can create a CloudBioLinux flavour for that. >> Once we know what to install. I will also want tests for biolib_hpc, >> when that gets production ready (btw). It is a long standing wish on >> my end to get this sorted. > > Perhaps I have misunderstood you, but if all the build slaves > run the tests via the same VM image, there is no diversity. > For Biopython we deliberately want inhomogeneous setups > for our buildslaves to try and cover the variety of potential > setups our end users will have. > > Peter The way this is handled via perl is to either use smoke testing during the build (as Peter alluded to earlier) or to run tests during CPAN installation. Either one requires a little more overhead, but in re: to the latter we do get a nice spread of OS and perl versions to test against: http://matrix.cpantesters.org/?dist=BioPerl+1.6.901 (obviously from that link we need windows devs to help out, they seem to be scarce these days :) We don't have smoke set up for bioperl yet (this would allow testing the repo on a more regular basis instead of per-release), though installation of Smolder on testing.open-bio.org was mentioned at one point but fell through the cracks. chris From pjotr.public14 at thebird.nl Fri Sep 30 14:38:47 2011 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 30 Sep 2011 16:38:47 +0200 Subject: [BioRuby] BioRuby developers worldwide Message-ID: <20110930143847.GA8366@thebird.nl> As if it is not enough, Raoul has combined biogems with the openstreetmap: http://biogems.info/usersmap.html South-America and Russia are conspicuously absent... Pj. From chmille4 at gmail.com Fri Sep 30 14:42:05 2011 From: chmille4 at gmail.com (Chase Miller) Date: Fri, 30 Sep 2011 10:42:05 -0400 Subject: [BioRuby] BioRuby developers worldwide In-Reply-To: <20110930143847.GA8366@thebird.nl> References: <20110930143847.GA8366@thebird.nl> Message-ID: Nice work! Really cool. (From the boston guy) Chase On Fri, Sep 30, 2011 at 10:38 AM, Pjotr Prins wrote: > As if it is not enough, Raoul has combined biogems with the > openstreetmap: > > http://biogems.info/usersmap.html > > South-America and Russia are conspicuously absent... > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby > From hlapp at drycafe.net Fri Sep 30 15:05:50 2011 From: hlapp at drycafe.net (Hilmar Lapp) Date: Fri, 30 Sep 2011 11:05:50 -0400 Subject: [BioRuby] BioRuby developers worldwide In-Reply-To: <20110930143847.GA8366@thebird.nl> References: <20110930143847.GA8366@thebird.nl> Message-ID: Antarctica, too, actually, which I find really disappointing. Let alone no kiwis. -hilmar On Sep 30, 2011, at 10:38 AM, Pjotr Prins wrote: > As if it is not enough, Raoul has combined biogems with the > openstreetmap: > > http://biogems.info/usersmap.html > > South-America and Russia are conspicuously absent... > > Pj. > _______________________________________________ > BioRuby Project - http://www.bioruby.org/ > BioRuby mailing list > BioRuby at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/bioruby -- =========================================================== : Hilmar Lapp -:- Durham, NC -:- hlapp at drycafe dot net : ===========================================================