From cjfields at illinois.edu Sat Feb 6 18:35:50 2010 From: cjfields at illinois.edu (Chris Fields) Date: Sat, 6 Feb 2010 17:35:50 -0600 Subject: [BioLib-dev] Tests Message-ID: <61254C3F-E78E-492E-ABF0-BED72FAD74AE@illinois.edu> Pjotr, Sorry I didn't get back with biolib test results, my Ubuntu setup went down last week so I have been caught up with getting that back up and running (should be back up on Monday). I'll start running tests on OS X to see how far I get and will let you know; I'll start documenting my attempts on the biolib wiki. chris From cjfields at illinois.edu Mon Feb 8 11:40:13 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 8 Feb 2010 10:40:13 -0600 Subject: [BioLib-dev] Perl file pointers Message-ID: Pjotr, Just noticed something significant with the current perl bindings for io_lib, in that there appears to be no type map conversion for perl file handles (get a "Error: TypeError in method 'fread_reading', argument 1 of type 'FILE *'"). Files work fine as they are passed as simple strings to the proper functions so I/O is performed at the C level. I'm guessing the appropriate mapping needs to be done in the perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can get it working, but for now we'll need to assume it's not implemented. chris From aasmith at cs.wisc.edu Mon Feb 8 14:23:51 2010 From: aasmith at cs.wisc.edu (Adam A Smith) Date: Mon, 8 Feb 2010 13:23:51 -0600 (CST) Subject: [BioLib-dev] Perl file pointers In-Reply-To: References: Message-ID: > I'm guessing the appropriate mapping needs to be done in the > perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can > get it working, but for now we'll need to assume it's not implemented. I ran into some of that when I was trying to SWIGify everything this last summer. (I was beginning to look into it myself, when I got swept up in the rest of my life.) Let me know if you find a solution--I'm very curious for my own sake. Adam From cjfields at illinois.edu Mon Feb 8 14:30:05 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 08 Feb 2010 13:30:05 -0600 Subject: [BioLib-dev] Perl file pointers In-Reply-To: References: Message-ID: <1265657405.7644.26.camel@cjfields> On Mon, 2010-02-08 at 13:23 -0600, Adam A Smith wrote: > > I'm guessing the appropriate mapping needs to be done in the > > perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can > > get it working, but for now we'll need to assume it's not implemented. > > I ran into some of that when I was trying to SWIGify everything this last > summer. (I was beginning to look into it myself, when I got swept up in > the rest of my life.) Let me know if you find a solution--I'm very > curious for my own sake. > > Adam I think it's just calling in a specific typemap, something along the lines of (for perl): %typemap(in) FILE * { $1 = PerlIO_findFILE(IoIFP(sv_2io($input))); } Investigating it, perl uses a PerlIO * instead of a simple FILE *, so the above is necessary. Python/Ruby/etc may not need that. Regardless, I haven't managed to get that working, so I'm scrounging CPAN for a few SWIG-derived packages to see if there is anything there. chris From cjfields at illinois.edu Mon Feb 8 14:34:27 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 08 Feb 2010 13:34:27 -0600 Subject: [BioLib-dev] Ubuntu tests Message-ID: <1265657667.7644.35.camel@cjfields> Everything seems fine on Ubuntu: From cjfields at illinois.edu Mon Feb 8 14:44:27 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 08 Feb 2010 13:44:27 -0600 Subject: [BioLib-dev] Ubuntu tests Message-ID: <1265658267.7644.46.camel@cjfields> Everything passes tests and installs with standard INSTALL setup using latest github code (off master). cjfields at pyrimidine:~$ uname -a Linux pyrimidine 2.6.31-17-generic #54-Ubuntu SMP Thu Dec 10 17:01:44 UTC 2009 x86_64 GNU/Linux chris From pjotr.public14 at thebird.nl Tue Feb 9 08:10:59 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 9 Feb 2010 14:10:59 +0100 Subject: [BioLib-dev] Perl file pointers In-Reply-To: <1265657405.7644.26.camel@cjfields> References: <1265657405.7644.26.camel@cjfields> Message-ID: <20100209131059.GA787@thebird.nl> Hi Chris, Thanks for testing! I would think the FILE* can be used as an opaque SWIG pointer to an object - just like FILE * is handled normally. Unless you need to access the contents of the struct, or when you need to pass it into another Perl method (which may be the case, for example, with your Perl::IO factory). If you need it fast, I may be able to look into it tomorrow. There may be a hint in the SWIG PERL macros that come with SWIG itself (/usr/share/swig?). SWIG is not well documented in this area. The guy who did the Perl Math GSL mappings had quite a few tricks. See http://leto.net/code/Math-GSL/. I can look into that in April. It is one of those areas in SWIG... What may also help is read the generated C code (at least, if C does not send you screaming). It helps. Pj. On Mon, Feb 08, 2010 at 01:30:05PM -0600, Chris Fields wrote: > On Mon, 2010-02-08 at 13:23 -0600, Adam A Smith wrote: > > > I'm guessing the appropriate mapping needs to be done in the > > > perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can > > > get it working, but for now we'll need to assume it's not implemented. > > > > I ran into some of that when I was trying to SWIGify everything this last > > summer. (I was beginning to look into it myself, when I got swept up in > > the rest of my life.) Let me know if you find a solution--I'm very > > curious for my own sake. > > > > Adam > > I think it's just calling in a specific typemap, something along the > lines of (for perl): > > %typemap(in) FILE * { > $1 = PerlIO_findFILE(IoIFP(sv_2io($input))); > } > > Investigating it, perl uses a PerlIO * instead of a simple FILE *, so > the above is necessary. Python/Ruby/etc may not need that. Regardless, > I haven't managed to get that working, so I'm scrounging CPAN for a few > SWIG-derived packages to see if there is anything there. > > chris > > _______________________________________________ > BioLib-dev mailing list > BioLib-dev at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/biolib-dev From cjfields at illinois.edu Tue Feb 9 18:22:56 2010 From: cjfields at illinois.edu (Chris Fields) Date: Tue, 9 Feb 2010 17:22:56 -0600 Subject: [BioLib-dev] Perl file pointers In-Reply-To: <20100209131059.GA787@thebird.nl> References: <1265657405.7644.26.camel@cjfields> <20100209131059.GA787@thebird.nl> Message-ID: <799BD356-C61A-48B9-9451-78E44CD1DCE6@illinois.edu> On Feb 9, 2010, at 7:10 AM, Pjotr Prins wrote: > Hi Chris, > > Thanks for testing! I would think the FILE* can be used as an > opaque SWIG pointer to an object - just like FILE * is handled > normally. Unless you need to access the contents of the struct, or > when you need to pass it into another Perl method (which may be the > case, for example, with your Perl::IO factory). > > If you need it fast, I may be able to look into it tomorrow. > > There may be a hint in the SWIG PERL macros that come with SWIG itself > (/usr/share/swig?). SWIG is not well documented in this area. The guy > who did the Perl Math GSL mappings had quite a few tricks. See > http://leto.net/code/Math-GSL/. I can look into that in April. It is > one of those areas in SWIG... What may also help is read the generated C > code (at least, if C does not send you screaming). It helps. > > Pj. Forgot about Duke Leto's code, so will look at that as well as the SWIG macros. We basically need some way of creating the C file pointer and passing it in for files that have multiple records, where we want to iterate through the reads. The prior typemapping code I posted is fairly typical for SWIG perl mappings, but it doesn't seem to work yet. io_lib also has mfread_reading which accepts mFILE *, not sure how to handle that yet. chris From pjotr.public14 at thebird.nl Wed Feb 10 01:43:21 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 10 Feb 2010 07:43:21 +0100 Subject: [BioLib-dev] Perl file pointers In-Reply-To: <20100209131059.GA787@thebird.nl> References: <1265657405.7644.26.camel@cjfields> <20100209131059.GA787@thebird.nl> Message-ID: <20100210064321.GA6851@thebird.nl> I just pushed a change to the rc1 branch. You can requery now using the opaque pointer *mFILE. Merely needed to add a few include files - see the patch with Perl test. In the test I read a ztr file with a single sequence. Doing a second read returns the same sequence. I am not sure why, the C code is not that easy to take apart. Rather then the expected NULL on error, a structure gets returned contain the same information as the first read. But at least it runs. See how it runs with a different test set. Pj. From cjfields at illinois.edu Wed Feb 10 17:18:21 2010 From: cjfields at illinois.edu (Chris Fields) Date: Wed, 10 Feb 2010 16:18:21 -0600 Subject: [BioLib-dev] Test Errors on Ubuntu Message-ID: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> Pjotr, Just noticed an odd error on Ubuntu using the latest github; basically, everything builds on Ubuntu just fine, but there appear to be odd issues with shared libs. This doesn't affect the example or emboss, but affyio and (strangely) io_lib are affected. The library in question is libR.so; this pops up with failing tests for affyio and io_lib with 'make test'. Here are the relevant errors from the test scripts (bootstrapping problem): cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl test_staden_io_lib.pl Can't load '/usr/local/lib/perl/5.10.0/staden_io_lib.so' for module staden_io_lib: libR.so: cannot open shared object file: No such file or directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. at /usr/local/lib/perl/5.10.0/biolib/staden_io_lib.pm line 10 Compilation failed in require at test_staden_io_lib.pl line 1. BEGIN failed--compilation aborted at test_staden_io_lib.pl line 1. cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl test_affyio.pl Can't load '/usr/local/lib/perl/5.10.0/affyio.so' for module affyio: libR.so: cannot open shared object file: No such file or directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. at /usr/local/lib/perl/5.10.0/biolib/affyio.pm line 10 Compilation failed in require at test_affyio.pl line 4. BEGIN failed--compilation aborted at test_affyio.pl line 4. For some reason the absolute path is dropped. I understand this affecting affyio, but it's strange that io_lib is also affected. OS X doesn't seem to be affected, but I'll pass on the config and build output to you off-list, or I could put it up on Issues on github if you would like. chris From pjotr.public14 at thebird.nl Thu Feb 11 11:44:47 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 11 Feb 2010 17:44:47 +0100 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> Message-ID: <20100211164447.GA18902@thebird.nl> On Wed, Feb 10, 2010 at 04:18:21PM -0600, Chris Fields wrote: > Pjotr, > > Just noticed an odd error on Ubuntu using the latest github; basically, > everything builds on Ubuntu just fine, but there appear to be odd issues > with shared libs. This doesn't affect the example or emboss, but affyio > and (strangely) io_lib are affected. > > The library in question is libR.so; this pops up with failing tests for > affyio and io_lib with 'make test'. Here are the relevant errors from > the test scripts (bootstrapping problem): > > cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl > test_staden_io_lib.pl > Can't load '/usr/local/lib/perl/5.10.0/staden_io_lib.so' for module > staden_io_lib: libR.so: cannot open shared object file: No such file or > directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. > at /usr/local/lib/perl/5.10.0/biolib/staden_io_lib.pm line 10 > Compilation failed in require at test_staden_io_lib.pl line 1. > BEGIN failed--compilation aborted at test_staden_io_lib.pl line 1. > > cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl > test_affyio.pl > Can't load '/usr/local/lib/perl/5.10.0/affyio.so' for module affyio: > libR.so: cannot open shared object file: No such file or directory > at /usr/lib/perl/5.10/DynaLoader.pm line 196. > at /usr/local/lib/perl/5.10.0/biolib/affyio.pm line 10 > Compilation failed in require at test_affyio.pl line 4. > BEGIN failed--compilation aborted at test_affyio.pl line 4. > For some reason the absolute path is dropped. I understand this > affecting affyio, but it's strange that io_lib is also affected. OS X > doesn't seem to be affected, but There should be no dependency, per se. If I do ./configure --with-staden &> test.out there is no libR.so reference. It appears that because you also built affyio it finds libR.so and, from then on, add it in for all cases. This is not necessarily bad behaviour. The odd thing is that it gives this error while you apparently have libR.so somewhere on your system. The location /usr/local/lib/perl/5.10.0/affyio.so is also a bit puzzling. But that is what the Perl interpreter is telling the build system. After above configure I can do: make install [ 94%] Built target biolib_staden_io_lib-0.0.6 [100%] Built target staden_io_lib Install the project... -- Install configuration: "Release" -- Installing: /usr/lib/libbiolib_staden_io_lib-0.0.6.so -- Installing: /usr/local/lib/perl/5.10.1/biolib/staden_io_lib.pm -- Installing: /usr/local/lib/perl/5.10.1/staden_io_lib.so Running ldd shows no libR.so dependency: ldd /usr/local/lib/perl/5.10.1/staden_io_lib.so linux-gate.so.1 => (0xb7eff000) libbiolib_staden_io_lib-0.0.6.so => /usr/lib/libbiolib_staden_io_lib-0.0.6.so (0xb7e52000) libperl.so.5.10 => /usr/lib/libperl.so.5.10 (0xb7d01000) libz.so.1 => /usr/lib/libz.so.1 (0xb7ceb000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7bfd000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7bc3000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7bb6000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7a6f000) libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7a6a000) libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7a51000) libcrypt.so.1 => /lib/i686/cmov/libcrypt.so.1 (0xb7a1f000) /lib/ld-linux.so.2 (0xb7f00000) ldd /usr/lib/libbiolib_staden_io_lib-0.0.6.so linux-gate.so.1 => (0xb7fdb000) libz.so.1 => /usr/lib/libz.so.1 (0xb7f4d000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7df2000) /lib/ld-linux.so.2 (0xb7fdc000) Which is fine. When I do a full install there, indeed, appears a dependency on libR.so. But that is OK. It is simply a side-product of the full build with libR available. Can you replicate above? Do you have libR.so on your system? If so, does it show correctly in the build logs? If not, there is a real problem, because it should not link to a non-existing library. The R library is located in tools/cmake-support/modules/FindRLibs.cmake if finds it by querying R itself. It should not find it if it is not there. I suspect you have to tell Perl what the load path is for the libR.so. Is R in a standard location? Can you try updating the load path? Maybe it is a Ubuntu thing: http://blog.andrewbeacock.com/2007/10/how-to-add-shared-libraries-to-linuxs.html > I'll pass on the config and build output to you off-list, or I could put > it up on Issues on github if you would like. I prefer mail, normally. Pj. From cjfields at illinois.edu Thu Feb 11 23:53:11 2010 From: cjfields at illinois.edu (Chris Fields) Date: Thu, 11 Feb 2010 22:53:11 -0600 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: <20100211164447.GA18902@thebird.nl> References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> Message-ID: On Feb 11, 2010, at 10:44 AM, Pjotr Prins wrote: > On Wed, Feb 10, 2010 at 04:18:21PM -0600, Chris Fields wrote: >> Pjotr, >> >> Just noticed an odd error on Ubuntu using the latest github; basically, >> everything builds on Ubuntu just fine, but there appear to be odd issues >> with shared libs. This doesn't affect the example or emboss, but affyio >> and (strangely) io_lib are affected. >> >> The library in question is libR.so; this pops up with failing tests for >> affyio and io_lib with 'make test'. Here are the relevant errors from >> the test scripts (bootstrapping problem): >> >> cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl >> test_staden_io_lib.pl >> Can't load '/usr/local/lib/perl/5.10.0/staden_io_lib.so' for module >> staden_io_lib: libR.so: cannot open shared object file: No such file or >> directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. >> at /usr/local/lib/perl/5.10.0/biolib/staden_io_lib.pm line 10 >> Compilation failed in require at test_staden_io_lib.pl line 1. >> BEGIN failed--compilation aborted at test_staden_io_lib.pl line 1. >> >> cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl >> test_affyio.pl >> Can't load '/usr/local/lib/perl/5.10.0/affyio.so' for module affyio: >> libR.so: cannot open shared object file: No such file or directory >> at /usr/lib/perl/5.10/DynaLoader.pm line 196. >> at /usr/local/lib/perl/5.10.0/biolib/affyio.pm line 10 >> Compilation failed in require at test_affyio.pl line 4. >> BEGIN failed--compilation aborted at test_affyio.pl line 4. > >> For some reason the absolute path is dropped. I understand this >> affecting affyio, but it's strange that io_lib is also affected. OS X >> doesn't seem to be affected, but > > There should be no dependency, per se. If I do > > ./configure --with-staden &> test.out > > there is no libR.so reference. It appears that because you also built > affyio it finds libR.so and, from then on, add it in for all cases. > This is not necessarily bad behaviour. > > The odd thing is that it gives this error while you apparently have > libR.so somewhere on your system. > > The location /usr/local/lib/perl/5.10.0/affyio.so is also a bit > puzzling. But that is what the Perl interpreter is telling the build > system. After above configure I can do: > > make install > [ 94%] Built target biolib_staden_io_lib-0.0.6 > [100%] Built target staden_io_lib > Install the project... > -- Install configuration: "Release" > -- Installing: /usr/lib/libbiolib_staden_io_lib-0.0.6.so > -- Installing: /usr/local/lib/perl/5.10.1/biolib/staden_io_lib.pm > -- Installing: /usr/local/lib/perl/5.10.1/staden_io_lib.so > > Running ldd shows no libR.so dependency: > ... > > Which is fine. When I do a full install there, indeed, appears a > dependency on libR.so. But that is OK. It is simply a side-product of > the full build with libR available. > > Can you replicate above? Do you have libR.so on your system? If so, > does it show correctly in the build logs? If not, there is a real > problem, because it should not link to a non-existing library. > > The R library is located in > > tools/cmake-support/modules/FindRLibs.cmake > > if finds it by querying R itself. It should not find it if it is not > there. One of the issues is that it appears the perl test script preferentially uses any installed staden_io_lib.so over any locally built one, or is only finding the library after it is installed. The missing libR.so triggered this; deleting the aforementioned library and running './configure --with-staden' works (no more libR.so needed). We should probably set a 'use lib' directive in the script to preferentially catch local module builds. Re: having libR.so being called from staden_io_lib is still an odd issue (particularly since it disappears when configuring staden alone). It only occurs when compiling both staden_io_lib and affyio, the others passed w/o problems. > I suspect you have to tell Perl what the load path is for the > libR.so. Is R in a standard location? Can you try updating the load > path? Maybe it is a Ubuntu thing: > > http://blog.andrewbeacock.com/2007/10/how-to-add-shared-libraries-to-linuxs.html Setting LD_LIBRARY_PATH or adding to /etc/ld.so.conf.d fixes the problem. libR.so is in /usr/lib/R/lib, which apparently is not added to ld.so.conf.d via apt-get. >> I'll pass on the config and build output to you off-list, or I could put >> it up on Issues on github if you would like. > > I prefer mail, normally. > > Pj. I think this sorts out most pertinent issues. I'm working on a simple BioPerl-specific set of modules for io_lib that will eventually replace the XS-based code from bioperl-ext. This gets us along that path a bit. Probably something to note for the install process, however. chris From pjotr.public14 at thebird.nl Fri Feb 12 00:21:00 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 12 Feb 2010 06:21:00 +0100 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> Message-ID: <20100212052100.GA24708@thebird.nl> On Thu, Feb 11, 2010 at 10:53:11PM -0600, Chris Fields wrote: > One of the issues is that it appears the perl test script > preferentially uses any installed staden_io_lib.so over any locally > built one, or is only finding the library after it is installed. > The missing libR.so triggered this; deleting the aforementioned > library and running './configure --with-staden' works (no more > libR.so needed). We should probably set a 'use lib' directive in > the script to preferentially catch local module builds. Good. The problem is that we can not distinguish between installed libs and later locally built ones. I can try and find a way for the build system to use the local ones in the tree first, but that may defeat the purpose - i.e. testing an installed system. Now, what would be a good policy? Also the naming. You can see I use versioning on the Clibs, but not on the Perl module - as it finds itself by its name. Maybe we can use different naming for the development tree - so the tests would require 'staden_iolib-testing.pm'. I could copies at build time which would not interfere with the main setup. I am not sure about all the implications though. But maybe it is better to specifically used LD_LIBRARY_PATH instead. Leave overrides to the user. That gives more predictable behaviour. Feel free to try some stuff and tell me what works for you. > Setting LD_LIBRARY_PATH or adding to /etc/ld.so.conf.d fixes the > problem. libR.so is in /usr/lib/R/lib, which apparently is not added > to ld.so.conf.d via apt-get. That is a bug, in my book. Debian does not do this. I suspect the Ubuntu guys figure that only R uses that lib, and therefore knows where to find it. > I think this sorts out most pertinent issues. I'm working on a > simple BioPerl-specific set of modules for io_lib that will > eventually replace the XS-based code from bioperl-ext. This gets us > along that path a bit. Probably something to note for the install > process, however. Cool :-). Once that is done we can find a way to do that automatically for other modules/libraries. BTW I am half way done with automated documentation. We should be able to generate POD files from SWIG fairly soon. Pj. From cjfields at illinois.edu Fri Feb 12 09:34:14 2010 From: cjfields at illinois.edu (Chris Fields) Date: Fri, 12 Feb 2010 08:34:14 -0600 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: <20100212052100.GA24708@thebird.nl> References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> <20100212052100.GA24708@thebird.nl> Message-ID: On Feb 11, 2010, at 11:21 PM, Pjotr Prins wrote: > On Thu, Feb 11, 2010 at 10:53:11PM -0600, Chris Fields wrote: >> One of the issues is that it appears the perl test script >> preferentially uses any installed staden_io_lib.so over any locally >> built one, or is only finding the library after it is installed. >> The missing libR.so triggered this; deleting the aforementioned >> library and running './configure --with-staden' works (no more >> libR.so needed). We should probably set a 'use lib' directive in >> the script to preferentially catch local module builds. > > Good. The problem is that we can not distinguish between installed > libs and later locally built ones. I can try and find a way for the > build system to use the local ones in the tree first, but that may > defeat the purpose - i.e. testing an installed system. > > Now, what would be a good policy? Well, it looks as if the other test scripts (emboss, example) do essentially what I described, adding the local directory first to @INC for perl: BEGIN{unshift(@INC, "../emboss")}; use biolib::emboss; I can try seeing if that helps. > Also the naming. You can see I use versioning on the Clibs, but not > on the Perl module - as it finds itself by its name. > > Maybe we can use different naming for the development tree - so the > tests would require 'staden_iolib-testing.pm'. I could copies at > build time which would not interfere with the main setup. I am not > sure about all the implications though. > > But maybe it is better to specifically used LD_LIBRARY_PATH instead. > Leave overrides to the user. That gives more predictable behaviour. > > Feel free to try some stuff and tell me what works for you. > >> Setting LD_LIBRARY_PATH or adding to /etc/ld.so.conf.d fixes the >> problem. libR.so is in /usr/lib/R/lib, which apparently is not added >> to ld.so.conf.d via apt-get. > > That is a bug, in my book. Debian does not do this. I suspect the > Ubuntu guys figure that only R uses that lib, and therefore knows > where to find it. Same thought here as well. I'm using the R project's build, though; there have been reported problems with Canonical's R packages. Regardless, it shouldn't be that hard to add the lib to ldconfig during installation. >> I think this sorts out most pertinent issues. I'm working on a >> simple BioPerl-specific set of modules for io_lib that will >> eventually replace the XS-based code from bioperl-ext. This gets us >> along that path a bit. Probably something to note for the install >> process, however. > > Cool :-). Once that is done we can find a way to do that > automatically for other modules/libraries. > > BTW I am half way done with automated documentation. We should be > able to generate POD files from SWIG fairly soon. > > Pj. Very nice! chris From pjotr.public14 at thebird.nl Sat Feb 13 00:58:32 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 13 Feb 2010 06:58:32 +0100 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> <20100212052100.GA24708@thebird.nl> Message-ID: <20100213055832.GA803@thebird.nl> On Fri, Feb 12, 2010 at 08:34:14AM -0600, Chris Fields wrote: > Well, it looks as if the other test scripts (emboss, example) do essentially what I described, adding the local directory first to @INC for perl: > > BEGIN{unshift(@INC, "../emboss")}; > use biolib::emboss; > > I can try seeing if that helps. Yes, that would be best. Explicit is good. With Ruby I usually use the -I switch to set the path. > Same thought here as well. I'm using the R project's build, though; > there have been reported problems with Canonical's R packages. > Regardless, it shouldn't be that hard to add the lib to ldconfig > during installation. Agreed. Possibly through the environment. Pj. From cjfields at illinois.edu Sat Feb 6 23:35:50 2010 From: cjfields at illinois.edu (Chris Fields) Date: Sat, 6 Feb 2010 17:35:50 -0600 Subject: [BioLib-dev] Tests Message-ID: <61254C3F-E78E-492E-ABF0-BED72FAD74AE@illinois.edu> Pjotr, Sorry I didn't get back with biolib test results, my Ubuntu setup went down last week so I have been caught up with getting that back up and running (should be back up on Monday). I'll start running tests on OS X to see how far I get and will let you know; I'll start documenting my attempts on the biolib wiki. chris From cjfields at illinois.edu Mon Feb 8 16:40:13 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 8 Feb 2010 10:40:13 -0600 Subject: [BioLib-dev] Perl file pointers Message-ID: Pjotr, Just noticed something significant with the current perl bindings for io_lib, in that there appears to be no type map conversion for perl file handles (get a "Error: TypeError in method 'fread_reading', argument 1 of type 'FILE *'"). Files work fine as they are passed as simple strings to the proper functions so I/O is performed at the C level. I'm guessing the appropriate mapping needs to be done in the perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can get it working, but for now we'll need to assume it's not implemented. chris From aasmith at cs.wisc.edu Mon Feb 8 19:23:51 2010 From: aasmith at cs.wisc.edu (Adam A Smith) Date: Mon, 8 Feb 2010 13:23:51 -0600 (CST) Subject: [BioLib-dev] Perl file pointers In-Reply-To: References: Message-ID: > I'm guessing the appropriate mapping needs to be done in the > perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can > get it working, but for now we'll need to assume it's not implemented. I ran into some of that when I was trying to SWIGify everything this last summer. (I was beginning to look into it myself, when I got swept up in the rest of my life.) Let me know if you find a solution--I'm very curious for my own sake. Adam From cjfields at illinois.edu Mon Feb 8 19:30:05 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 08 Feb 2010 13:30:05 -0600 Subject: [BioLib-dev] Perl file pointers In-Reply-To: References: Message-ID: <1265657405.7644.26.camel@cjfields> On Mon, 2010-02-08 at 13:23 -0600, Adam A Smith wrote: > > I'm guessing the appropriate mapping needs to be done in the > > perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can > > get it working, but for now we'll need to assume it's not implemented. > > I ran into some of that when I was trying to SWIGify everything this last > summer. (I was beginning to look into it myself, when I got swept up in > the rest of my life.) Let me know if you find a solution--I'm very > curious for my own sake. > > Adam I think it's just calling in a specific typemap, something along the lines of (for perl): %typemap(in) FILE * { $1 = PerlIO_findFILE(IoIFP(sv_2io($input))); } Investigating it, perl uses a PerlIO * instead of a simple FILE *, so the above is necessary. Python/Ruby/etc may not need that. Regardless, I haven't managed to get that working, so I'm scrounging CPAN for a few SWIG-derived packages to see if there is anything there. chris From cjfields at illinois.edu Mon Feb 8 19:34:27 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 08 Feb 2010 13:34:27 -0600 Subject: [BioLib-dev] Ubuntu tests Message-ID: <1265657667.7644.35.camel@cjfields> Everything seems fine on Ubuntu: From cjfields at illinois.edu Mon Feb 8 19:44:27 2010 From: cjfields at illinois.edu (Chris Fields) Date: Mon, 08 Feb 2010 13:44:27 -0600 Subject: [BioLib-dev] Ubuntu tests Message-ID: <1265658267.7644.46.camel@cjfields> Everything passes tests and installs with standard INSTALL setup using latest github code (off master). cjfields at pyrimidine:~$ uname -a Linux pyrimidine 2.6.31-17-generic #54-Ubuntu SMP Thu Dec 10 17:01:44 UTC 2009 x86_64 GNU/Linux chris From pjotr.public14 at thebird.nl Tue Feb 9 13:10:59 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Tue, 9 Feb 2010 14:10:59 +0100 Subject: [BioLib-dev] Perl file pointers In-Reply-To: <1265657405.7644.26.camel@cjfields> References: <1265657405.7644.26.camel@cjfields> Message-ID: <20100209131059.GA787@thebird.nl> Hi Chris, Thanks for testing! I would think the FILE* can be used as an opaque SWIG pointer to an object - just like FILE * is handled normally. Unless you need to access the contents of the struct, or when you need to pass it into another Perl method (which may be the case, for example, with your Perl::IO factory). If you need it fast, I may be able to look into it tomorrow. There may be a hint in the SWIG PERL macros that come with SWIG itself (/usr/share/swig?). SWIG is not well documented in this area. The guy who did the Perl Math GSL mappings had quite a few tricks. See http://leto.net/code/Math-GSL/. I can look into that in April. It is one of those areas in SWIG... What may also help is read the generated C code (at least, if C does not send you screaming). It helps. Pj. On Mon, Feb 08, 2010 at 01:30:05PM -0600, Chris Fields wrote: > On Mon, 2010-02-08 at 13:23 -0600, Adam A Smith wrote: > > > I'm guessing the appropriate mapping needs to be done in the > > > perl-specific SWIG files in src/mappings/swig/perl. I'll see if I can > > > get it working, but for now we'll need to assume it's not implemented. > > > > I ran into some of that when I was trying to SWIGify everything this last > > summer. (I was beginning to look into it myself, when I got swept up in > > the rest of my life.) Let me know if you find a solution--I'm very > > curious for my own sake. > > > > Adam > > I think it's just calling in a specific typemap, something along the > lines of (for perl): > > %typemap(in) FILE * { > $1 = PerlIO_findFILE(IoIFP(sv_2io($input))); > } > > Investigating it, perl uses a PerlIO * instead of a simple FILE *, so > the above is necessary. Python/Ruby/etc may not need that. Regardless, > I haven't managed to get that working, so I'm scrounging CPAN for a few > SWIG-derived packages to see if there is anything there. > > chris > > _______________________________________________ > BioLib-dev mailing list > BioLib-dev at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/biolib-dev From cjfields at illinois.edu Tue Feb 9 23:22:56 2010 From: cjfields at illinois.edu (Chris Fields) Date: Tue, 9 Feb 2010 17:22:56 -0600 Subject: [BioLib-dev] Perl file pointers In-Reply-To: <20100209131059.GA787@thebird.nl> References: <1265657405.7644.26.camel@cjfields> <20100209131059.GA787@thebird.nl> Message-ID: <799BD356-C61A-48B9-9451-78E44CD1DCE6@illinois.edu> On Feb 9, 2010, at 7:10 AM, Pjotr Prins wrote: > Hi Chris, > > Thanks for testing! I would think the FILE* can be used as an > opaque SWIG pointer to an object - just like FILE * is handled > normally. Unless you need to access the contents of the struct, or > when you need to pass it into another Perl method (which may be the > case, for example, with your Perl::IO factory). > > If you need it fast, I may be able to look into it tomorrow. > > There may be a hint in the SWIG PERL macros that come with SWIG itself > (/usr/share/swig?). SWIG is not well documented in this area. The guy > who did the Perl Math GSL mappings had quite a few tricks. See > http://leto.net/code/Math-GSL/. I can look into that in April. It is > one of those areas in SWIG... What may also help is read the generated C > code (at least, if C does not send you screaming). It helps. > > Pj. Forgot about Duke Leto's code, so will look at that as well as the SWIG macros. We basically need some way of creating the C file pointer and passing it in for files that have multiple records, where we want to iterate through the reads. The prior typemapping code I posted is fairly typical for SWIG perl mappings, but it doesn't seem to work yet. io_lib also has mfread_reading which accepts mFILE *, not sure how to handle that yet. chris From pjotr.public14 at thebird.nl Wed Feb 10 06:43:21 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Wed, 10 Feb 2010 07:43:21 +0100 Subject: [BioLib-dev] Perl file pointers In-Reply-To: <20100209131059.GA787@thebird.nl> References: <1265657405.7644.26.camel@cjfields> <20100209131059.GA787@thebird.nl> Message-ID: <20100210064321.GA6851@thebird.nl> I just pushed a change to the rc1 branch. You can requery now using the opaque pointer *mFILE. Merely needed to add a few include files - see the patch with Perl test. In the test I read a ztr file with a single sequence. Doing a second read returns the same sequence. I am not sure why, the C code is not that easy to take apart. Rather then the expected NULL on error, a structure gets returned contain the same information as the first read. But at least it runs. See how it runs with a different test set. Pj. From cjfields at illinois.edu Wed Feb 10 22:18:21 2010 From: cjfields at illinois.edu (Chris Fields) Date: Wed, 10 Feb 2010 16:18:21 -0600 Subject: [BioLib-dev] Test Errors on Ubuntu Message-ID: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> Pjotr, Just noticed an odd error on Ubuntu using the latest github; basically, everything builds on Ubuntu just fine, but there appear to be odd issues with shared libs. This doesn't affect the example or emboss, but affyio and (strangely) io_lib are affected. The library in question is libR.so; this pops up with failing tests for affyio and io_lib with 'make test'. Here are the relevant errors from the test scripts (bootstrapping problem): cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl test_staden_io_lib.pl Can't load '/usr/local/lib/perl/5.10.0/staden_io_lib.so' for module staden_io_lib: libR.so: cannot open shared object file: No such file or directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. at /usr/local/lib/perl/5.10.0/biolib/staden_io_lib.pm line 10 Compilation failed in require at test_staden_io_lib.pl line 1. BEGIN failed--compilation aborted at test_staden_io_lib.pl line 1. cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl test_affyio.pl Can't load '/usr/local/lib/perl/5.10.0/affyio.so' for module affyio: libR.so: cannot open shared object file: No such file or directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. at /usr/local/lib/perl/5.10.0/biolib/affyio.pm line 10 Compilation failed in require at test_affyio.pl line 4. BEGIN failed--compilation aborted at test_affyio.pl line 4. For some reason the absolute path is dropped. I understand this affecting affyio, but it's strange that io_lib is also affected. OS X doesn't seem to be affected, but I'll pass on the config and build output to you off-list, or I could put it up on Issues on github if you would like. chris From pjotr.public14 at thebird.nl Thu Feb 11 16:44:47 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Thu, 11 Feb 2010 17:44:47 +0100 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> Message-ID: <20100211164447.GA18902@thebird.nl> On Wed, Feb 10, 2010 at 04:18:21PM -0600, Chris Fields wrote: > Pjotr, > > Just noticed an odd error on Ubuntu using the latest github; basically, > everything builds on Ubuntu just fine, but there appear to be odd issues > with shared libs. This doesn't affect the example or emboss, but affyio > and (strangely) io_lib are affected. > > The library in question is libR.so; this pops up with failing tests for > affyio and io_lib with 'make test'. Here are the relevant errors from > the test scripts (bootstrapping problem): > > cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl > test_staden_io_lib.pl > Can't load '/usr/local/lib/perl/5.10.0/staden_io_lib.so' for module > staden_io_lib: libR.so: cannot open shared object file: No such file or > directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. > at /usr/local/lib/perl/5.10.0/biolib/staden_io_lib.pm line 10 > Compilation failed in require at test_staden_io_lib.pl line 1. > BEGIN failed--compilation aborted at test_staden_io_lib.pl line 1. > > cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl > test_affyio.pl > Can't load '/usr/local/lib/perl/5.10.0/affyio.so' for module affyio: > libR.so: cannot open shared object file: No such file or directory > at /usr/lib/perl/5.10/DynaLoader.pm line 196. > at /usr/local/lib/perl/5.10.0/biolib/affyio.pm line 10 > Compilation failed in require at test_affyio.pl line 4. > BEGIN failed--compilation aborted at test_affyio.pl line 4. > For some reason the absolute path is dropped. I understand this > affecting affyio, but it's strange that io_lib is also affected. OS X > doesn't seem to be affected, but There should be no dependency, per se. If I do ./configure --with-staden &> test.out there is no libR.so reference. It appears that because you also built affyio it finds libR.so and, from then on, add it in for all cases. This is not necessarily bad behaviour. The odd thing is that it gives this error while you apparently have libR.so somewhere on your system. The location /usr/local/lib/perl/5.10.0/affyio.so is also a bit puzzling. But that is what the Perl interpreter is telling the build system. After above configure I can do: make install [ 94%] Built target biolib_staden_io_lib-0.0.6 [100%] Built target staden_io_lib Install the project... -- Install configuration: "Release" -- Installing: /usr/lib/libbiolib_staden_io_lib-0.0.6.so -- Installing: /usr/local/lib/perl/5.10.1/biolib/staden_io_lib.pm -- Installing: /usr/local/lib/perl/5.10.1/staden_io_lib.so Running ldd shows no libR.so dependency: ldd /usr/local/lib/perl/5.10.1/staden_io_lib.so linux-gate.so.1 => (0xb7eff000) libbiolib_staden_io_lib-0.0.6.so => /usr/lib/libbiolib_staden_io_lib-0.0.6.so (0xb7e52000) libperl.so.5.10 => /usr/lib/libperl.so.5.10 (0xb7d01000) libz.so.1 => /usr/lib/libz.so.1 (0xb7ceb000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7bfd000) libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7bc3000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7bb6000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7a6f000) libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7a6a000) libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7a51000) libcrypt.so.1 => /lib/i686/cmov/libcrypt.so.1 (0xb7a1f000) /lib/ld-linux.so.2 (0xb7f00000) ldd /usr/lib/libbiolib_staden_io_lib-0.0.6.so linux-gate.so.1 => (0xb7fdb000) libz.so.1 => /usr/lib/libz.so.1 (0xb7f4d000) libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7df2000) /lib/ld-linux.so.2 (0xb7fdc000) Which is fine. When I do a full install there, indeed, appears a dependency on libR.so. But that is OK. It is simply a side-product of the full build with libR available. Can you replicate above? Do you have libR.so on your system? If so, does it show correctly in the build logs? If not, there is a real problem, because it should not link to a non-existing library. The R library is located in tools/cmake-support/modules/FindRLibs.cmake if finds it by querying R itself. It should not find it if it is not there. I suspect you have to tell Perl what the load path is for the libR.so. Is R in a standard location? Can you try updating the load path? Maybe it is a Ubuntu thing: http://blog.andrewbeacock.com/2007/10/how-to-add-shared-libraries-to-linuxs.html > I'll pass on the config and build output to you off-list, or I could put > it up on Issues on github if you would like. I prefer mail, normally. Pj. From cjfields at illinois.edu Fri Feb 12 04:53:11 2010 From: cjfields at illinois.edu (Chris Fields) Date: Thu, 11 Feb 2010 22:53:11 -0600 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: <20100211164447.GA18902@thebird.nl> References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> Message-ID: On Feb 11, 2010, at 10:44 AM, Pjotr Prins wrote: > On Wed, Feb 10, 2010 at 04:18:21PM -0600, Chris Fields wrote: >> Pjotr, >> >> Just noticed an odd error on Ubuntu using the latest github; basically, >> everything builds on Ubuntu just fine, but there appear to be odd issues >> with shared libs. This doesn't affect the example or emboss, but affyio >> and (strangely) io_lib are affected. >> >> The library in question is libR.so; this pops up with failing tests for >> affyio and io_lib with 'make test'. Here are the relevant errors from >> the test scripts (bootstrapping problem): >> >> cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl >> test_staden_io_lib.pl >> Can't load '/usr/local/lib/perl/5.10.0/staden_io_lib.so' for module >> staden_io_lib: libR.so: cannot open shared object file: No such file or >> directory at /usr/lib/perl/5.10/DynaLoader.pm line 196. >> at /usr/local/lib/perl/5.10.0/biolib/staden_io_lib.pm line 10 >> Compilation failed in require at test_staden_io_lib.pl line 1. >> BEGIN failed--compilation aborted at test_staden_io_lib.pl line 1. >> >> cjfields at pyrimidine:~/src/biolib/src/mappings/swig/perl/test$ perl >> test_affyio.pl >> Can't load '/usr/local/lib/perl/5.10.0/affyio.so' for module affyio: >> libR.so: cannot open shared object file: No such file or directory >> at /usr/lib/perl/5.10/DynaLoader.pm line 196. >> at /usr/local/lib/perl/5.10.0/biolib/affyio.pm line 10 >> Compilation failed in require at test_affyio.pl line 4. >> BEGIN failed--compilation aborted at test_affyio.pl line 4. > >> For some reason the absolute path is dropped. I understand this >> affecting affyio, but it's strange that io_lib is also affected. OS X >> doesn't seem to be affected, but > > There should be no dependency, per se. If I do > > ./configure --with-staden &> test.out > > there is no libR.so reference. It appears that because you also built > affyio it finds libR.so and, from then on, add it in for all cases. > This is not necessarily bad behaviour. > > The odd thing is that it gives this error while you apparently have > libR.so somewhere on your system. > > The location /usr/local/lib/perl/5.10.0/affyio.so is also a bit > puzzling. But that is what the Perl interpreter is telling the build > system. After above configure I can do: > > make install > [ 94%] Built target biolib_staden_io_lib-0.0.6 > [100%] Built target staden_io_lib > Install the project... > -- Install configuration: "Release" > -- Installing: /usr/lib/libbiolib_staden_io_lib-0.0.6.so > -- Installing: /usr/local/lib/perl/5.10.1/biolib/staden_io_lib.pm > -- Installing: /usr/local/lib/perl/5.10.1/staden_io_lib.so > > Running ldd shows no libR.so dependency: > ... > > Which is fine. When I do a full install there, indeed, appears a > dependency on libR.so. But that is OK. It is simply a side-product of > the full build with libR available. > > Can you replicate above? Do you have libR.so on your system? If so, > does it show correctly in the build logs? If not, there is a real > problem, because it should not link to a non-existing library. > > The R library is located in > > tools/cmake-support/modules/FindRLibs.cmake > > if finds it by querying R itself. It should not find it if it is not > there. One of the issues is that it appears the perl test script preferentially uses any installed staden_io_lib.so over any locally built one, or is only finding the library after it is installed. The missing libR.so triggered this; deleting the aforementioned library and running './configure --with-staden' works (no more libR.so needed). We should probably set a 'use lib' directive in the script to preferentially catch local module builds. Re: having libR.so being called from staden_io_lib is still an odd issue (particularly since it disappears when configuring staden alone). It only occurs when compiling both staden_io_lib and affyio, the others passed w/o problems. > I suspect you have to tell Perl what the load path is for the > libR.so. Is R in a standard location? Can you try updating the load > path? Maybe it is a Ubuntu thing: > > http://blog.andrewbeacock.com/2007/10/how-to-add-shared-libraries-to-linuxs.html Setting LD_LIBRARY_PATH or adding to /etc/ld.so.conf.d fixes the problem. libR.so is in /usr/lib/R/lib, which apparently is not added to ld.so.conf.d via apt-get. >> I'll pass on the config and build output to you off-list, or I could put >> it up on Issues on github if you would like. > > I prefer mail, normally. > > Pj. I think this sorts out most pertinent issues. I'm working on a simple BioPerl-specific set of modules for io_lib that will eventually replace the XS-based code from bioperl-ext. This gets us along that path a bit. Probably something to note for the install process, however. chris From pjotr.public14 at thebird.nl Fri Feb 12 05:21:00 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Fri, 12 Feb 2010 06:21:00 +0100 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> Message-ID: <20100212052100.GA24708@thebird.nl> On Thu, Feb 11, 2010 at 10:53:11PM -0600, Chris Fields wrote: > One of the issues is that it appears the perl test script > preferentially uses any installed staden_io_lib.so over any locally > built one, or is only finding the library after it is installed. > The missing libR.so triggered this; deleting the aforementioned > library and running './configure --with-staden' works (no more > libR.so needed). We should probably set a 'use lib' directive in > the script to preferentially catch local module builds. Good. The problem is that we can not distinguish between installed libs and later locally built ones. I can try and find a way for the build system to use the local ones in the tree first, but that may defeat the purpose - i.e. testing an installed system. Now, what would be a good policy? Also the naming. You can see I use versioning on the Clibs, but not on the Perl module - as it finds itself by its name. Maybe we can use different naming for the development tree - so the tests would require 'staden_iolib-testing.pm'. I could copies at build time which would not interfere with the main setup. I am not sure about all the implications though. But maybe it is better to specifically used LD_LIBRARY_PATH instead. Leave overrides to the user. That gives more predictable behaviour. Feel free to try some stuff and tell me what works for you. > Setting LD_LIBRARY_PATH or adding to /etc/ld.so.conf.d fixes the > problem. libR.so is in /usr/lib/R/lib, which apparently is not added > to ld.so.conf.d via apt-get. That is a bug, in my book. Debian does not do this. I suspect the Ubuntu guys figure that only R uses that lib, and therefore knows where to find it. > I think this sorts out most pertinent issues. I'm working on a > simple BioPerl-specific set of modules for io_lib that will > eventually replace the XS-based code from bioperl-ext. This gets us > along that path a bit. Probably something to note for the install > process, however. Cool :-). Once that is done we can find a way to do that automatically for other modules/libraries. BTW I am half way done with automated documentation. We should be able to generate POD files from SWIG fairly soon. Pj. From cjfields at illinois.edu Fri Feb 12 14:34:14 2010 From: cjfields at illinois.edu (Chris Fields) Date: Fri, 12 Feb 2010 08:34:14 -0600 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: <20100212052100.GA24708@thebird.nl> References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> <20100212052100.GA24708@thebird.nl> Message-ID: On Feb 11, 2010, at 11:21 PM, Pjotr Prins wrote: > On Thu, Feb 11, 2010 at 10:53:11PM -0600, Chris Fields wrote: >> One of the issues is that it appears the perl test script >> preferentially uses any installed staden_io_lib.so over any locally >> built one, or is only finding the library after it is installed. >> The missing libR.so triggered this; deleting the aforementioned >> library and running './configure --with-staden' works (no more >> libR.so needed). We should probably set a 'use lib' directive in >> the script to preferentially catch local module builds. > > Good. The problem is that we can not distinguish between installed > libs and later locally built ones. I can try and find a way for the > build system to use the local ones in the tree first, but that may > defeat the purpose - i.e. testing an installed system. > > Now, what would be a good policy? Well, it looks as if the other test scripts (emboss, example) do essentially what I described, adding the local directory first to @INC for perl: BEGIN{unshift(@INC, "../emboss")}; use biolib::emboss; I can try seeing if that helps. > Also the naming. You can see I use versioning on the Clibs, but not > on the Perl module - as it finds itself by its name. > > Maybe we can use different naming for the development tree - so the > tests would require 'staden_iolib-testing.pm'. I could copies at > build time which would not interfere with the main setup. I am not > sure about all the implications though. > > But maybe it is better to specifically used LD_LIBRARY_PATH instead. > Leave overrides to the user. That gives more predictable behaviour. > > Feel free to try some stuff and tell me what works for you. > >> Setting LD_LIBRARY_PATH or adding to /etc/ld.so.conf.d fixes the >> problem. libR.so is in /usr/lib/R/lib, which apparently is not added >> to ld.so.conf.d via apt-get. > > That is a bug, in my book. Debian does not do this. I suspect the > Ubuntu guys figure that only R uses that lib, and therefore knows > where to find it. Same thought here as well. I'm using the R project's build, though; there have been reported problems with Canonical's R packages. Regardless, it shouldn't be that hard to add the lib to ldconfig during installation. >> I think this sorts out most pertinent issues. I'm working on a >> simple BioPerl-specific set of modules for io_lib that will >> eventually replace the XS-based code from bioperl-ext. This gets us >> along that path a bit. Probably something to note for the install >> process, however. > > Cool :-). Once that is done we can find a way to do that > automatically for other modules/libraries. > > BTW I am half way done with automated documentation. We should be > able to generate POD files from SWIG fairly soon. > > Pj. Very nice! chris From pjotr.public14 at thebird.nl Sat Feb 13 05:58:32 2010 From: pjotr.public14 at thebird.nl (Pjotr Prins) Date: Sat, 13 Feb 2010 06:58:32 +0100 Subject: [BioLib-dev] Test Errors on Ubuntu In-Reply-To: References: <1265840301.26211.10.camel@pyrimidine.igb.uiuc.edu> <20100211164447.GA18902@thebird.nl> <20100212052100.GA24708@thebird.nl> Message-ID: <20100213055832.GA803@thebird.nl> On Fri, Feb 12, 2010 at 08:34:14AM -0600, Chris Fields wrote: > Well, it looks as if the other test scripts (emboss, example) do essentially what I described, adding the local directory first to @INC for perl: > > BEGIN{unshift(@INC, "../emboss")}; > use biolib::emboss; > > I can try seeing if that helps. Yes, that would be best. Explicit is good. With Ruby I usually use the -I switch to set the path. > Same thought here as well. I'm using the R project's build, though; > there have been reported problems with Canonical's R packages. > Regardless, it shouldn't be that hard to add the lib to ldconfig > during installation. Agreed. Possibly through the environment. Pj.