[Bioperl-l] bioperl reorganization

[Chris Fields]

On Jul 17, 2009, at 11:54 AM, Sendu Bala wrote:

> Chris Fields wrote:
>> On Jul 16, 2009, at 2:22 AM, Robert Buels wrote:
>>>> And finally, and I am saying this with the utmost respect and  
>>>> sincerest thanks for everything Sendu is doing and has done for  
>>>> BioPerl, but I'm not convinced we should keep using  
>>>> Bio::Root::Build. It does make some things convenient, but at the  
>>>> cost of additional bugs (2-3 at last count), some API breakage  
>>>> (some methods conflict with Module::Build), and a bit of a  
>>>> chicken-and-egg dilemma that particularly impacts  
>>>> subdistributions (attempting to fall back to Module::Build  
>>>> doesn't work due to API issues).
> [snip]
>> http://bugzilla.open-bio.org/show_bug.cgi?id=2792
>> http://bugzilla.open-bio.org/show_bug.cgi?id=2831
>> http://bugzilla.open-bio.org/show_bug.cgi?id=2859
>> http://bugzilla.open-bio.org/show_bug.cgi?id=2832 (this one is more  
>> a TODO)
>> Note that the author of Bio::Root::Build hasn't touched these, so  
>> my inclination is to convert over to plain ol' Module::Build.
>
> Well, it hardly had to be me that had to add CPANPLUS support. And  
> they're all P2 normal and minor bugs. And you never (iirc)  
> encouraged me to solve them to help out with your release. I did  
> offer to help (generally), but you never took me up on that offer.  
> But...

Just to note, most of issues with Bio::Root::Build popped up after the  
final core 1.6.0 was released and prior to the release of run/db/ 
network (how I found out about the inability to fall back to M::B).  
Regardless...

I'm not sure why I am the one that needs to approve fixing something  
in bioperl-live; I'm the 1.6.x release pumpkin, not the bioperl-live  
pumpkin. Core developers keep an eye on bioperl-live and svn code.  
This is just as much a bioperl-live bug as a 1.6.x bug (the bugs were  
filed fairly soon after the releases, IIRC). If it seriously breaks  
bioperl's API or can feasibly be merged into 1.6.x, I can cherry-pick  
around it.  I don't see fixes to make Bio::Root::Build compliant with  
M::B falling into the 'cherry-picking' category, but who knows?

Furthermore, again respectfully, but one shouldn't need encouragement  
to fix this.  Bio::Root::Build has very useful methods, but I didn't  
write that particular code, so I'm really not the best one to debug  
it. I can certainly try but I can't guarantee how long that will take  
for obvious reasons.

It's a nice way to assign priority but don't hang too much on the 'P2'  
designation, seeing as Bugzilla automatically defaults new bugs to P2  
status (it has to be changed to something else manually).  These are  
not blockers per se, but they are fairly serious bugs if they cause  
any installation issues.  The CPANPLUS ones are the most troubling  
(lots of CPAN Testers give an UNKNOWN status due to CPANPLUS issues,  
possibly due to bad META.yml).

>> On API and the 'chicken-or-egg' issue:
>> Several methods within Bio::Root::Build override Module::Build  
>> methods but break API, in that they accept, generate, or process  
>> different (sometimes bioperl-specific) data than what the same  
>> Module::Build methods expect.  I think 'requires' and 'recommends'  
>> fall into this cateory, as well as some meta data generation, such  
>> as META.yaml and PPM.  Other bits are more akin to syntactic sugar  
>> (automated installation via CPAN, network checking, etc).  This may  
>> cause bugs as noted above, which goes to demonstrate that too much  
>> 'sugar' can send you into a coma ;>
>
> ... You're right, it's a bit of a mess. For 1.5.2 I felt all the  
> extra stuff that made it easier to install was absolutely required.  
> And the ultimate purpose of Bio::Root::Build (as it's called now)  
> was to make installation easier for everyone. If it makes it harder,  
> and/or if the current maintainer thinks they can deal with any  
> support requests that arise from just using Module::Build directly,  
> then go ahead and do away with it.
>
> But while BioPerl is still monolithic, how will people be able to  
> choose which external dependencies they want to install? That's the  
> question that must be resolved before getting rid of  
> Bio::Root::Build. You'd also need to resolve the network tests  
> issue. And, well, I guess all the other issues that Bio::Root:Build  
> solves.

I mentioned two options.  The first was to revert back to  
Module::Build. The second was to have Bio::Root::Build methods comply  
with the Module::Build API.  It's pretty obvious which one I favor,  
but I also mention we can certainly follow the second option  
<emphasis>IF<\emphasis> sub-distributions have a way to fall back to  
Module::Build and install core, either if it's not present or isn't at  
the correct version.  Doing so requires Module::Build API compliance.

Fixing Bio::Root::Build towards that end would solve the network  
issue. Robert suggested another option as well (separate out those  
tests into another directory tree), though thinking about it more I  
don't think that's immediately tenable with all the network tests  
mixed in.

As for the external dependencies, we're falling into the trap of  
thinking general users need to install bioperl-live (and thus are  
using a tarball and 'perl Build.PL').  Everyday users should use CPAN  
(or PPM when we have that running); devs and advanced users can use  
bioperl-live.  A standard CPAN install should take care of most  
required dependencies; we should be able to push additional  
'dependencies' onto the required queue if the user wants them.

If needed we could keep the CPAN sugar to make it a bit easier, but i  
think it needs to be turned off by default and prompted for, and not  
run at all when in CPAN/CPANPLUS shell. Those should be easy enough to  
fix; the latter is a matter of looking for the env. vars  
$CPAN_IS_RUNNING and/or $CPANPLUS_IS_RUNNING (I believe those are  
correct).

>> It also causes a bit of a 'chicken-or-egg' issue with  
>> subdistributions wanting to use Bio::Root::Build, in that one has  
>> to check for the presence of Bio::Root::Build first and then  
>> completely bail if it isn't present.  One can't fall back to  
>> Module::Build due to the API difference.
>
> For small sub-distributions that have no optional external  
> dependencies (all of the BioPerl subdists?), they can be changed to  
> just using pure Module::Build, while core retains Bio::Root::Build  
> as long as core is monolithic.
>
> (For 1.5.2, the subdists each came bundled with ModuleBuildBioPerl,  
> so I didn't have this issue.)

The bugs mainly pertain to bp/M::DB API conflicts.  We could use  
either/or if the API was the same, but it would be nice to have some  
consistency and not have to choose between one or the other (or worse,  
change from one to the other if a dependency is added).

chris