[Open-bio-l] Fwd: [Utilities-announce] NCBI Revised E-utility Usage Policy

[Chris Fields]

On Mar 25, 2010, at 11:13 AM, Peter wrote:

> On Thu, Mar 25, 2010 at 3:43 PM, Chris Fields wrote:
> >
> > Let's play devil's advocate.  The best way I can think of to
> > describe this is to lay out a possible scenario.  Suppose an
> > end user (out of possibly thousands of end users, scattered
> > across many IPs) uses one of the eutils modules/classes where
> > the tool is set but the email isn't (our current status).
> > They set 'email' to their local one, and then proceed to
> > somehow abuse NCBI's rules and are blocked.  In order to be
> > reinstated, they will have to register both the tool and
> > email with NCBI.  Until then, does this block anyone else
> > with the same tool?  Just those from that IP?  Not clear at
> > the moment.
> 
> I had understood that so far the NCBI just blocked by IP.
> But only they know for sure, and it may change.

Yes.  And they have every right to change; they have been hammered by spam for years now, and the tool/email has always been implied to be required (thought really never enforced).  So it was only a matter of time before theydid something about it.

> > To proceed further, now the user registers the tool and
> > email (both need to be registered to unblock).  If I
> > understand the eutils documents correctly, as stated, only
> > one email (supposedly the software developers) is registered
> > per tool (also supposedly the software developers).  The
> > 'Bio*' tool name could end up being registered by anyone
> > (wittingly or unwittingly), using their own personal email.
> 
> Ah. That is possible - although I had been assuming the NCBI
> would be looking at the *combination* of tool+email, that
> may not be the case.
> 
> > If another user uses the same tool name with a different
> > email, would they be blocked?  If not, and that user tries
> > to register as above (after subsequent abuse), would a
> > conflict occur and they be notified of the prior
> > registration?  Again, it's not clear what happens.
> 
> Indeed.
> 
> > We have until probably sometime in May to decide a course
> > of action (June 1 is the enforcement date I believe), but
> > this relies on NCBI clarifying a few things first.  The
> > current documentation (at least to me) does seem to indicate
> > that each tool must have a single corresponding email when
> > registered.  Unless it is clarified, from my perspective
> > the only safe course that addresses all concerns is to
> > leave both tool and email unset, and then register a
> > respective toolkit/email to keep it within the specific
> > dev group as a safeguard.  That last bit is for many
> > reasons I've already outlined; an additional one is the
> > fact that we already have a default set for 'tool' now
> > (and have had one set for a while), so by legacy anyone
> > using older versions will have 'Bio*' preset already when
> > June 1 hits.
> 
> Biopython has also been setting a default tool value
> (with no default email) for some time.


Right.  Same with BioPerl, for many years now, with a few different names (some class-specific, others toolkit-based).  Kind of a mess, really.


> > BTW, I don't consider the above scenario out of the realm
> > of possibility, particularly if they truly intend on
> > enforcing the rules this time around.  We've had many
> > users who have asked the question 'how can I download
> > my batch of 1,00,000 records via eutils'.  Potential
> > lack of common sense doesn't stop the persistent or
> > the desperate.
> 
> Agreed - sooner or later someone will do something silly
> with the Bio* Entrez wrappers. Continuing to play Devil's
> advocate, let's suppose we had a default tool and email
> set. That *could* result in the NCBI blocking all users
> of that Bio* toolkit running with this default tool+email
> which would be a big problem (even if just for a day or
> two while talking to the NCBI). Therefore I don't think
> we should be setting a default tool+email (without
> clarification from the NCBI).

Yes, agreed.

> > Right, but my point is there are no intermediaries,
> > the news goes straight to the list.  We're not reliant
> > on (possibly busy, possibly absent) developers for
> > second-hand news.  We've been bitten by this before
> > many times with NCBI, both with eutils and BLAST
> > changes, a good many which NCBI announced but not
> > passed on to our mailing list.
> >
> > Saying this now, it makes me wonder whether we should
> > have a master list of some sort to gather such
> > announcements that may impact developers (eutils,
> > BLAST, GenBank/EMBL/UniProt releases, etc).
> 
> This is an excellent idea, but need not be linked to
> any default email used for Entrez.
> 
> Peter

Agreed.  Adding the tool's related email to the NCBI eutil mailing list is supposed to be optional, anyway.

chris