[Biopython] help with ncbiWWW

[pejvak.moghimi at york.ac.uk]

Thank you very much for clarifying it.

As a matter of fact, it has failed. Few times I just left it to run for about half-a-day to see what happens, and it finally came back with some errors, but I cannot remember what errors specifically. Though, I think they were to do with some sort of timeout. 

Cheers,
Pej.

Sent from Mail for Windows 10

From: Peter Cock
Sent: 26 July 2017 15:24
To: Pejvak Moghimi
Cc: Biopython Mailing List
Subject: Re: [Biopython] help with ncbiWWW

That does help, thank you.

First of all that tells me you are using Windows and your Python is
from Anaconda (probably not important here).

Now, I had been guessing the code was getting stuck while actually
connecting to the NCBI and waiting an update - which is where that
socket timeout would come into play.

I see now the problem is when Biopython checks for an update,
waits for a bit, checks for an update, waits for a bit, ... and never
gives up:

https://github.com/biopython/biopython/blob/biopython-170/Bio/Blast/NCBIWWW.py#L164

The code increases the wait interval to 120s (two minutes), but
currently has no (optional) maximum total waiting time. Adding
this as an option seems sensible (e.g. a maximum total waiting
time of say 5 or 10 mins).

Also, it would be good to check if the NCBI is returning some
clue or error message which our code does not understand...

>From your initial description is sounds like you have not found
any single example which fails - so this is going to be hard to
test.

Peter

On Wed, Jul 26, 2017 at 3:04 PM, Pejvak Moghimi
<pejvak.moghimi at york.ac.uk> wrote:
> Hi Peter,
>
> Here it is:
>
> Traceback (most recent call last):
>
>   File "<ipython-input-107-561cd74d2097>", line 1, in <module>
>     runfile('D:/Dropbox/Pejvak
> Moghimi/DMT_project/blast_for_clav_seqs/blastScript(altered).py',
> wdir='D:/Dropbox/Pejvak Moghimi/DMT_project/blast_for_clav_seqs')
>
>   File
> "C:\Users\pezhv\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py",
> line 880, in runfile
>     execfile(filename, namespace)
>
>   File
> "C:\Users\pezhv\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py",
> line 102, in execfile
>     exec(compile(f.read(), filename, 'exec'), namespace)
>
>   File "D:/Dropbox/Pejvak
> Moghimi/DMT_project/blast_for_clav_seqs/blastScript(altered).py", line 116,
> in <module>
>     result_handle = NCBIWWW.qblast("blastp", "nr", sequence,
> hitlist_size=500, entrez_query = orgn_specified)
>
>   File "C:\Users\pezhv\Anaconda3\lib\site-packages\Bio\Blast\NCBIWWW.py",
> line 164, in qblast
>     time.sleep(wait)
>
>
> Cheers,
> Pej.
>
>
> On 26 July 2017 at 14:57, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>
>> Hi Pej.
>>
>> Hmm. Maybe setting the timeout is not going to solve your
>> problem. I was hoping that would be a neat solution.
>>
>> Can you show us the stack trace when you had to stop a job
>> please?
>>
>> I assume you are using control+c to do this, in which case
>> Python ought to stop with the exception KeyboardInterrupt.
>> What I am interested in here is where in the code Python
>> is getting stuck. That would be a good clue.
>>
>> Peter
>>
>> On Wed, Jul 26, 2017 at 2:47 PM, Pejvak Moghimi
>> <pejvak.moghimi at york.ac.uk> wrote:
>> > Hi Peter,
>> >
>> > That solution, so far, does not seem to have worked nor with 10 neither
>> > with
>> > 30 second options.
>> >
>> > Cheers,
>> > Pej.
>> >
>> > On 26 July 2017 at 13:29, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>> >>
>> >> I am hoping that putting this near the start of your script will
>> >> apply the default timeout to all your BLAST calls (or other
>> >> network calls, e.g. NCBI Entrez):
>> >>
>> >> import socket
>> >> socket.setdefaulttimeout(30)  # timeout in seconds
>> >>
>> >> Peter
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.open-bio.org/pipermail/biopython/attachments/20170726/2b7cc2f5/attachment.html>