[Biopython] Blast DB keeps crashing nodes

[Willis, Jordan R]

Hello Biopython,

I was wondering if anyone has worked extensively with the Blast Database locally.

I am blasting millions of sequences using Biopython as my backend framework. I am using a high throughput computer cluster to blast each sequence. Rather than submit two million jobs, I have divided the fast files up into 50 or so.

The problem I am facing is a memory issue. I'm not sure, but I think that the Database is cacheing itself and not clearing before the next sequence is queried. In that regard, the next job calls upon the database again, and so on….

The memory builds up until it finally crashes the node. Has anyone dealt with this issue before?

Thanks,
Jordan