[Biopython] multiprocessing problem with pysam
Michal
mictadlo at gmail.com
Sun May 15 01:35:07 EDT 2011
Hello,
Thank you Brad. I have written the following new code:
import re
import os
import pysam
from pprint import pprint
from multiprocessing import Pool
class Test():
def __init__(self, bam_filename, cultivars):
self.__bam_fh = pysam.Samfile(bam_filename, "rb")
self.__cultivars = cultivars
def run(self, ref_name):
print os.getpid(), ref_name, self.__cultivars
return (os.getpid(), ref_name)
if __name__ == '__main__':
cultivars = 'Ja,Ea,As'.replace(' ', '').split(',')
bam_filename = "/media/usb/tests/test.bam"
bamfile = pysam.Samfile(bam_filename, "rb")
ref_names = bamfile.references
ref_lengths = bamfile.lengths
bamfile.close()
# for ref_name in ref_names:
# Test(bam_filename, cultivars).run(ref_names)
pool = Pool()
results = dict(pool.imap_unordered(
Test(bam_filename, cultivars).run, ref_names))
pool.close()
pool.join()
pprint(results)
and got the follwing error:
Exception in thread Thread-2:
Traceback (most recent call last):
File "/home/mictadlo/apps/python/lib/python2.7/threading.py", line
530, in __bootstrap_inner
self.run()
File "/home/mictadlo/apps/python/lib/python2.7/threading.py", line
483, in run
self.__target(*self.__args, **self.__kwargs)
File
"/home/mictadlo/apps/python/lib/python2.7/multiprocessing/pool.py", line
285, in _handle_tasks
put(task)
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup
__builtin__.instancemethod failed
I have search and found two possible solution for this problem:
* http://www.doughellmann.com/PyMOTW/multiprocessing/communication.html
* http://www.rueckstiess.net/research/snippets/show/ca1d7d90
However, is there a better way to solve it or the above solution are not
good?
Thank you in advance.
Michal
More information about the Biopython
mailing list