Ersetzen Gurke mit Dill und Pymongo Anruf

Question

Ich endlich verstanden Beispiel, wie Essiggurke mit Dill aus der folgenden Diskussion zu ersetzen: pickle-dill. Zum Beispiel arbeitete der folgende Code für mich

import os 
import dill 
import multiprocessing 

def run_dill_encoded(what): 
    fun, args = dill.loads(what) 
    return fun(*args) 

def apply_async(pool, fun, args): 
    return pool.apply_async(run_dill_encoded, (dill.dumps((fun, args)),)) 

if __name__ == '__main__': 

    pool = multiprocessing.Pool(5) 
    results = [apply_async(pool, lambda x: x*x, args=(x,)) for x in range(1,7)] 
    output = [p.get() for p in results] 
    print(output) 

Ich habe versucht, die gleiche Philosophie anwenden pymongo. Der folgende Code

import os 
import dill 
import multiprocessing 
import pymongo 

def run_dill_encoded(what): 
    fun, args = dill.loads(what) 
    return fun(*args) 


def apply_async(pool, fun, args): 
    return pool.apply_async(run_dill_encoded, (dill.dumps((fun, args)),)) 


def write_to_db(value_to_insert):   
    client = pymongo.MongoClient('localhost', 27017) 
    db = client['somedb'] 
    collection = db['somecollection'] 
    result = collection.insert_one({"filed1": value_to_insert}) 
    client.close() 

if __name__ == '__main__': 
    pool = multiprocessing.Pool(5) 
    results = [apply_async(pool, write_to_db, args=(x,)) for x in ['one', 'two', 'three']] 
    output = [p.get() for p in results] 
    print(output) 

produziert Fehler:

multiprocessing.pool.RemoteTraceback: 
""" 
Traceback (most recent call last): 
    File "C:\Python34\lib\multiprocessing\pool.py", line 119, in worker 
    result = (True, func(*args, **kwds)) 
    File "C:\...\temp2.py", line 10, in run_dill_encoded 
    return fun(*args) 
    File "C:\...\temp2.py", line 21, in write_to_db 
    client = pymongo.MongoClient('localhost', 27017) 
NameError: name 'pymongo' is not defined 
""" 

The above exception was the direct cause of the following exception: 

Traceback (most recent call last): 
    File "C:/.../temp2.py", line 32, in <module> 
    output = [p.get() for p in results] 
    File "C:/.../temp2.py", line 32, in <listcomp> 
    output = [p.get() for p in results] 
    File "C:\Python34\lib\multiprocessing\pool.py", line 599, in get 
    raise self._value 
NameError: name 'pymongo' is not defined 

Process finished with exit code 1 

Was ist falsch?

Answer 1

Wie ich in den Kommentaren erwähnt habe, müssen Sie eine import pymongo innerhalb der Funktion write_to_db setzen. Dies liegt daran, dass die serialisierte Funktion keine der globalen Referenzen mitnimmt, wenn sie an den anderen Prozessbereich gesendet wird.