所以我尝试在字典的字典上使用多处理管理器,这是我最初的尝试:
from multiprocessing import Process, Manager
def task(stat):
test['z'] += 1
test['y']['Y0'] += 5
if __name__ == '__main__':
test = Manager().dict({'x': {'X0': 10, 'X1': 20}, 'y': {'Y0': 0, 'Y1': 0}, 'z': 0})
p = Process(target=task, args=(test,))
p.start()
p.join()
print(test)
当然,当我运行这个时,输出不是我所期望的,z
正确更新,同时y
不变!这是输出:
{'x': {'X0': 10, 'X1': 20}, 'y': {'Y0': 0, 'Y1': 0}, 'z': 1}
然后我google了一下,找到了解释here https://stackoverflow.com/questions/9436757/how-does-multiprocessing-manager-work-in-python,显然嵌套的字典也必须是Manager().dict()
s 而不是普通的 python 字典(从 Python 3.6 开始可能)。所以我做了以下事情:
from multiprocessing import Process, Manager
def task(stat):
test['z'] += 1
test['y']['Y0'] += 5
if __name__ == '__main__':
test = Manager().dict({'x': Manager().dict({'X0': 10, 'X1': 20}), 'y': Manager().dict({'Y0': 0, 'Y1': 0}), 'z': 0})
p = Process(target=task, args=(test,))
p.start()
p.join()
print(test)
print(test['y'])
但它并没有正常工作,而是出现了这个无法解释的错误,为了清楚起见,将其分为三个部分。第一部分对应于test['y']['Y0'] += 5
而第二个只是print(test)
最后一个是输出print(test['y'])
Process Process-4:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "shit.py", line 5, in task
test['y']['Y0'] += 5
File "<string>", line 2, in __getitem__
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
File "/usr/lib/python3.7/multiprocessing/managers.py", line 920, in RebuildProxy
return func(token, serializer, incref=incref, **kwds)
File "/usr/lib/python3.7/multiprocessing/managers.py", line 770, in __init__
self._incref()
File "/usr/lib/python3.7/multiprocessing/managers.py", line 824, in _incref
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 492, in Client
c = SocketClient(address)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
{'x': <DictProxy object, typeid 'dict' at 0x7f01de2c5860>, 'y': <DictProxy object, typeid 'dict' at 0x7f01de2c5898>, 'z': 1}
Traceback (most recent call last):
File "test.py", line 16, in <module>
print(test['y'])
File "<string>", line 2, in __getitem__
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
File "/usr/lib/python3.7/multiprocessing/managers.py", line 920, in RebuildProxy
return func(token, serializer, incref=incref, **kwds)
File "/usr/lib/python3.7/multiprocessing/managers.py", line 770, in __init__
self._incref()
File "/usr/lib/python3.7/multiprocessing/managers.py", line 824, in _incref
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 492, in Client
c = SocketClient(address)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
我不知道为什么会发生这种情况。内部字典显然是被创建的(如输出的第二部分所示)。但由于某种原因,它们根本无法读取或写入!为什么会发生这种情况?
Extra:如果我通过 python 控制台(而不是脚本)运行相同的 python 代码,错误将从FileNotFoundError
to ConnectionRefusedError
。但具有相同的精确回溯!