Getting serialization errors while trying to submit Qiskit Circuit List to distributed compute engine (DASK Client) of Qiskit Aer Simulator

[iotaisolutions]

Informations

• Qiskit Aer version: 0.30.0
• Python version: 3.8
• Operating system: Ubuntu 20.04
• dask-kubernetes : 2021.3.1
• distributed : 2021.9.0
• minikube : 1.23.1
• Docker: 20.10.7

What is the current behavior?

With respect to project: qiskit-advocate/qamp-fall-21#39, I am trying to simulate few simple Qiskit circuit lists on a Qiskit Aer Simulator, with compute engine as Dask on Kubernetes Cluster, but continuously facing serialization (pickling) error(s).

I am even getting these errors while testing a DASK code example available on Qiskit Documentation Portal : https://qiskit.org/documentation/apidoc/parallel.html

Error Messages

Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/worker.py", line 3809, in dumps_function
result = cache_dumps[func]
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/utils.py", line 1366, in getitem
value = super().getitem(key)
File "/usr/lib/python3.8/collections/init.py", line 1010, in getitem
raise KeyError(key)
KeyError: <bound method AerBackend._run of AerSimulator('aer_simulator')>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/protocol/pickle.py", line 49, in dumps
result = pickle.dumps(x, **dump_kwargs)
AttributeError: Can't pickle local object 'Client.init..'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "dask_test_1.py", line 34, in
q_exec()
File "dask_test_1.py", line 30, in q_exec
result = qbackend.run(circ_list).result()
File "/home/ubuntu/.local/lib/python3.8/site-packages/qiskit/utils/deprecation.py", line 62, in wrapper
return func(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.8/site-packages/qiskit/providers/aer/backends/aerbackend.py", line 222, in run
aer_job.submit()
File "/home/ubuntu/.local/lib/python3.8/site-packages/qiskit/providers/aer/jobs/aerjobset.py", line 90, in submit
aer_job.submit()
File "/home/ubuntu/.local/lib/python3.8/site-packages/qiskit/providers/aer/jobs/aerjob.py", line 58, in submit
self._future = self._executor.submit(self._fn, self._qobj, self._job_id)
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/client.py", line 1550, in submit
futures = self._graph_to_futures(
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/client.py", line 2571, in _graph_to_futures
dsk = dsk.dask_distributed_pack(self, keyset, annotations)
File "/home/ubuntu/.local/lib/python3.8/site-packages/dask/highlevelgraph.py", line 1063, in dask_distributed_pack
"state": layer.dask_distributed_pack(
File "/home/ubuntu/.local/lib/python3.8/site-packages/dask/highlevelgraph.py", line 421, in dask_distributed_pack
dsk = toolz.valmap(dumps_task, dsk)
File "/home/ubuntu/.local/lib/python3.8/site-packages/toolz/dicttoolz.py", line 83, in valmap
rv.update(zip(d.keys(), map(func, d.values())))
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/worker.py", line 3847, in dumps_task
return {"function": dumps_function(task[0]), "args": warn_dumps(task[1:])}
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/worker.py", line 3811, in dumps_function
result = pickle.dumps(func, protocol=4)
File "/home/ubuntu/.local/lib/python3.8/site-packages/distributed/protocol/pickle.py", line 60, in dumps
result = cloudpickle.dumps(x, **dump_kwargs)
File "/home/ubuntu/.local/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/ubuntu/.local/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 602, in dump
return Pickler.dump(self, obj)
TypeError: cannot pickle '_asyncio.Task' object

Steps to reproduce the problem

Execute the DASK code example available on Qiskit Documentation Portal : https://qiskit.org/documentation/apidoc/parallel.html

What is the expected behavior?

As Cloudpickple module is now available for serialization, so flaws of pickle should not be realized, as both lamda functions & inner class can be managed by cloudpickle.

Suggested solutions

Though serializers option (serializers=['cloudpickle']) of Dask Client was mentioned, but it seems some additional option is required to force the pickling / serialization of Quantum Objects, being submitted by AerSimulator.

[chriseclectic]

Should be fixed in #1365