Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster backend for shot parallelization #1298

Closed
wants to merge 22 commits into from

Conversation

hitomitak
Copy link
Contributor

Summary

Enhancement of #1281

Enhance the execution option to enable shot parallelization.
When a user gives dask client as executor, Aer can execute a simulation by a shot on the distributed machines like HPC clusters.

When the noise simulator executes with the executor option, AerJobSet object copies Qobj or circuits and distributes them to the workers. After simulations, AerJobSet collects each result, sums up with each count, and combines them into one result object.

Example Usage:
Threadpool execution:

exc = ThreadPoolExecutor(max_workers=2)
qbackend = AerSimulator(noise_model=noise_model)
result = qbackend.run(circ_list, executor=exc).result()

Dask execution (Local only):

exc = Client(address=LocalCluster(n_workers=1, processes=True))
qbackend = AerSimulator(noise_model=noise_model)
result = qbackend.run(circ_list, executor=exc).result()

Dask execution (Multiple Nodes):
After setup dask environment in all nodes, set up a cluster in a master node as a following command.

% dask-ssh dask1 dask2 dask3 &

And simulation executes.

exc = Client('dask1:8786')
qbackend = AerSimulator(noise_model=noise_model)
result = qbackend.run(circ_list, executor=exc).result()

Details and comments

I executed QV (10 depth, 100 shots ) with a noise model and measured the execution time on 3VMs with Dask executor.
Env: 3VMs, OS: CentOS8.0, Memory 64 GB, CPU: 32 x 2.0 GHz or higher Cores.
1 Node : a normal Aer simulator
1 Node (DASK) : only one node with dask executor
2 Node (DASK) : two nodes with dask executor
3 Node (DASK) : three nodes with dask executor

qv_noise

@hitomitak hitomitak force-pushed the cluster_backend_shot branch 2 times, most recently from c62c63d to 5fd0ffb Compare August 2, 2021 01:53
@hitomitak hitomitak deleted the cluster_backend_shot branch August 25, 2021 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants