Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance regression simulating circuits with a noise model in Aer 0.13.0 #1985

Closed
wshanks opened this issue Nov 3, 2023 · 1 comment
Closed
Assignees
Labels
bug Something isn't working

Comments

@wshanks
Copy link
Contributor

wshanks commented Nov 3, 2023

Informations

  • Qiskit Aer version: 0.13.0
  • Python version: 3.11.6
  • Operating system: Fedora Linux 38

What is the current behavior?

I am seeing AerSimulator runs that took around 0.3 seconds with Aer 0.12.2 take 6-8 seconds with Aer 0.13.0.

Steps to reproduce the problem

Here is my test script. It requires a the rb.qpy file with the circuits which I am attaching along with the code blocks in this post here. These are randomized benchmarking circuits.

import importlib.metadata
from time import perf_counter

from qiskit import qpy

from qiskit_aer import AerSimulator
from qiskit_aer.noise import NoiseModel, depolarizing_error


p1q = 0.004
p2q = 0.04
pvz = 0.0

# basis gates
basis_gates = ["rz", "sx", "cx"]

# setup noise model
sx_error = depolarizing_error(p1q, 1)
rz_error = depolarizing_error(pvz, 1)
cx_error = depolarizing_error(p2q, 2)

noise_model = NoiseModel()
noise_model.add_all_qubit_quantum_error(sx_error, "sx")
noise_model.add_all_qubit_quantum_error(rz_error, "rz")
noise_model.add_all_qubit_quantum_error(cx_error, "cx")

# Need level1 for consecutive gate cancellation for reference EPC value calculation
transpiler_options = {
    "basis_gates": basis_gates,
}

# Aer simulator
backend = AerSimulator(noise_model=noise_model, seed_simulator=123)

pkg = importlib.metadata.distribution("qiskit-terra")
print(f"qiskit-terra version: {pkg.version}")
pkg = importlib.metadata.distribution("qiskit-aer")
print(f"qiskit-aer version: {pkg.version}")

with open("rb.qpy", "rb") as f:
    circs = qpy.load(f)

start = perf_counter()
job = backend.run(circs)
job.result()
print(f"Aer simulator run: {perf_counter() - start}")

With Aer 0.12.2, the script prints around 0.2 seconds for the run while for 0.13.0 it prints around 7.9 seconds. A colleague tested on macOS and saw 0.05 seconds vs 4.1 seconds for the two Aer versions.

What is the expected behavior?

No change in performance

Suggested solutions

I do not have a suggested solution, but here is the code that generated the circuits. It requires qiskit-experiments. It is based on a test from the qiskit-experiments test suite.

from qiskit import qpy

import qiskit_aer.version
from qiskit_aer import AerSimulator
from qiskit_aer.noise import NoiseModel, depolarizing_error
from qiskit_experiments.library import randomized_benchmarking as rb


p1q = 0.004
p2q = 0.04
pvz = 0.0

# basis gates
basis_gates = ["rz", "sx", "cx"]

# setup noise model
sx_error = depolarizing_error(p1q, 1)
rz_error = depolarizing_error(pvz, 1)
cx_error = depolarizing_error(p2q, 2)

noise_model = NoiseModel()
noise_model.add_all_qubit_quantum_error(sx_error, "sx")
noise_model.add_all_qubit_quantum_error(rz_error, "rz")
noise_model.add_all_qubit_quantum_error(cx_error, "cx")

noise_model = noise_model

# Need level1 for consecutive gate cancellation for reference EPC value calculation
transpiler_options = {
    "basis_gates": basis_gates,
}

# Aer simulator
backend = AerSimulator(noise_model=noise_model, seed_simulator=123)

exp = rb.StandardRB(
    physical_qubits=(0, 1),
    lengths=list(range(1, 30, 3)),
    seed=123,
    backend=backend,
)
exp.analysis.set_options(gate_error_ratio=None)
transpiler_options = {"optimization_level": 1}
exp.set_transpile_options(**transpiler_options)

circs = exp._transpiled_circuits()

with open("rb.qpy", "wb") as f:
    qpy.dump(circs, f)
@wshanks wshanks added the bug Something isn't working label Nov 3, 2023
@doichanj doichanj self-assigned this Nov 6, 2023
@doichanj
Copy link
Collaborator

doichanj commented Nov 6, 2023

I found Aer 0.13.0 uses stabilizer method but 0.12.2 uses density_matrix method.
I think there is some issue in selecting method when method="automatic" is set (or no method is set)
When I set method="density_matrix", I got the similar or faster simulation time on 0.13.0

wshanks added a commit to wshanks/qiskit-experiments that referenced this issue Nov 28, 2023
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
  and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
  Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
wshanks added a commit to wshanks/qiskit-experiments that referenced this issue Nov 28, 2023
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
  and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
  Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
github-merge-queue bot pushed a commit to qiskit-community/qiskit-experiments that referenced this issue Nov 29, 2023
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
github-merge-queue bot pushed a commit to qiskit-community/qiskit-experiments that referenced this issue Nov 29, 2023
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
github-merge-queue bot pushed a commit to qiskit-community/qiskit-experiments that referenced this issue Nov 29, 2023
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
itoko pushed a commit to itoko/qiskit-experiments that referenced this issue Dec 12, 2023
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
nkanazawa1989 pushed a commit to nkanazawa1989/qiskit-experiments that referenced this issue Jan 17, 2024
+ Unpin qiskit-aer extra requirement now that 0.13.1 has been released
and we do not have to worry about
Qiskit/qiskit-aer#1985
+ Remove workaround in test for
Qiskit/qiskit-aer#1975

Note that the tests will not pass with 0.13.0 because the workaround was
removed, but the workaround was for an Aer bug so we don't reflect it as
an incomapitibility with qiskit-experiments.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants