Benchmarks can't execute due to Cirq's metapackaging strategy #4695
Labels
area/dev
area/packages
help wanted
kind/health
For CI/testing/release process/refactoring/technical debt items
no QC knowledge needed
Want to contribute to Cirq, but don't know quantum computing? This issue is for you.
Comments
vtomole
added
no QC knowledge needed
CirqBot
pushed a commit
that referenced
this issue
Sep 7, 2022
Part of #5833 and #3840 Fixes #4695 Running the benchmarks on master currently gives the following output for circuit construction benchmarks: ```bash [ 75.00%] ··· circuit_construction.NDCircuit.time_circuit_construction ok [ 75.00%] ··· ===================== ============= ============= ============= ============ -- Depth(D) --------------------- ------------------------------------------------------ Number of Qubits(N) 1 10 100 1000 ===================== ============= ============= ============= ============ 1 38.3±0.2μs 235±3μs 2.12±0.01ms 21.2±0.1ms 10 109±0.6μs 901±4μs 8.73±0.02ms 89.2±0.2ms 100 770±4μs 7.32±0.02ms 74.4±0.4ms 764±9ms 1000 7.30±0.02ms 72.0±0.2ms 739±4ms 7.57±0.03s ===================== ============= ============= ============= ============ [ 87.50%] ··· circuit_construction.SurfaceCode_Rotated_Memory_Z.time_surface_code_circuit_construction ok [ 87.50%] ··· ========== ============= distance ---------- ------------- 3 1.97±0.01ms 5 7.67±0.07ms 7 22.6±0.1ms 9 59.8±0.4ms 11 122±0.7ms 13 232±1ms 15 407±2ms 17 701±2ms 19 1.09±0.02s 21 1.60±0.02s 23 2.25±0.01s 25 3.24±0.06s ========== ============= [100.00%] ··· circuit_construction.SurfaceCode_Rotated_Memory_Z.track_surface_code_circuit_operation_count ok [100.00%] ··· ========== ========= distance ---------- --------- 3 369 5 3225 7 12985 9 36369 11 82401 13 162409 15 290025 17 481185 19 754129 21 1129401 23 1629849 25 2280625 ========== ========= ``` Note that the circuit construction for d21 surface code takes only 1.6 seconds. This is because we are constructing moment-by-moment instead of appending operations (in which case the earliest strategy takes a significant amount of time). Going forward, we should add more benchmarks for different use cases. Once this is merged, https://cirq-infra.uc.r.appspot.com/ would be updated with the new benchmarks and we can see the improvement / regressions we make to circuit construction performance over time / commits. cc @dabacon @maffoo @dstrain115
rht
pushed a commit
to rht/Cirq
that referenced
this issue
May 1, 2023
…#5845) Part of quantumlib#5833 and quantumlib#3840 Fixes quantumlib#4695 Running the benchmarks on master currently gives the following output for circuit construction benchmarks: ```bash [ 75.00%] ··· circuit_construction.NDCircuit.time_circuit_construction ok [ 75.00%] ··· ===================== ============= ============= ============= ============ -- Depth(D) --------------------- ------------------------------------------------------ Number of Qubits(N) 1 10 100 1000 ===================== ============= ============= ============= ============ 1 38.3±0.2μs 235±3μs 2.12±0.01ms 21.2±0.1ms 10 109±0.6μs 901±4μs 8.73±0.02ms 89.2±0.2ms 100 770±4μs 7.32±0.02ms 74.4±0.4ms 764±9ms 1000 7.30±0.02ms 72.0±0.2ms 739±4ms 7.57±0.03s ===================== ============= ============= ============= ============ [ 87.50%] ··· circuit_construction.SurfaceCode_Rotated_Memory_Z.time_surface_code_circuit_construction ok [ 87.50%] ··· ========== ============= distance ---------- ------------- 3 1.97±0.01ms 5 7.67±0.07ms 7 22.6±0.1ms 9 59.8±0.4ms 11 122±0.7ms 13 232±1ms 15 407±2ms 17 701±2ms 19 1.09±0.02s 21 1.60±0.02s 23 2.25±0.01s 25 3.24±0.06s ========== ============= [100.00%] ··· circuit_construction.SurfaceCode_Rotated_Memory_Z.track_surface_code_circuit_operation_count ok [100.00%] ··· ========== ========= distance ---------- --------- 3 369 5 3225 7 12985 9 36369 11 82401 13 162409 15 290025 17 481185 19 754129 21 1129401 23 1629849 25 2280625 ========== ========= ``` Note that the circuit construction for d21 surface code takes only 1.6 seconds. This is because we are constructing moment-by-moment instead of appending operations (in which case the earliest strategy takes a significant amount of time). Going forward, we should add more benchmarks for different use cases. Once this is merged, https://cirq-infra.uc.r.appspot.com/ would be updated with the new benchmarks and we can see the improvement / regressions we make to circuit construction performance over time / commits. cc @dabacon @maffoo @dstrain115
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description of the issue
See: #1030 (comment)
Cirq version
'0.14.0.dev'
Solution: Tell asv where the subpackages are. For example 024f6fb.
The text was updated successfully, but these errors were encountered: