Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing and documenting how the Estimator calculates stds #12670

Merged
merged 17 commits into from
Jul 18, 2024

Conversation

SamFerracin
Copy link
Contributor

Summary

This PR slightly modifies the formula used by the Estimator to calculate stds, and adds details on how the stds are computed.

Details and comments

Addresses issue 1751.

@SamFerracin SamFerracin requested review from a team as code owners June 26, 2024 14:20
@qiskit-bot qiskit-bot added the Community PR PRs from contributors that are not 'members' of the Qiskit repo label Jun 26, 2024
@qiskit-bot
Copy link
Collaborator

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core
  • @ajavadia
  • @levbishop
  • @t-imamichi

@SamFerracin SamFerracin marked this pull request as draft June 26, 2024 14:20
@SamFerracin SamFerracin marked this pull request as ready for review June 26, 2024 14:21
@qiskit-bot
Copy link
Collaborator

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core
  • @ajavadia
  • @levbishop
  • @t-imamichi

@SamFerracin SamFerracin removed the Community PR PRs from contributors that are not 'members' of the Qiskit repo label Jun 26, 2024
@coveralls
Copy link

coveralls commented Jun 26, 2024

Pull Request Test Coverage Report for Build 9681788876

Details

  • 2 of 2 (100.0%) changed or added relevant lines in 1 file are covered.
  • 15 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.01%) to 89.761%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 3 93.13%
crates/qasm2/src/parse.rs 12 97.15%
Totals Coverage Status
Change from base Build 9680727079: 0.01%
Covered Lines: 63786
Relevant Lines: 71062

💛 - Coveralls

@SamFerracin SamFerracin requested a review from ihincks June 28, 2024 15:36
@coveralls
Copy link

coveralls commented Jun 28, 2024

Pull Request Test Coverage Report for Build 9715313071

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
  • 384 unchanged lines in 12 files lost coverage.
  • Overall coverage decreased (-0.006%) to 89.744%

Files with Coverage Reduction New Missed Lines %
qiskit/circuit/library/standard_gates/r.py 1 97.73%
crates/qasm2/src/expr.rs 1 94.02%
crates/accelerate/src/isometry.rs 1 99.65%
crates/accelerate/src/sampled_exp_val.rs 1 88.71%
crates/accelerate/src/convert_2q_block_matrix.rs 3 93.02%
crates/qasm2/src/lex.rs 7 91.35%
crates/accelerate/src/pauli_exp_val.rs 8 89.68%
crates/accelerate/src/sparse_pauli_op.rs 15 93.27%
qiskit/circuit/quantumcircuit.py 17 95.69%
crates/accelerate/src/euler_one_qubit_decomposer.rs 20 89.68%
Totals Coverage Status
Change from base Build 9680727079: -0.006%
Covered Lines: 63801
Relevant Lines: 71092

💛 - Coveralls

@t-imamichi t-imamichi added mod: primitives Related to the Primitives module Changelog: Bugfix Include in the "Fixed" section of the changelog labels Jul 5, 2024
@t-imamichi
Copy link
Member

t-imamichi commented Jul 5, 2024

Thank you for fixing the issue. Could you write a release note of bugfix?
Could you also update the title of this PR because it fixes stds of BackendEstimatorV2 in addition to the documentation?

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
@SamFerracin SamFerracin changed the title Documenting how the Estimator calculates stds Fixing and documenting how the Estimator calculates stds Jul 5, 2024
qiskit/primitives/backend_estimator_v2.py Outdated Show resolved Hide resolved
compatible with Pauli-based observables.
compatible with Pauli-based observables. More formally, given an observable of the type
:math:`O=\sum_{i=1}^Na_iP_i`, where :math:`a_i` is a complex number and :math:`P_i` is a
Pauli operator, the estimator calculates the expectation :math:`E(P_i)` of each :math:`P_i`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mathbb or rn for the E's?


.. math::

\frac{\sum_{i=1}^{n}|a_i|\sqrt{\textrm{Var}\big(P_i\big)}}{\sqrt{N}}\:,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is \: ? I've never used that

Suggested change
\frac{\sum_{i=1}^{n}|a_i|\sqrt{\textrm{Var}\big(P_i\big)}}{\sqrt{N}}\:,
\frac{\sum_{i=1}^{n}|a_i|\sqrt{\textrm{Var}\big(P_i\big)}}{\sqrt{N}},

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the link to that paper which suggests this? it's not intuitive to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eq. 5 of https://arxiv.org/abs/1908.06942. I added a reference to it in the docstring

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\: leaves a blank space. I normally use it when I have a comma or a break at the end of a full-line equation

@coveralls
Copy link

coveralls commented Jul 5, 2024

Pull Request Test Coverage Report for Build 9809468048

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
  • 774 unchanged lines in 35 files lost coverage.
  • Overall coverage increased (+0.1%) to 89.854%

Files with Coverage Reduction New Missed Lines %
qiskit/circuit/library/standard_gates/r.py 1 97.73%
crates/qasm2/src/expr.rs 1 94.02%
qiskit/circuit/library/standard_gates/rxx.py 1 97.56%
qiskit/transpiler/passes/routing/commuting_2q_gate_routing/pauli_2q_evolution_commutation.py 1 97.92%
qiskit/circuit/library/standard_gates/rzx.py 1 97.56%
crates/accelerate/src/isometry.rs 1 99.65%
crates/accelerate/src/sampled_exp_val.rs 1 88.71%
qiskit/quantum_info/operators/symplectic/clifford.py 1 91.87%
qiskit/circuit/library/standard_gates/rz.py 2 95.83%
qiskit/circuit/library/standard_gates/ryy.py 2 95.12%
Totals Coverage Status
Change from base Build 9680727079: 0.1%
Covered Lines: 65202
Relevant Lines: 72564

💛 - Coveralls

SamFerracin and others added 2 commits July 5, 2024 10:57
Co-authored-by: Ian Hincks <ian.hincks@gmail.com>
@coveralls
Copy link

coveralls commented Jul 5, 2024

Pull Request Test Coverage Report for Build 9810207246

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
  • 787 unchanged lines in 36 files lost coverage.
  • Overall coverage increased (+0.09%) to 89.837%

Files with Coverage Reduction New Missed Lines %
qiskit/circuit/library/standard_gates/r.py 1 97.73%
crates/qasm2/src/expr.rs 1 94.02%
qiskit/circuit/library/standard_gates/rxx.py 1 97.56%
qiskit/transpiler/passes/routing/commuting_2q_gate_routing/pauli_2q_evolution_commutation.py 1 97.92%
qiskit/circuit/library/standard_gates/rzx.py 1 97.56%
crates/accelerate/src/isometry.rs 1 99.65%
crates/accelerate/src/sampled_exp_val.rs 1 88.71%
qiskit/quantum_info/operators/symplectic/clifford.py 1 91.87%
qiskit/circuit/library/standard_gates/rz.py 2 95.83%
qiskit/circuit/library/standard_gates/ryy.py 2 95.12%
Totals Coverage Status
Change from base Build 9680727079: 0.09%
Covered Lines: 65189
Relevant Lines: 72564

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jul 5, 2024

Pull Request Test Coverage Report for Build 9810752069

Details

  • 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
  • 4 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.02%) to 89.835%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 4 92.37%
Totals Coverage Status
Change from base Build 9807934619: 0.02%
Covered Lines: 65188
Relevant Lines: 72564

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jul 5, 2024

Pull Request Test Coverage Report for Build 9811721107

Details

  • 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
  • 3 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.03%) to 89.845%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 3 92.62%
Totals Coverage Status
Change from base Build 9807934619: 0.03%
Covered Lines: 65195
Relevant Lines: 72564

💛 - Coveralls

@jsaroni
Copy link

jsaroni commented Jul 9, 2024

In improving the backend_estimator_v2.py documentation, (unrelated to stds, sorry), there is this comment under _preprocess_pub

# calculate expectation values for each pair of parameter value set and pauli

but the lines following the comment don't actually calculate the expectation values.

        # calculate expectation values for each pair of parameter value set and pauli
        param_obs_map = defaultdict(set)
        for index in np.ndindex(*bc_param_ind.shape):
            param_index = bc_param_ind[index]
            param_obs_map[param_index].update(bc_obs[index])

@t-imamichi
Copy link
Member

t-imamichi commented Jul 11, 2024

Good catch, @jsaroni. We will update the comment when we update the method.

…e29.yaml

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
@coveralls
Copy link

coveralls commented Jul 11, 2024

Pull Request Test Coverage Report for Build 9994949577

Details

  • 3 of 3 (100.0%) changed or added relevant lines in 1 file are covered.
  • 8 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.02%) to 89.908%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 2 92.62%
crates/qasm2/src/parse.rs 6 97.15%
Totals Coverage Status
Change from base Build 9992274403: 0.02%
Covered Lines: 65798
Relevant Lines: 73184

💛 - Coveralls

@t-imamichi t-imamichi requested a review from ihincks July 18, 2024 10:35
Copy link
Member

@t-imamichi t-imamichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SamFerracin SamFerracin added this pull request to the merge queue Jul 18, 2024
Merged via the queue into Qiskit:main with commit a7e177b Jul 18, 2024
15 checks passed
Procatv pushed a commit to Procatv/qiskit-terra-catherines that referenced this pull request Aug 1, 2024
)

* riverlane paper

* docs

* improvement

* empty

* fix linting and add abs

* Update qiskit/primitives/backend_estimator_v2.py

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>

* CR

* Update qiskit/primitives/backend_estimator_v2.py

Co-authored-by: Ian Hincks <ian.hincks@gmail.com>

* CR

* indent

* Update releasenotes/notes/backend-estimator-v2-variance-905c953415ad0e29.yaml

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>

---------

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
Co-authored-by: Ian Hincks <ian.hincks@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: Bugfix Include in the "Fixed" section of the changelog mod: primitives Related to the Primitives module
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

6 participants