Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BaseResult functionality to primitives module #8091

Merged
merged 28 commits into from
Sep 14, 2022

Conversation

pedrorrivero
Copy link
Member

@pedrorrivero pedrorrivero commented May 19, 2022

Summary

Adds a primitive result base class to provide common functionality to all inheriting result dataclasses.

Closes #8089
Changelog: New Feature

Details and comments

  • Adds result.num_experiments property.
  • Adds result.experiments property.
  • Checks for consistency in the number of experiments across data fields after instantiation (i.e. on __post_init__).

To do

  • Translate tests from pytest to unittest
  • Add release note
  • Complete docstring (e.g. autosummary)

@pedrorrivero pedrorrivero requested review from a team, ikkoham and t-imamichi as code owners May 19, 2022 17:45
@qiskit-bot
Copy link
Collaborator

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the the following people are requested to review this:

@pedrorrivero pedrorrivero marked this pull request as draft May 19, 2022 18:05
@pedrorrivero pedrorrivero changed the title WIP: Add BaseResult functionality to primitives module Add BaseResult functionality to primitives module May 19, 2022
@coveralls
Copy link

coveralls commented May 24, 2022

Pull Request Test Coverage Report for Build 3051563525

  • 40 of 40 (100.0%) changed or added relevant lines in 4 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.01%) to 84.347%

Totals Coverage Status
Change from base Build 3050113514: 0.01%
Covered Lines: 57998
Relevant Lines: 68761

💛 - Coveralls

@ikkoham ikkoham marked this pull request as ready for review May 25, 2022 01:02
@ikkoham ikkoham marked this pull request as draft May 25, 2022 01:02
@pedrorrivero pedrorrivero marked this pull request as ready for review June 8, 2022 18:17
@ikkoham ikkoham added Changelog: New Feature Include in the "Added" section of the changelog mod: primitives Related to the Primitives module labels Jun 9, 2022
@ikkoham ikkoham added this to the 0.21 milestone Jun 9, 2022
Copy link
Contributor

@ikkoham ikkoham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of the features are useful and they are great from a code standpoint. I leave some minor comments. But, from an interface point of view, it would be good to have @ajavadia and @levbishop review.

qiskit/primitives/__init__.py Outdated Show resolved Hide resolved
test/python/primitives/test_result.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ikkoham ikkoham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. LGTM. But I would like to wait more comments.

@pedrorrivero pedrorrivero force-pushed the primitive-base-result branch 2 times, most recently from bdcb36c to 0753bc9 Compare June 21, 2022 16:18
@t-imamichi
Copy link
Member

Looks good to me too.

Copy link
Contributor

@blakejohnson blakejohnson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I think the name experiments is a poor choice. In the process of developing OpenQASM3, we proposed a generalized "circuit" concept that supplants what has previously been called an "experiment" in terra. This word is also overloaded by the "experiment" concept in qiskit-experiments.

I realize that there are many residual uses of "experiments" in terra, but would it make sense to already start the move away by settling on "circuits" here? i.e. num_circuits and circuits as the property names?

@mtreinish mtreinish modified the milestones: 0.21, 0.22 Jun 23, 2022
@pedrorrivero
Copy link
Member Author

pedrorrivero commented Jun 24, 2022

I don't think that would work @blakejohnson . Some of the primitives (if not all) currently hold a list of circuits that they can use to run "experiments". That means that the user can request n different experiments on anyone of theses circuits (e.g. for different observables in Estimator), and at that point num_circuits will become misleading.

I am very curious to learn how this distinction is being handled in OpenQASM3.

I have no personal attachment to the word "experiment" though, and I'll be happy to change it if we come up with a better alternative. What I like about it though, is that it conveys the fact that we are referring to something atomic; as opposed to "job", for example, which can encapsulate several of these atomic elements.

ikkoham
ikkoham previously approved these changes Sep 9, 2022
Copy link
Contributor

@ikkoham ikkoham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (need to fix lint and tests). Thank you! (I was about to point out that base_result does not use __future__.annotations, but you already fixed it.)

@ikkoham
Copy link
Contributor

ikkoham commented Sep 9, 2022

We cannot see the docs of BasePrimitivesResult because it's not indexed. so you need to add BasePrimitivesResult to the module docstrings in primitives/__init__.py.

@pedrorrivero pedrorrivero requested a review from ikkoham September 9, 2022 10:18
ikkoham
ikkoham previously approved these changes Sep 13, 2022
Copy link
Contributor

@ikkoham ikkoham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LTGM, Thanks!

@ikkoham ikkoham requested a review from t-imamichi September 13, 2022 16:02
@ikkoham
Copy link
Contributor

ikkoham commented Sep 13, 2022

I want to wait @t-imamichi's approval.

"""
for value in self._field_values: # type: Sequence
# TODO: enforce all data fields to be tuples instead of sequences
if not isinstance(value, (Sequence, ndarray)) or isinstance(value, str):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering whether the check isinstance(value, str) is enough or not. What if bytes object is given?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check each entry of Sequence perhaps? Do you have any suggestion, @ikkoham?

Copy link
Member Author

@pedrorrivero pedrorrivero Sep 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with checking each sequence entry is that the types will differ for each data attribute (e.g. values and metadata), and between primitives (e.g. sampler and estimator). So those checks should probably live under the particular primitives' __post_init__ methods (e.g. Estimator.__post_init__), instead of inside BasePrimitiveResult.

Is this what you are referring to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I just wondered whether there is any simpler way of this check.

@t-imamichi
Copy link
Member

This PR looks good overall. I have a minor comment. I'm wondering about the validation check of the data. #8091 (comment)

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
@pedrorrivero
Copy link
Member Author

I have already excluded bytes from the possible types (@t-imamichi was correct, str was not enough).

I don't think that this type check is comprehensive though (surely better than no check at all), but the best approach to solve it should be by enforcing an specific type (e.g. tuple) as proposed in #8105

@t-imamichi
Copy link
Member

Thank you, @pedrorrivero. This PR looks good now.
I also have the same impression that the type check might not be comprehensive. But I don't have a good idea. We can leave it as future work.

Copy link
Contributor

@ikkoham ikkoham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's merge this PR!

@mergify mergify bot merged commit ae29802 into Qiskit:main Sep 14, 2022
@pedrorrivero pedrorrivero deleted the primitive-base-result branch September 14, 2022 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: New Feature Include in the "Added" section of the changelog mod: primitives Related to the Primitives module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Number of experiments in primitive results
7 participants