Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data processing #22

Merged
merged 56 commits into from
Apr 23, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
3d25f40
* Carved out data processor from PR #20
eggerdj Mar 26, 2021
375caa0
* Added node_output property.
eggerdj Mar 26, 2021
9994758
* Ran Black.
eggerdj Mar 26, 2021
0fc66ba
* Added a better methodology for checking the input requirements.
eggerdj Mar 26, 2021
a0cbd65
* Lint fix.
eggerdj Mar 26, 2021
839ab97
* Added population shots fix and corresponding tests.
eggerdj Mar 29, 2021
38d6675
* Unified ToReal and ToImag.
eggerdj Mar 29, 2021
0be6f1f
* Reformatted the DataProcessor to a list of nodes rather than pointers.
eggerdj Mar 31, 2021
3f95b1e
* Added history functionality to the data processor.
eggerdj Mar 31, 2021
1883a91
* Made history of data processor a property.
eggerdj Apr 1, 2021
6b6051c
* Fixed docstring.
eggerdj Apr 1, 2021
e32341e
* Added _process to the IQPart data actions.
eggerdj Apr 1, 2021
681546f
* Changed node_output to a class variable.
eggerdj Apr 1, 2021
93cdfae
* Added the option to initialize the DataProcessor with given DataAct…
eggerdj Apr 1, 2021
21374e5
* Removed Kernel and Discriminator. They will be for a future PR.
eggerdj Apr 1, 2021
5f7c082
* Added docstring from Will.
eggerdj Apr 9, 2021
8838f62
* Moved docstring.
eggerdj Apr 12, 2021
77d0bb0
Update qiskit_experiments/data_processing/base.py
eggerdj Apr 12, 2021
81ddf28
Update qiskit_experiments/data_processing/base.py
eggerdj Apr 12, 2021
6af38d9
* Aligned code to _process.
eggerdj Apr 12, 2021
bd230b5
* Made data processor callable.
eggerdj Apr 12, 2021
c9801a7
* Renamed base.py to data_action.py.
eggerdj Apr 12, 2021
ca2d365
* Made nodes callable.
eggerdj Apr 12, 2021
06095d2
* Removed history property, added call_with_history.
eggerdj Apr 12, 2021
22acc21
* Renamed Population to Probability.
eggerdj Apr 13, 2021
cefcf73
* Metadata in processed_data.
eggerdj Apr 13, 2021
a0d0903
* Refactored _process(Dict[str, Any]) -> Dict[str, Any] to _process(A…
eggerdj Apr 13, 2021
1bc7c83
* Added option to specifiy which nodes to include in the history.
eggerdj Apr 13, 2021
02c46a1
Merge branch 'main' into data_processor
eggerdj Apr 13, 2021
650d9c6
Update qiskit_experiments/data_processing/nodes.py
eggerdj Apr 16, 2021
078fd07
* Removed __init__ from DataAction.
eggerdj Apr 16, 2021
d0a85c1
Merge branch 'data_processor' of github.com:eggerdj/qiskit-experiment…
eggerdj Apr 16, 2021
4899bfa
* Added the option to turn of validation.
eggerdj Apr 16, 2021
b06e3eb
Update qiskit_experiments/data_processing/data_processor.py
eggerdj Apr 16, 2021
597d60c
* Simplified validation of IQ data.
eggerdj Apr 16, 2021
055d6cc
Update qiskit_experiments/data_processing/nodes.py
eggerdj Apr 16, 2021
a498bf0
Update qiskit_experiments/data_processing/nodes.py
eggerdj Apr 16, 2021
a35b522
* Removed unnecessary wrapping of _process.
eggerdj Apr 16, 2021
74ec8f8
* Polished docstrings and ran black.
eggerdj Apr 16, 2021
ec911bd
Update qiskit_experiments/data_processing/data_action.py
eggerdj Apr 16, 2021
83ed8ff
* Removed unnecessary code in DataProcessingError.
eggerdj Apr 16, 2021
2ffb0d3
* Rewrote doc string.
eggerdj Apr 16, 2021
657f17b
* IQ data is now of type float and not complex.
eggerdj Apr 16, 2021
58ca872
* Fixed validate issue.
eggerdj Apr 20, 2021
d749d95
* Added error message to __call__ and call_with_history.
eggerdj Apr 20, 2021
cab9339
* Improved docstring.
eggerdj Apr 20, 2021
4c9acae
* Impoved class docstring.
eggerdj Apr 20, 2021
d910933
* Changed how DataProcessor._nodes are initialized in __init__.
eggerdj Apr 20, 2021
c250ad8
* Changed behavior of empty data processor.
eggerdj Apr 20, 2021
bc00e26
* Refactored call and call_with_history to use the call_internal func…
eggerdj Apr 21, 2021
81caca7
* Fixed, lint, black, and docstrings.
eggerdj Apr 21, 2021
bcce8eb
Merge branch 'main' into data_processor
eggerdj Apr 21, 2021
2452a86
Update qiskit_experiments/data_processing/data_action.py
eggerdj Apr 22, 2021
a79d270
* Added type hint to call_with_history
eggerdj Apr 22, 2021
b19c31b
Merge branch 'data_processor' of github.com:eggerdj/qiskit-experiment…
eggerdj Apr 22, 2021
5942ede
Update qiskit_experiments/data_processing/data_processor.py
eggerdj Apr 22, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions qiskit_experiments/data_processing/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# This code is part of Qiskit.
#
# (C) Copyright IBM 2021.
#
# This code is licensed under the Apache License, Version 2.0. You may
# obtain a copy of this license in the LICENSE.txt file in the root directory
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
#
# Any modifications or derivative works of this code must retain this
# copyright notice, and modified files need to carry a notice indicating
# that they have been altered from the originals.

"""Qiskit experiments calibration data processing roots."""

from .base import DataAction
from .nodes import (
# data acquisition node
Discriminator,
Kernel,
# value formatter node
Population,
eggerdj marked this conversation as resolved.
Show resolved Hide resolved
ToImag,
ToReal,
)

from .data_processor import DataProcessor
170 changes: 170 additions & 0 deletions qiskit_experiments/data_processing/base.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# This code is part of Qiskit.
#
# (C) Copyright IBM 2021.
#
# This code is licensed under the Apache License, Version 2.0. You may
# obtain a copy of this license in the LICENSE.txt file in the root directory
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
#
# Any modifications or derivative works of this code must retain this
# copyright notice, and modified files need to carry a notice indicating
# that they have been altered from the originals.

"""Defines the steps that can be used to analyse data."""

from abc import ABCMeta, abstractmethod
from enum import Enum
from typing import Any, Dict, List

from qiskit_experiments.data_processing.exceptions import DataProcessorError


class DataAction(metaclass=ABCMeta):
"""
Abstract action which is a single action done on measured data to process it.
Each subclass of DataAction must define the type of data that it accepts as input
using decorators.
"""

node_type = None
prev_node = ()

def __init__(self):
"""Create new data analysis routine."""
self._child = None

@property
def child(self) -> "DataAction":
"""Return the child of this data processing step."""
return self._child

def append(self, component: "DataAction"):
"""Add new data processing routine.

Args:
component: New data processing routine.

Raises:
DataProcessorError: If the previous node is None (i.e. a root node)
"""
if not component.prev_node:
raise DataProcessorError(
f"Analysis routine {component.__class__.__name__} is a root"
f"node. This routine cannot be appended to another node."
)

if self._child is None:
if isinstance(self, component.prev_node):
self._child = component
else:
raise DataProcessorError(
f"Analysis routine {component.__class__.__name__} "
f"cannot be appended after {self.__class__.__name__}"
)
else:
self._child.append(component)

@property
@abstractmethod
def node_output(self) -> str:
"""Returns the key in the data dict where the DataAction added the processed data."""

@property
@abstractmethod
def node_inputs(self) -> List[str]:
"""Returns a list of input data that the node can process."""

@abstractmethod
def process(self, data: Dict[str, Any]):
"""
Applies the data processing step to the data.

Args:
data: the data to which the data processing step will be applied.
"""

def check_required(self, data: Dict[str, Any]):
eggerdj marked this conversation as resolved.
Show resolved Hide resolved
"""Checks that the given data contains the right key.

Args:
data: The data to check for the correct keys.

Raises:
DataProcessorError: if the key is not found.
"""
for key in data.keys():
if key in self.node_inputs:
return

raise DataProcessorError(f"None of {self.node_inputs} are in the given data.")

def format_data(self, data: Dict[str, Any]):
"""
Apply the data action of this node and call the child node's format_data method.

Args:
data: A dict containing the data. The action nodes in the data
processor will raise errors if the data does not contain the
appropriate data.
"""
self.check_required(data)

processed_data = self.process(data)

if self._child:
self._child.format_data(processed_data)


class NodeType(Enum):
"""Type of node that can be supported by the analysis steps."""

KERNEL = 1
DISCRIMINATOR = 2
IQDATA = 3
COUNTS = 4
POPULATION = 5


def kernel(cls: DataAction):
"""A decorator to give kernel attribute to node."""
cls.node_type = NodeType.KERNEL
return cls


def discriminator(cls: DataAction):
"""A decorator to give discriminator attribute to node."""
cls.node_type = NodeType.DISCRIMINATOR
return cls


def iq_data(cls: DataAction):
"""A decorator to give iqdata attribute to node."""
cls.node_type = NodeType.IQDATA
return cls


def counts(cls: DataAction):
"""A decorator to give counts attribute to node."""
cls.node_type = NodeType.COUNTS
return cls


def population(cls: DataAction):
"""A decorator to give population attribute to node."""
cls.node_type = NodeType.POPULATION
return cls


def prev_node(*nodes: DataAction):
"""A decorator to specify the available previous nodes."""

try:
nodes = list(nodes)
except TypeError:
nodes = [nodes]

def add_nodes(cls: DataAction):
cls.prev_node = tuple(nodes)
return cls

return add_nodes
115 changes: 115 additions & 0 deletions qiskit_experiments/data_processing/data_processor.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# This code is part of Qiskit.
#
# (C) Copyright IBM 2021.
#
# This code is licensed under the Apache License, Version 2.0. You may
# obtain a copy of this license in the LICENSE.txt file in the root directory
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
#
# Any modifications or derivative works of this code must retain this
# copyright notice, and modified files need to carry a notice indicating
# that they have been altered from the originals.

"""Class that ties together actions on the data."""
eggerdj marked this conversation as resolved.
Show resolved Hide resolved

from typing import Any, Dict, Union

from qiskit.qobj.utils import MeasLevel
from qiskit_experiments.data_processing.nodes import Kernel, Discriminator
from qiskit_experiments.data_processing.base import NodeType, DataAction


class DataProcessor:
"""
Defines the actions done on the measured data to bring it in a form usable
by the calibration analysis classes.
"""

eggerdj marked this conversation as resolved.
Show resolved Hide resolved
def __init__(self):
eggerdj marked this conversation as resolved.
Show resolved Hide resolved
"""Create an empty chain of data ProcessingSteps."""
self._root_node = None

def append(self, node: DataAction):
eggerdj marked this conversation as resolved.
Show resolved Hide resolved
"""
Append new data action node to this data processor.

Args:
node: A DataAction that will process the data.
"""
if self._root_node:
self._root_node.append(node)
else:
self._root_node = node

def meas_level(self) -> MeasLevel:
"""
Returns:
measurement level: MeasLevel.CLASSIFIED is returned if the end data is discriminated,
MeasLevel.KERNELED is returned if a kernel is defined but no discriminator, and
MeasLevel.RAW is returned is no kernel is defined.
"""
kernel = DataProcessor.check_kernel(self._root_node)
if kernel and isinstance(kernel, Kernel):
discriminator = DataProcessor.check_discriminator(self._root_node)
if discriminator and isinstance(discriminator, Discriminator):

# classified level if both system kernel and discriminator are defined
return MeasLevel.CLASSIFIED

# kerneled level if only system kernel is defined
return MeasLevel.KERNELED

# otherwise raw level is requested
return MeasLevel.RAW

def output_key(self) -> str:
"""Return the key to look for in the data output by the processor."""
if self._root_node:
node = self._root_node
while node.child:
node = node.child

return node.node_output

return "counts"

def format_data(self, data: Dict[str, Any]):
"""
Format Qiskit result data.

This method sequentially calls stored child data processing nodes
with its `format_data` methods. Once all child nodes have called,
input data is converted into expected data format.

Args:
data: The data, typically from an ExperimentData instance, that needs to
be processed. This dict also contains the metadata of each experiment.
"""
if self._root_node:
self._root_node.format_data(data)

@classmethod
def check_kernel(cls, node: DataAction) -> Union[None, DataAction]:
"""Return the stored kernel in the workflow."""
if not node:
return None

if node.node_type == NodeType.KERNEL:
return node
else:
if not node.child:
return None
return cls.check_kernel(node.child)

@classmethod
def check_discriminator(cls, node: DataAction):
"""Return stored discriminator in the workflow."""
if not node:
return None

if node.node_type == NodeType.DISCRIMINATOR:
return node
else:
if not node.child:
return None
return cls.check_discriminator(node.child)
28 changes: 28 additions & 0 deletions qiskit_experiments/data_processing/exceptions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# This code is part of Qiskit.
#
# (C) Copyright IBM 2021.
#
# This code is licensed under the Apache License, Version 2.0. You may
# obtain a copy of this license in the LICENSE.txt file in the root directory
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
#
# Any modifications or derivative works of this code must retain this
# copyright notice, and modified files need to carry a notice indicating
# that they have been altered from the originals.

"""Exceptions for data processing."""

from qiskit.exceptions import QiskitError


class DataProcessorError(QiskitError):
"""Errors raised by the calibration module."""

def __init__(self, *message):
"""Set the error message."""
super().__init__(*message)
self.message = " ".join(message)

def __str__(self):
"""Return the message."""
return repr(self.message)
eggerdj marked this conversation as resolved.
Show resolved Hide resolved
Loading