Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for custom backend transpiler stages #8648

Merged
merged 16 commits into from
Oct 4, 2022
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion qiskit/compiler/transpiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ def transpile(
hls_config: Optional[HLSConfig] = None,
init_method: str = None,
optimization_method: str = None,
ignore_backend_default_methods: bool = False,
) -> Union[QuantumCircuit, List[QuantumCircuit]]:
"""Transpile one or more circuits, according to some desired transpilation targets.

Expand Down Expand Up @@ -276,6 +277,11 @@ def callback_func(**kwargs):
plugin is not used. You can see a list of installed plugins by
using :func:`~.list_stage_plugins` with ``"optimization"`` for the
``stage_name`` argument.
ignore_backend_default_methods: If set to ``True`` any default methods specified by
a backend will be ignored. Some backends specify alternative default methods
to support custom compilation target passes/plugins which support backend
specific compilation techniques. If you'd prefer that these did not run and
use the defaults for a given optimization level this can be used.
jlapeyre marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a more descriptive name that we can use here? I'm not sure that "backend default methods" will be clear to a user without context. Maybe something like ignore_backend_defined_plugins or ignore_backend_supplied_plugins?

Copy link
Member Author

@mtreinish mtreinish Sep 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to changing the name, what about ignore_backend_supplied_default_methods? I'm hesitant about the two suggestions because the plugin could be defined or supplied from somewhere else. For example, I could see a backend using the synthesis translation method (assuming it actually worked reliably which I'm skeptical of) which is supplied and defined in terra itself. Or more realistically a plugin defined in a different package that's not terra or the provider.

Copy link
Contributor

@jlapeyre jlapeyre Oct 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also see my comment above trying to make the text a bit more clear.... I didn't realize I had not done "finish your review".
It's also not clear to me whether "method" means instance method or has the more generic, non-PL meaning. This is a recurrent problem in documentation.
EDIT: Re-reading the doc string, it's clear that this refers to an instance method.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the context of transpile() and generate_preset_pass_manager() the "method" I'm referring to is the *method kwargs (init_method, layout_method, routing_method, translation_method, optimization_method, scheduling_method). This new option is to disable changing the default values that a BackendV2 object (from the backend kwarg) can optionally set for these kwargs. I'm open to changing the word method if people have suggestions, but I was using it try and be consistent with the rest of the transpile() api.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I see that it's also clear in the doc that it means "instance method". It makes sense to me that whatever the name of the method in question here (eg gnore_backend_supplied_default_methods) , including the word "method" makes sense because it controls which methods will be called.
(I don't know why my comment appears twice in my view of this page. Probably due to the way I finished the review.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed it to ignore_backend_supplied_default_methods in d004080


Returns:
The transpiled circuit(s).
Expand Down Expand Up @@ -344,6 +350,7 @@ def callback_func(**kwargs):
hls_config,
init_method,
optimization_method,
ignore_backend_default_methods,
)
# Get transpile_args to configure the circuit transpilation job(s)
if coupling_map in unique_transpile_args:
Expand Down Expand Up @@ -426,7 +433,7 @@ def _log_transpile_time(start_time, end_time):
def _combine_args(shared_transpiler_args, unique_config):
# Pop optimization_level to exclude it from the kwargs when building a
# PassManagerConfig
level = shared_transpiler_args.get("optimization_level")
level = shared_transpiler_args.pop("optimization_level")
pass_manager_config = shared_transpiler_args
pass_manager_config.update(unique_config.pop("pass_manager_config"))
pass_manager_config = PassManagerConfig(**pass_manager_config)
Expand Down Expand Up @@ -597,6 +604,7 @@ def _parse_transpile_args(
hls_config,
init_method,
optimization_method,
ignore_backend_default_methods,
) -> Tuple[List[Dict], Dict]:
"""Resolve the various types of args allowed to the transpile() function through
duck typing, overriding args, etc. Refer to the transpile() docstring for details on
Expand Down Expand Up @@ -669,6 +677,12 @@ def _parse_transpile_args(
}

list_transpile_args = []
if not ignore_backend_default_methods:
if scheduling_method is None and hasattr(backend, "get_scheduling_stage_method"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to me like restricting the flexibility of StagedPassManager where we can define transpiration with arbitrary combination of passes, i.e. the class can take stages argument. I have no idea how current configuration of passes is generic, but alternatively backend can provide full set of staged passes rather than providing a particular subset passes that Qiskit transpiler defines. For example, some ecosystem software may offer a service that overrides an IBM backend with fancy pass manager that makes circuit more robust to noise -- I also think we no longer say this is a plugin. If this means to be a plugin, perhaps current implementation is fine.

Copy link
Member Author

@mtreinish mtreinish Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are 2 pieces here, for the preset passmanagers/transpile() vs a more generic interface. I think for transpile() we still want to have a more structured set of default stages. This was actually a big part of defining the StagedPassManager for me was being able to formally define the stages for the default passmanagers to enable hooking into it like this. For this interface I was trying to enable backends to hook into the defaults stages with custom implementations (via plugins) that are specific for the needs of a backend. This is more about influencing the defaults to be better for the custom constraints of a particular backend. A user or a backend provider can still implement a fully custom StagedPassManager with additional or just different stages, but that is a lower level interface than what this PR is for. The idea for this is to basically enable users who run transpile(circuit, backend) where backend has custom constraints of some kind that the default preset passmanagers aren't able to handle. Having this interface enables the backend to easily keep transpile() working so those custom constraints can be handled cleanly without having to reimplement the entire pass manager and keep the rest of the user experience the same. For this PR I started with just translation and scheduling as I figured these stages were the most likely to be the ones which would have a need which was backend specific. The other thing is the default stages in the preset passmanager can (and probably will) be expanded in the future. So if we have a change in the global needs for the stages we use in the default pipeline we can add stages in the future (we've documented stability expectations around this in the class docs: https://qiskit.org/documentation/stubs/qiskit.transpiler.StagedPassManager.html?highlight=stagedpassmanager#qiskit.transpiler.StagedPassManager and basically said that additions are expected in the future).

The second part is what the plugin usage. If you look at earlier iterations of this PR, I had this interface be a bit more generic similar to what you're describing here. Instead of working with plugins backends would return PassManagers for each stage implementation (see cf6a7ec where this changed). The consensus on this interface in discussions about this PR was that because we have a plugin interface for providing alternative implementations of the default stages (which was added in #8305) was that for this backend interface to leverage that new plugin interface would be ideal. This has 2 effects, it simplifies the code since we have a single entrypoint for adding custom methods for stages and also for custom stage implementations for a backend it advertises these over the standard plugin interface which hopefully makes them globally discoverable outside of just the default use when targeting a backend.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okey, so we think current default stages of qiskit transpiler is sort of stable in future. My only concern was the stability of new API. To me current mechanism based off of #8305 seems reasonable.

scheduling_method = backend.get_scheduling_stage_method()
if translation_method is None and hasattr(backend, "get_translation_stage_method"):
translation_method = backend.get_translation_stage_method()

for key, value in {
"inst_map": inst_map,
"coupling_map": coupling_map,
Expand Down
47 changes: 47 additions & 0 deletions qiskit/providers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,53 @@ def _define(self):
transpiler will ensure that it continues to be well supported by Qiskit
moving forward.

.. _custom_transpiler_backend:

Custom Transpiler Passes
^^^^^^^^^^^^^^^^^^^^^^^^
The transpiler supports the ability for backends to provide custom transpiler
stage implementations to facilitate hardware specific optimizations and
circuit transformations. Currently there are two stages supported,
``get_translation_stage_method()`` and ``get_scheduling_stage_method()``
which allow a backend to specify string plugin names to be used as the default
translation and scheduling stages, respectively. These
hook points in a :class:`~.BackendV2` class can be used if your
backend has requirements for compilation that are not met by the
current backend/:class:`~.Target` interface. Please also consider
submitting a Github issue describing your use case as there is interest
in improving these interfaces to be able to describe more hardware
architectures in greater depth.

To leverage these hook points you just need to add the methods to your
:class:`~.BackendV2` implementation and have them return a string plugin name.
For example::


class Mybackend(BackendV2):

def get_scheduling_stage_method(self):
return "SpecialDD"

def get_translation_stage_method(self):
return "BasisTranslatorWithCustom1qOptimization"

This snippet of a backend implementation will now have the :func:`~.transpile`
function use the ``SpecialDD`` plugin for the scheduling stage and
the ``BasisTranslatorWithCustom1qOptimization`` plugin for the translation
stage by default when the target is set to ``Mybackend``. Note that users may override these choices
by explicitly selecting a different plugin name. For this interface to work though transpiler
stage plugins must be implemented for the returned plugin name. You can refer
to :mod:`qiskit.transpiler.preset_passmanagers.plugin` module documentation for
details on how to implement plugins. The typical expectation is that if your backend
requires custom passes as part of a compilation stage the provider package will
include the transpiler stage plugins that use those passes. However, this is not
required and any valid method (from a built-in method or external plugin) can
be used.

This way if these two compilation steps are **required** for running or providing
efficient output on ``Mybackend`` the transpiler will be able to perform these
custom steps without any manual user input.

Run Method
----------

Expand Down
20 changes: 20 additions & 0 deletions qiskit/providers/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,26 @@ class BackendV2(Backend, ABC):
will build a :class:`~qiskit.providers.models.BackendConfiguration` object
and :class:`~qiskit.providers.models.BackendProperties` from the attributes
defined in this class for backwards compatibility.

A backend object can optionally contain methods named
``get_translation_stage_method`` and ``get_scheduling_stage_method``. If these
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be renamed to get_translation_stage_plugin?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I'll update this now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I misread what you wrote here, I'm not sure we want to rename this. I had standardized on method because it's not actually restricted to plugins. It basically is the return of these methods are for changing the default for translation_method and scheduling_method values on transpile(). They don't necessarily have to be a plugin (well at least until we make everything a plugin per #8661). But I can change the name if you'd prefer using plugin

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I think this is clearer post- #8661 . I was trying to avoid the confusion between method as in {layout,routing,scheduling}_method and method as in "python function".

I think either get_translation_method (matching the language from transpile(..., layout_method='foo') or get_translation_stage_plugin (matching the language from #8661 and #8305) would be okay.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed it get_$FOO_stage_plugin in 214d9f7 (my shell messed up the commit message on it).

methods are present on a backend object and this object is used for
:func:`~.transpile` or :func:`~.generate_preset_pass_manager` the
transpilation process will default to using the output from those methods
as the scheduling stage and the translation compilation stage. This
enables a backend which has custom requirements for compilation to specify a
stage plugin for these stages to enable custom transformation of
the circuit to ensure it is runnable on the backend. These hooks are enabled
by default and should only be used to enable extra compilation steps
if they are **required** to ensure a circuit is executable on the backend or
have the expected level of performance. These methods are passed no input
arguments and are expected to return a ``str`` representing the method name
which is should be a stage plugin (see: :mod:`qiskit.transpiler.preset_passmanagers.plugin`
mtreinish marked this conversation as resolved.
Show resolved Hide resolved
for more details on plugins). The typical expected use case is for a backend
provider to implement a stage plugin for ``translation`` or ``scheduling``
that contains the custom compilation passes and then for the hook methods on
the backend object to return the plugin name so that :func:`~.transpile` will
use it by default when targetting the backend.
"""

version = 2
Expand Down
7 changes: 4 additions & 3 deletions qiskit/transpiler/passmanager_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ def __init__(
hls_config=None,
init_method=None,
optimization_method=None,
optimization_level=None,
kdk marked this conversation as resolved.
Show resolved Hide resolved
):
"""Initialize a PassManagerConfig object

Expand Down Expand Up @@ -84,7 +83,6 @@ def __init__(
init_method (str): The plugin name for the init stage plugin to use
optimization_method (str): The plugin name for the optimization stage plugin
to use.
optimization_level (int): The optimization level being used for compilation.
"""
self.initial_layout = initial_layout
self.basis_gates = basis_gates
Expand All @@ -105,7 +103,6 @@ def __init__(
self.unitary_synthesis_plugin_config = unitary_synthesis_plugin_config
self.target = target
self.hls_config = hls_config
self.optimization_level = optimization_level

@classmethod
def from_backend(cls, backend, **pass_manager_options):
Expand Down Expand Up @@ -157,6 +154,10 @@ def from_backend(cls, backend, **pass_manager_options):
if res.target is None:
if backend_version >= 2:
res.target = backend.target
if res.scheduling_method is None and hasattr(backend, "get_scheduling_stage_method"):
res.scheduling_method = backend.get_scheduling_stage_method()
if res.translation_method is None and hasattr(backend, "get_translation_stage_method"):
res.translation_method = backend.get_translation_stage_method()
return res

def __str__(self):
Expand Down
1 change: 0 additions & 1 deletion qiskit/transpiler/preset_passmanagers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,6 @@ def generate_preset_pass_manager(
hls_config=hls_config,
init_method=init_method,
optimization_method=optimization_method,
optimization_level=optimization_level,
)

if backend is not None:
Expand Down
10 changes: 2 additions & 8 deletions qiskit/transpiler/preset_passmanagers/level3.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,6 @@ def level_3_pass_manager(pass_manager_config: PassManagerConfig) -> StagedPassMa
unitary_synthesis_plugin_config = pass_manager_config.unitary_synthesis_plugin_config
target = pass_manager_config.target
hls_config = pass_manager_config.hls_config
# Override an unset optimization_level for stage plugin use.
# it will be restored to None before this is returned
optimization_level = pass_manager_config.optimization_level
if optimization_level is None:
pass_manager_config.optimization_level = 3

# Layout on good qubits if calibration info available, otherwise on dense links
_given_layout = SetLayout(initial_layout)
Expand Down Expand Up @@ -314,14 +309,13 @@ def _unroll_condition(property_set):
sched = common.generate_scheduling(
instruction_durations, scheduling_method, timing_constraints, inst_map
)
elif isinstance(scheduling_method, PassManager):
sched = scheduling_method
else:
sched = plugin_manager.get_passmanager_stage(
"scheduling", scheduling_method, pass_manager_config, optimization_level=3
)

# Restore PassManagerConfig optimization_level override
pass_manager_config.optimization_level = optimization_level

return StagedPassManager(
init=init,
layout=layout,
Expand Down
18 changes: 18 additions & 0 deletions releasenotes/notes/add-backend-custom-passes-cddfd05c8704a4b1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
features:
- |
The :class:`~.BackendV2` class now has support for two new optional hook
points enabling backends to inject custom compilation steps as part of
:func:`~.transpile` and :func:`~.generate_preset_pass_manager`. If a
:class:`~.BackendV2` implementation includes the methods
``get_scheduling_stage_method()`` or ``get_translation_stage_method()`` the
transpiler will use the returned string as the default value for
the ``scheduling_method`` and ``translation_method`` arguments. This enables
backends to run additional custom transpiler passes when targetting that
backend by leveraging the transpiler stage
:mod:`~qiskit.transpiler.preset_passmanagers.plugin` interface.
For more details on how to use this see :ref:`custom_transpiler_backend`.
- |
Added a new keyword argument, ``ignore_backend_default_methods``, to the
:func:`~.transpile` function can be used to disable a backend's custom
default method if the target backend has one set.
13 changes: 5 additions & 8 deletions releasenotes/notes/stage-plugin-interface-47daae40f7d0ad3c.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,9 @@ features:
``optimization_method`` which are used to specify alternative plugins to
use for the ``init`` stage and ``optimization`` stages respectively.
- |
The :class:`~.PassManagerConfig` class has 3 new attributes,
:attr:`~.PassManagerConfig.init_method`,
:attr:`~.PassManagerConfig.optimization_method`, and
:attr:`~.PassManagerConfig.optimization_level` along with matching keyword
arguments on the constructor methods. The first two attributes represent
The :class:`~.PassManagerConfig` class has 2 new attributes,
:attr:`~.PassManagerConfig.init_method` and
:attr:`~.PassManagerConfig.optimization_method`
along with matching keyword arguments on the constructor methods. These represent
the user specified ``init`` and ``optimization`` plugins to use for
compilation. The :attr:`~.PassManagerConfig.optimization_level` attribute
represents the compilations optimization level if specified which can
be used to inform stage plugin behavior.
compilation.
Loading