Quilt Extensions

[braised-babbage]

Description

Tentative PR for adding Quilt features. Mostly setting this up to start discussions around the full set (breaking & backwards-compatible) of changes.

Checklist

The primary purpose is:

• Full set of changes documented and agreed upon.

In addition, we have the usual items:

• The changelog is updated, including author and PR number (@username, gh-xxx).
• Adequate unit test coverage.
• All new and existing tests pass locally and on Travis CI.
• Parameters and return values have type hints with PEP 484 syntax.
• Functions and classes have useful Sphinx-style docstrings.
• All code follows Black style and obeys flake8 conventions.
• The docs have been updated accordingly.

[braised-babbage]

Since this is quite a large PR, I have tried to categorize the changes below. Use this as a reference (but be wary, it's possible I missed something).

Docs

Quilt documentation has been added to the PyQuil docs.

TODO: proofread docs

See:

• docs/source/apidocs/gates.rst (added Quilt instructions)
• docs/source/apidocs/program.rst (document additional fields on the Program class)
• docs/source/apidocs/waveforms.rst (new file: description of raw and template waveforms)
• docs/source/conf.py
• docs/source/index.rst
• docs/source/quilt_getting_started.ipynb (new file: getting started notebook)
• docs/source/quilt_parametric.ipynb (new file: notebook showcasing
  parametric programs with sweeps over frequency, phase, and scale)
• docs/source/quilt_proposal.md (new file: port of original quilt proposal. should give a high level "linguistic" overview)
• docs/source/quilt_raw_capture.ipynb (new file: performing raw capture with Quilt)
• docs/source/quilt_waveforms.ipynb (new file: plotting and manipulating waveforms)

Parser

The parser has been extended to support Quilt keywords and constructs
(e.g. waveforms, frames). The should be backwards compatible, i.e. the
textual representation of a vanilla Quil program should parse as a
Program object containing only vanilla Quil instructions, and the
objects representing these instructions have not channged.

Changes to vanilla PyQuil:

• It is now allowed to use a keyword as a PRAGMA type. This is
  needed e.g. for PRAGMA DELAY, since Quilt adds a DELAY
  instruction.
• Bugfix in string parsing: strings cannot contain double quotes
  (e.g. we have no string escaping mechanism in Quil)

See:

• pyquil/_parser/PyQuilListener.py (added parser methods for Quilt
  objects: frames, calibrations, waveforms, new instructions, etc,
  also PRAGMA change)
• pyquil/_parser/Quil.g4 (revised grammar to support Quilt, also
  PRAGMA and string change)
• pyquil/_parser/gen3/Quil.tokens (autogenerated)
• pyquil/_parser/gen3/QuilLexer.py (autogenerated)
• pyquil/_parser/gen3/QuilLexer.tokens (autogenerated)
• pyquil/_parser/gen3/QuilListener.py (autogenerated)
• pyquil/_parser/gen3/QuilParser.py (autogenerated)

Syntax Objects And Manipulation

The Quil AST has been extended to include Quilt syntax objects. Most
of these are entirely new, but there are also some changes to the
basic PyQuil Program.

Changes to vanilla PyQuil:

• In general, several functions which previously took arguments of
  type Qubit and QubitPlaceholder now also handle
  FormalArgument. This is mostly because Quilt calibrations can be
  specified on either specific qubits or with formal arguments
  (i.e. working on any qubit), and their bodies can contain Quil gates
  which reference this formal argument. Most of the functions affected
  are internal, e.g. _format_qubit_string. We mention here the ones
  which are not internal:
  • unpack_qubit
  • Gate constructor
• Program now has additional properties for Quilt calibrations,
  waveforms, and frames. The behavior is such that for ordinary Quil
  programs the values of these will be null-like (e.g. empty list or
  dict as appropriate).
• Program.out now takes an optional argument calibrations
  indicating whether or not the text of active Quilt calibrations
  should be serialized. This is needed because some services do not
  support Quilt, and so a vanilla Quil program + calibrations can be
  handled via "stripping" the calibrations in this way.
• Program now has a few additional methods for manipulating Quilt
  calibrations: calibrate, get_calibration, and
  match_calibrations.
• Program.is_protoquil now takes an optional flag indicating whether
  quilt programs are allowed (false by default). Program class also
  has a new method, validate_protoquil_or_quilt, which does what it
  says on the tin.
• STANDARD_INSTRUCTIONS (in gates.py), which maps Quil instruction names to
  corresponding constructor functions, now includes an entry for
  "DECLARE".

See:

• pyquil/quilatom.py (change to unpack_qubit, added new classes
  for Quilt frames and waveforms)
• pyquil/quilbase.py (add classes for Quilt instructions, also allow
  FormalArgument in Gate constructor)
• pyquil/quiltwaveforms.py (new file: dataclasses for Quilt template waveforms)
• pyquil/quil.py (numerous changes to Program, for details see above)
• pyquil/quiltcalibrations.py (new file: routines for manipulating
  Quilt calibrations, used by Program)
• pyquil/gates.py (added constructor functions for quilt syntax objects)

Arithmetic Rewriting

Parametric compilation, even on vanilla PyQuil, requires some
client-side hackery in the form of "arithmetic rewriting" to basically
account for the fact that the execution hardware is limited in its
ability to evaluate arithmetic expressions at run-time. Likewise,
Quil(t) values undergo a change of units and representation during
compilation to a binary, and so to support parametric compilation some
of this transformation must be applied "algebraically" on the client
side to support parametric compilation.

Changes to vanilla PyQuil:

• rewrite_arithmetic now converts phase angles from radians to
  revolutions. This was previously done during construction of the QPU
  patch table (in QPU._build_patch_values).
• rewrite_arithmetic now supportes Quilt frame mutations.

TODO: check just how badly things could break due to not having backwards compatibility in rewrite_arithmetic

See:

• pyquil/api/_rewrite_arithmetic.py (above changes)

Memory Management and Readout

Quil only supports one type of readout: a MEASURE which yields a
bit. With Quilt, we now support several kinds of readout (MEASURE,
CAPTURE, and RAW-CAPTURE) resulting in values of different
types. Likewise, the previous PyQuil made some simplifying assumptions
about how readout was conducted (e.g. special treatment of the "ro"
register) which are no longer appropriate for Quilt programs.

Changes to vanilla PyQuil:

• QPUCompiler.native_quil_to_executable now receives the
  executable's ro_sources directly from the QuiltBinaryExecutableResponse.
• The translator requires that memory regions are either read-only or
  write-only, i.e. we do not allow mixed read+write in Quilt. Note
  that this was also the case with Quil, but due mostly to the fact
  that readout was to bits and gate parameters are real, so there was
  not much chance of collision.
• The patch_table field on a QPURequest now only includes read-only
  values, i.e. gate or frame parameters needed for parametric
  compilation.
• Various assumptions in QPU.run about receiving bitstring values
  from the QPU results have been removed. We instead extract arbitrary
  values according to the ro_sources field on the underlying
  executable. In the case that there is no readout in the user
  program, the value of the "ro" region is taken by the client to be
  the empty array (i.e. of length 0).
• Construction of the patch table no longer changes phase units from
  radians to revolutions (in QPU._build_patch_values). This is
  instead handled by rewrite_arithmetic.

TODO: check that qc.run doesn't barf if the user measures to
something other than "ro"

TODO: deprecate _collect_classical_memory_write_locations

See:

• pyquil/api/_qpu.py (above changes)
• pyquil/api/_compiler.py (change to handling of ro_sources)

Other QAM, QPU, Compilation Updates

Changes to vanilla PyQuil:

• A number of methods which previously took RPCQ objects of type
  BinaryExecutableResponse have been changed to expect
  QuiltBinaryExecutableResponse. Likewise, BinaryExecutableRequest
  is mostly subsumed by QuiltBinaryExecutableRequest. For a list of
  non-internal functions, methods, and types impacted by this:
  • AbstractCompiler.native_quil_to_executable
  • QAM.load
  • QPU.load
  • ExecutableDesignator
• QPUCompiler.native_quil_to_executable no longer warns if applied
  to a program object with no quilc metadata. In principle, some users
  might want to skip quilc altogether and compile quilt programs
  directly with this method.
• QPUCompiler has a new method, get_quilt_calibrations, which
  returns a Program object with the Rigetti native Quilt
  calibrations for the current device.

See:

• pyquil/api/_qac.py (changed to use QuiltBinaryExecutableResponse)
• pyquil/api/_qam.py (changed to use QuiltBinaryExecutableResponse)
• pyquil/api/_quantum_computer.py (changed to use QuiltBinaryExecutableResponse)
• pyquil/api/_compiler.py (changed to use QuiltBinaryExecutable*, )

• Misc. Calibration Management and Program Printing

As mentioned above, the new Program.out allows for calibrations to
be excluded (default) or included in a printed program. This is used
at some places when constructing payloads for RPCQ calls.

TODO: Explicitly handle Quil vs Quilt cases in appropriate RPCQ calls,
e.g. quil_to_native_quil. Right now things are a bit ad-hoc.

Changes to vanilla PyQuil:

• (Backwards compatible) generate_rb_sequence now adds calibrations
  from the interleaving gate program to the resulting Clifford program

See:

• pyquil/api/_qvm.py (excluded calibrations from QVM payloads)
• pyquil/api/_base_connect.py (exclude calibrations from payloads)
• pyquil/api/_benchmark.py (exclude calibrations from payloads,
  inherit calibrations when constructing RB sequences)
• pyquil/api/_compiler.py (exclude calibrations in
  quil_to_native_quil call)

[notmgsk]

@kilimanjaro Over in the quilt-demo-types-etc branch I've made some changes that I'd like you to look through. Mostly it is just correcting type hints, but I'd like you to go through commits (which should be self-contained) and see if I'm missing or misunderstanding something. I think the meat of the changes are due to the FormalArgument additions.

Some questions:

• Do Quilt waveforms map directly onto the RPCQ waveforms?
• In RPCQ, TemplateWaveform defines all of scale, phase, detuning, and duration, but in pyquil TemplateWaveform only defines duration and then most subclasses define the rest. That's ok but seems like a lot of duplication and an opportunity for stuff to get out of sync. Can we move those properties into TemplateWaveform? Doing that would also fix the issue in TemplateWaveform._update_envelope where it is possible that self.scale etc. may be undefined (rather than just having a default of None).
  • After moving those properties into TemplateWaveform I see the issue -- if a dataclass's parent defines optional properties then the child class cannot define non-optional properties. Funky.
• Is this a no-op?

[notmgsk]

• TODO Make sure QVMCompiler has the same interface (where reasonable) as QPUCompiler

[braised-babbage]

@kilimanjaro Over in the quilt-demo-types-etc branch I've made some changes that I'd like you to look through. Mostly it is just correcting type hints, but I'd like you to go through commits (which should be self-contained) and see if I'm missing or misunderstanding something. I think the meat of the changes are due to the FormalArgument additions.

I went through this and think it looks fine. As I wrote it, wasn't sure exactly what to do with FormalArgument, since it is now legal in many existing Quil instructions if they are in the body of a calibration definition. As it stands I think including them in with QubitDesignator is fine, although outside of the body of a calibration definition it is a syntax error to use FormalArgument instead of Qubit or QubitPlaceholder.

I also noticed that this comment is hanging

pyquil/pyquil/gates.py

Line 1061 in 500dd63

# Mypy doesn't support a complex type hint here on args. Particularly,

• Do Quilt waveforms map directly onto the RPCQ waveforms?

Yes, with the only bit of fiddling being related to the meaning of the units involved (e.g. in Quilt phases are radians, in RPCQ phases are in revolutions aka 'units of tau').

• In RPCQ, TemplateWaveform defines all of scale, phase, detuning, and duration, but in pyquil TemplateWaveform only defines duration and then most subclasses define the rest. That's ok but seems like a lot of duplication and an opportunity for stuff to get out of sync. Can we move those properties into TemplateWaveform? Doing that would also fix the issue in TemplateWaveform._update_envelope where it is possible that self.scale etc. may be undefined (rather than just having a default of None).

  • After moving those properties into TemplateWaveform I see the issue -- if a dataclass's parent defines optional properties then the child class cannot define non-optional properties. Funky.

The issue is as you mentioned. I'm not happy about having the code duplication of those fields across all the individual waveform classes, but the alternative seemed to be jettisoning the use of dataclasses, which would require its own form of boilerplate.

• Is this a no-op?
  This link is broken for me.

[notmgsk]

@kilimanjaro oopsie: https://github.com/rigetti/pyquil/blob/quilt-demo/pyquil/quiltwaveforms.py#L54

[notmgsk]

Coolbeans. Are you happy with me merging that branch into this one?

[braised-babbage]

https://github.com/rigetti/pyquil/blob/quilt-demo/pyquil/quiltwaveforms.py#L54

Ah, that actually does something: it evaluates arithmetic expressions. So what is syntactically as 2+3-4 is reduced to 1, and then the later check only needs to distinguish real from complex from other (if there was a memory reference in the original expression).

[braised-babbage]

Coolbeans. Are you happy with me merging that branch into this one?

Yes, please do so.

[notmgsk]

• TODO get_quilt_calibrations is wonderfully (woefully?) slow