🐛 Cannot train RL model on oqc_lucy

[qubitzer]

Environment information

• OS: Windows 11
• Python: 3.10.9
• mqt.predictor: 2.1.1

Entire python environment:
absl-py==2.1.0
ale-py==0.10.1
annotated-types==0.7.0
asttokens==3.0.0
beautifulsoup4==4.12.3
bqskit==1.2.0
bqskitrs==0.4.1
certifi==2024.12.14
cffi==1.17.1
charset-normalizer==3.4.1
cloudpickle==3.1.1
colorama==0.4.6
comm==0.2.2
contourpy==1.3.1
cryptography==44.0.0
cycler==0.12.1
debugpy==1.8.12
decorator==5.1.1
dill==0.3.9
docplex==2.29.241
exceptiongroup==1.2.2
executing==2.2.0
Farama-Notifications==0.0.4
fastdtw==0.3.4
filelock==3.17.0
fonttools==4.55.4
frozendict==2.4.6
fsspec==2024.12.0
graphviz==0.20.3
grpcio==1.69.0
gymnasium==1.0.0
h5py==3.12.1
html5lib==1.1
ibm-cloud-sdk-core==3.22.1
ibm-platform-services==0.59.1
idna==3.10
inflection==0.5.1
ipykernel==6.29.5
ipython==8.31.0
ipywidgets==8.1.5
jedi==0.19.2
Jinja2==3.1.5
joblib==1.4.2
jupyter_client==8.6.3
jupyter_core==5.7.2
jupyterlab_widgets==3.0.13
kiwisolver==1.4.8
lark==1.2.2
lxml==5.3.0
Markdown==3.7
markdown-it-py==3.0.0
MarkupSafe==3.0.2
matplotlib==3.10.0
matplotlib-inline==0.1.7
mdurl==0.1.2
more-itertools==10.6.0
mpmath==1.3.0
mqt.bench==1.1.9
mqt.predictor==2.1.1
multitasking==0.0.11
Nasdaq-Data-Link==1.0.4
nest-asyncio==1.6.0
networkx==3.4.2
numpy==1.26.4
opencv-python==4.11.0.86
packaging==24.2
pandas==2.2.3
parso==0.8.4
pbr==6.1.0
peewee==3.17.8
pillow==11.1.0
platformdirs==4.3.6
prompt_toolkit==3.0.50
protobuf==5.29.3
psutil==6.1.1
pure_eval==0.2.3
pycparser==2.22
pydantic==2.9.2
pydantic_core==2.23.4
pygame==2.6.1
Pygments==2.19.1
PyJWT==2.10.1
pyparsing==3.2.1
pyspnego==0.11.2
python-dateutil==2.9.0.post0
pytket==1.39.0
pytket-qiskit==0.62.0
pytz==2024.2
pywin32==308
pyzmq==26.2.0
qiskit==1.3.2
qiskit-aer==0.16.0
qiskit-algorithms==0.3.1
qiskit-finance==0.4.1
qiskit-ibm-runtime==0.34.0
qiskit-nature==0.7.2
qiskit-optimization==0.6.1
qwasm==1.0.1
requests==2.32.3
requests_ntlm==1.3.0
rich==13.9.4
rustworkx==0.15.1
sb3_contrib==2.4.0
scikit-learn==1.6.1
scipy==1.15.1
six==1.17.0
soupsieve==2.6
sspilib==0.2.0
stable_baselines3==2.4.1
stack-data==0.6.3
stevedore==5.4.0
symengine==0.13.0
sympy==1.13.1
tensorboard==2.18.0
tensorboard-data-server==0.7.2
threadpoolctl==3.5.0
torch==2.5.1
tornado==6.4.2
tqdm==4.67.1
traitlets==5.14.3
typing_extensions==4.12.2
tzdata==2025.1
urllib3==2.3.0
wcwidth==0.2.13
webencodings==0.5.1
websocket-client==1.8.0
Werkzeug==3.1.3
widgetsnbextension==4.0.13
yfinance==0.2.52

Description

When trying to run

import mqt.predictor

rl_pred = mqt.predictor.rl.Predictor(
    figure_of_merit="expected_fidelity", device_name="oqc_lucy"
)
rl_pred.train_model(timesteps=1024, model_name="sample_model_rl")

I run into "ValueError: Two-qubit fidelity for gate ecr and qubits 6 and 7 not available."

Expected behavior

No response

How to Reproduce

Execute mentioned code in given python env.

[nquetschlich]

Thank you very much for raising this issue as well, @qubitzer. I will investigate the cause for this and come back to you afterwards.

[nquetschlich]

I now investigated and the error is the following: The OQC Lucy device has a connectivity that is unidirectional, e.g., the directionality of a two-qubit gate applied to two connected physical qubits matters and only on direction is valid.
Due to reasons I do not fully understand yet, the RL compiler is in some cases not able to match this unidirectional connectivity and assumes bidirectional community. This leads to compiled quantum circuit with invalid gates---causing the error you have experienced.

While I will try to fix that issue as soon as possible, I also created a PR (#359) that simplifies the usage of the ML part since it now allows to specific a custom list of devices instead of using all supported ones from MQT Bench:

import mqt.predictor
ml_pred = mqt.predictor.ml.Predictor(devices=["ibm_washington", "ibm_montreal"], figure_of_merit="expected_fidelity")

ml_pred.generate_compiled_circuits(timeout=600) 
training_data, name_list, scores_list = ml_pred.generate_trainingdata_from_qasm_files()
mqt.predictor.ml.helper.save_training_data(training_data, name_list, scores_list, figure_of_merit="expected_fidelity")
ml_pred.train_random_forest_classifier()

Please feel free to check out that branch, I will continue to work on it tomorrow.