From 3fc442792132a78f2414e8fa48774695b50fd82b Mon Sep 17 00:00:00 2001 From: Frank Harkins Date: Tue, 14 May 2024 10:45:58 +0100 Subject: [PATCH 1/5] Ask contributors to use `tox` for final notebook execution. --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 2775f65fea7..afcd1a03295 100644 --- a/README.md +++ b/README.md @@ -103,6 +103,9 @@ This staging environment can be useful to see how the docs are rendering before ## Execute notebooks +Before submitting a new notebook or code changes to a notebook, you must run the notebook using `tox -- --write ` and commit the results. +This means we can be sure all notebooks work and that users will see the same results when they run using the environment we recommend. + To execute notebooks in a fixed Python environment, first install `tox` using [pipx](https://pipx.pypa.io/stable/): From 65c6b88d882557ec33270b94a08c7777c84fff66 Mon Sep 17 00:00:00 2001 From: Frank Harkins Date: Tue, 14 May 2024 12:29:04 +0100 Subject: [PATCH 2/5] Update README.md Co-authored-by: Eric Arellano <14852634+Eric-Arellano@users.noreply.github.com> --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index afcd1a03295..8d2cccb7c69 100644 --- a/README.md +++ b/README.md @@ -103,8 +103,7 @@ This staging environment can be useful to see how the docs are rendering before ## Execute notebooks -Before submitting a new notebook or code changes to a notebook, you must run the notebook using `tox -- --write ` and commit the results. -This means we can be sure all notebooks work and that users will see the same results when they run using the environment we recommend. +Before submitting a new notebook or code changes to a notebook, you must run the notebook using `tox -- --write ` and commit the results. (If the notebook submits jobs, also use the argument `--submit-jobs`). This means we can be sure all notebooks work and that users will see the same results when they run using the environment we recommend. To execute notebooks in a fixed Python environment, first install `tox` using [pipx](https://pipx.pypa.io/stable/): From 966d32fe2fd424ca09de252b2b7b9c5406cb98ef Mon Sep 17 00:00:00 2001 From: Frank Harkins Date: Tue, 14 May 2024 12:35:55 +0100 Subject: [PATCH 3/5] Remove duplicated text and neaten --- README.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 8d2cccb7c69..d63d5589b20 100644 --- a/README.md +++ b/README.md @@ -103,7 +103,11 @@ This staging environment can be useful to see how the docs are rendering before ## Execute notebooks -Before submitting a new notebook or code changes to a notebook, you must run the notebook using `tox -- --write ` and commit the results. (If the notebook submits jobs, also use the argument `--submit-jobs`). This means we can be sure all notebooks work and that users will see the same results when they run using the environment we recommend. +Before submitting a new notebook or code changes to a notebook, you must run +the notebook using `tox -- --write ` and commit the results. +If the notebook submits jobs, also use the argument `--submit-jobs`. This means +we can be sure all notebooks work and that users will see the same results when +they run using the environment we recommend. To execute notebooks in a fixed Python environment, first install `tox` using [pipx](https://pipx.pypa.io/stable/): @@ -112,7 +116,10 @@ To execute notebooks in a fixed Python environment, first install `tox` using pipx install tox ``` -You also need to install a few system dependencies: TeX, Poppler, and graphviz. On macOS, you can run `brew install mactex-no-gui poppler graphviz`. On Ubuntu, you can run `apt-get install texlive-pictures texlive-latex-extra poppler-utils graphviz`. +You also need to install a few system dependencies: TeX, Poppler, and graphviz. +On macOS, you can run `brew install mactex-no-gui poppler graphviz`. On Ubuntu, +you can run `apt-get install texlive-pictures texlive-latex-extra poppler-utils +graphviz`. - To execute all notebooks, run tox. ```sh @@ -130,20 +137,20 @@ You also need to install a few system dependencies: TeX, Poppler, and graphviz. > [!NOTE] > If your notebook submits hardware jobs to Qiskit Runtime, you must add it to > [`scripts/nb-tester/notebooks.toml`](scripts/nb-tester/notebooks.toml). If it -> can be run with simulators, i.e., the circuit is not too large, add it to `notebooks_that_submit_jobs`. -> Otherwise, add it to `notebooks_no_mock`. +> can be run with simulators, i.e., the circuit is not too large, add it to +> `notebooks_that_submit_jobs`. Otherwise, add it to `notebooks_no_mock`. > If your notebook uses the latex circuit drawer (`qc.draw("latex")`), you must > add it to the "Check for notebooks that require LaTeX" step in > `.github/workflows/notebook-test.yml`. When you make a pull request changing a notebook that doesn't submit jobs, you -can get a version of that notebook that was executed in a uniform environment -from CI. To do this, click "Show all checks" in the info box at the bottom of -the pull request page on GitHub, then choose "Details" for the "Test notebooks" -job. From the job page, click "Summary", then download "Executed notebooks". -Otherwise, if your notebook does submit jobs, you need to run it locally with -`tox -- --write --submit-jobs `. +can get a version of that notebook that was executed by tox from CI. To do +this, click "Show all checks" in the info box at the bottom of the pull request +page on GitHub, then choose "Details" for the "Test notebooks" job. From the +job page, click "Summary", then download "Executed notebooks". Otherwise, if +your notebook does submit jobs, you need to run it locally using the steps +mentioned earlier. ### Ignoring warnings From 303c859e8c976e8ae76fc5a75bf67ae738a4a5ab Mon Sep 17 00:00:00 2001 From: Frank Harkins Date: Tue, 14 May 2024 13:34:59 +0100 Subject: [PATCH 4/5] Update section on notebooks.toml --- README.md | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index d63d5589b20..1b7167ca1b0 100644 --- a/README.md +++ b/README.md @@ -134,16 +134,6 @@ graphviz`. tox -- optional/paths/to/notebooks.ipynb --write ``` -> [!NOTE] -> If your notebook submits hardware jobs to Qiskit Runtime, you must add it to -> [`scripts/nb-tester/notebooks.toml`](scripts/nb-tester/notebooks.toml). If it -> can be run with simulators, i.e., the circuit is not too large, add it to -> `notebooks_that_submit_jobs`. Otherwise, add it to `notebooks_no_mock`. - -> If your notebook uses the latex circuit drawer (`qc.draw("latex")`), you must -> add it to the "Check for notebooks that require LaTeX" step in -> `.github/workflows/notebook-test.yml`. - When you make a pull request changing a notebook that doesn't submit jobs, you can get a version of that notebook that was executed by tox from CI. To do this, click "Show all checks" in the info box at the bottom of the pull request @@ -152,6 +142,30 @@ job page, click "Summary", then download "Executed notebooks". Otherwise, if your notebook does submit jobs, you need to run it locally using the steps mentioned earlier. +### Adding a new notebook + +When adding a new notebook, you'll need to tell the testing tools how to handle it. +To do this, add the file path to `scripts/nb-tester/notebooks.toml`. There are +four categories: + +* `notebooks_normal_test`: Notebooks to be run normally in CI. These notebooks + can't submit jobs as the queue times are too long and it will waste + resources. You _can_ interact with IBM Quantum to retrieve jobs and backend + information. +* `notebooks_that_submit_jobs`: Notebooks that submit jobs, but that are small + enough to run on a 5-qubit simulator. We will test these notebooks in CI by + patching `least_busy` to return a 5-qubit fake backend. +* `notebooks_no_mock`: For notebooks that can't be tested using the 5-qubit + simulator patch. We skip testing these in CI and instead run them twice per + month. Any notebooks with cells that take more than five minutes to run are + also deemed too big for CI. Try to avoid adding notebooks to this category if + possible. +* `notebooks_exclude`: Notebooks to be ignored. + +If your notebook uses the latex circuit drawer (`qc.draw("latex")`), you must +also add it to the "Check for notebooks that require LaTeX" step in +`.github/workflows/notebook-test.yml`. + ### Ignoring warnings We don't want users to see warnings that can be avoided, so it's best to fix From 2b2df9fb949a73042968174f1dfcaa2c18f8a1ea Mon Sep 17 00:00:00 2001 From: Frank Harkins Date: Tue, 14 May 2024 13:50:27 +0100 Subject: [PATCH 5/5] lint --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 1b7167ca1b0..50ded01f176 100644 --- a/README.md +++ b/README.md @@ -148,19 +148,19 @@ When adding a new notebook, you'll need to tell the testing tools how to handle To do this, add the file path to `scripts/nb-tester/notebooks.toml`. There are four categories: -* `notebooks_normal_test`: Notebooks to be run normally in CI. These notebooks +- `notebooks_normal_test`: Notebooks to be run normally in CI. These notebooks can't submit jobs as the queue times are too long and it will waste resources. You _can_ interact with IBM Quantum to retrieve jobs and backend information. -* `notebooks_that_submit_jobs`: Notebooks that submit jobs, but that are small +- `notebooks_that_submit_jobs`: Notebooks that submit jobs, but that are small enough to run on a 5-qubit simulator. We will test these notebooks in CI by patching `least_busy` to return a 5-qubit fake backend. -* `notebooks_no_mock`: For notebooks that can't be tested using the 5-qubit +- `notebooks_no_mock`: For notebooks that can't be tested using the 5-qubit simulator patch. We skip testing these in CI and instead run them twice per month. Any notebooks with cells that take more than five minutes to run are also deemed too big for CI. Try to avoid adding notebooks to this category if possible. -* `notebooks_exclude`: Notebooks to be ignored. +- `notebooks_exclude`: Notebooks to be ignored. If your notebook uses the latex circuit drawer (`qc.draw("latex")`), you must also add it to the "Check for notebooks that require LaTeX" step in