On demand pull request benchmark

[trexfeathers]

🚀 Pull Request

Description

Enhances the benchmark-check GHA workflow to allow developers to request that their pull request be benchmarked for performance shifts. This is achieved by adding the benchmark_this Request that this pull request be benchmarked to check if it introduces performance shifts label.

Demonstrations

• On demand benchmark demo trexfeathers/iris#48
• On demand benchmark demo: skip trexfeathers/iris#49
• On demand benchmark demo: non-PR triggers trexfeathers/iris#50

TODO:

• What's New

---

Consult Iris pull request check list

[trexfeathers]

This is ready to go now ✔

[lbdreyer]

All looks good and I can see this being very useful!

[trexfeathers]

Many thanks @lbdreyer!

[lbdreyer]

It looks like the overnight benchmark failed due to exceeding the allowed time (6 hours):

Usually benchmarks seem to take 1 or 2 hours

[trexfeathers]

This is actually un-related. It looks like it was confused by the merge commit from v3.5.x - this branch originated 29 commits ago, so it tried to work through all of them (see this line in the workflow run).

I don't currently have a solution for this. It's down to the git intelligence in the following places:

• The benchmark-check workflow
• ASV get_hashes_from_range()

At least it only happens during merge-backs to main.

[trexfeathers]

... It's down to the git intelligence in the following places:

• The benchmark-check workflow
• ASV get_hashes_from_range()

Also this block in bm_runner.py