Feature request: specify `resolve` for user-defined benchmarks.

[oraluben]

pyperformance itself uses resolve_default_benchmark to set version of every benchmark to the version of the package.
For other projects who want to define their own benchmarks, this could also be helpful, so such projects don't need to update the version field every time, however, I didn't find any mechanism that can be used for that.

Looking forward to your opinions.

[ericsnowcurrently]

You can add a "version" field to the [project] section of the benchmark's pyproject.toml. However, this isn't so helpful if you have many benchmarks or if you want the benchmark version to match the version of the project to which the benchmarks belong. That's basically the motivation behind the mechanism in resolve_default_benchmark() that you noted.

There is a mechanism available. A benchmark can inherit metadata through the "inherits" field in the [tool.pypeformance] section of the benchmark's pyproject.toml. (The code is in _resolve_base() in pyperformance/_benchmark_metadata.py.)

For example, set "inherits: .." and that benchmark will inherit metadata from ../base.toml. (See https://github.com/pyston/python-macrobenchmarks for an example.) Then you could set the version in that one file. IIRC I added support for a couple other cases. In fact, you should be able to inherit from the top-level project's pyproject.toml.

(I didn't test that machinery very thoroughly, so there may be bugs.)

[oraluben]

Thanks @ericsnowcurrently! The inherits mechanism does meet my requirement.

However I do spot an issue that blocks the usage of inherits = "..".
I add a related test case in #141, in case you're interested.

[oraluben]

I'm closing this for the original question has been answered, the broken inherits is a different case.