Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

don't allow ? in non-result v2 benchmark function defs, require either Result or () return type #13277

Closed

Conversation

sam0x17
Copy link
Contributor

@sam0x17 sam0x17 commented Jan 30, 2023

Changes the v2 benchmarking syntax to only accept () or Result<(), BenchmarkError> as valid return types on benchmark function defs. This change is merely syntactic sugar as the return type is ultimately discarded since the code in these function defs is loaded into a bunch of trait impls.

  • Pallet using this syntax (balances)
  • Parsing of valid return types
  • Compile error messages for valid return types
  • UI tests for valid return types
  • Compile error when there is a ? operator but return type is ()
  • UI test for ? operator
  • use alias to BenchmarkResult and simplified parsing
  • update UI tests for new simplified parsing
  • Update docs

Related Issues

@github-actions github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Jan 30, 2023
@sam0x17 sam0x17 self-assigned this Jan 30, 2023
@sam0x17 sam0x17 force-pushed the sam-explicit-return-type-on-benchmarking-function-defs branch from 2def575 to 546bc07 Compare January 31, 2023 07:11
@sam0x17 sam0x17 added D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit C1-low PR touches the given topic and has a low impact on builders. labels Jan 31, 2023
@sam0x17 sam0x17 changed the title require a Result return type when using ? operator in benchmarking functions don't allow ? in non-result benchmark function defs, require either Result or () return type Jan 31, 2023
@sam0x17 sam0x17 changed the title don't allow ? in non-result benchmark function defs, require either Result or () return type don't allow ? in non-result v2 benchmark function defs, require either Result or () return type Jan 31, 2023
@sam0x17 sam0x17 marked this pull request as ready for review February 1, 2023 00:05
@sam0x17 sam0x17 requested a review from ggwpez February 1, 2023 00:05
@github-actions github-actions bot added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Feb 1, 2023
frame/balances/src/benchmarking.rs Show resolved Hide resolved
frame/support/procedural/src/benchmark.rs Outdated Show resolved Hide resolved
frame/support/procedural/src/benchmark.rs Outdated Show resolved Hide resolved
frame/support/procedural/src/benchmark.rs Show resolved Hide resolved
@sam0x17
Copy link
Contributor Author

sam0x17 commented Feb 1, 2023

So I am going to re-work this a bit to use static_assertions::assert_type_eq_all! and an alias called BenchmarkResult that points to Result<(), BenchmarkError>, which will also vastly simplify the code and the UX

@sam0x17 sam0x17 requested a review from ggwpez February 1, 2023 23:25
// check for question mark ops as these aren't allowed here
let mut visitor = ExprTryVisitor::new();
visitor.visit_item_fn(item_fn);
if let Some(expr) = visitor.results.first() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its good to have the syntax clarification, but not sure if this check here actually works.
It would still be possible to just return Err("".into()) instead of using ? and it would exclude #[block]s from using it.
I dont think we can/have to explicitly check this, or?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have a visitor that checks all return statements and applies a static_assertions::assert_type_eq_all! check to each one verifying that it is compatible with BenchmarkResult, and that wouldn't be too hard, but honestly now that BenchmarkResult is so short I am leaning towards just requiring -> BenchmarkResult on all benchmark function defs, which would greatly simplify things and wouldn't be a very heavy lift. Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way would work though I think

@sam0x17
Copy link
Contributor Author

sam0x17 commented Feb 2, 2023

Based on my discussion with @ggwpez I think the best course of action is to actually close this and re-open #13224. There are some clear benefits to generating real functions and one of those benefits is all these weird edge cases will be resolved for free. Additionally some issues, like not using a benchmark parameter within the benchmarking code, will now get warnings that will cause CI to fail (as it should) instead of us having to write crazy visitor patterns and things to identify these ourselves.

So we've come full circle, #13224 is a good idea 😆

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

benchmarking v2 syntax is confusingly lax about return types, leading to confusion
2 participants