Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Increase contracts bench stability #13581

Closed
wants to merge 5 commits into from
Closed

Increase contracts bench stability #13581

wants to merge 5 commits into from

Conversation

ggwpez
Copy link
Member

@ggwpez ggwpez commented Mar 10, 2023 •
edited
Loading

The FRAME benchmarking does not handle the case well where a linear component as only a range of [0, =1] since in this case it has to calculate a linear formula from just two points which is unstable.

Now bumping these upper limits back API_BENCHMARK_BATCHES should improve stability. There are some more benches which could profit from this, but let's first see if we go into the right direction by re-benching.

This partially reverts #12526

ggwpez added 2 commits March 10, 2023 14:37
@ggwpez
Bump contract benches repetition
5ae22be 
The FRAME benchmarking does not handle the case well where a linear
component as only a range of [0, =1] since in this case it has to
calculate a linear formula from just two points which is unstable.

Now bumping these upper limits back API_BENCHMARK_BATCHES should
improve stability.

This partially reverts 2eb8ad2

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez
Bump some more
e023626 
These also show up as asymptotically unstable. There are some more,
but let's see first if we go into the right direction.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez
Revert "Bump some more"
025a934 
This reverts commit e023626.
@ggwpez
Copy link
Member Author

ggwpez commented Mar 10, 2023

bot clean
bot bench $ pallet dev pallet_contracts
bot bench $ pallet dev pallet_contracts

@command-bot
Copy link

command-bot bot commented Mar 10, 2023 •
edited
Loading

@ggwpez https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2508367 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 38-7c808830-dd44-4959-9cef-31b6e861ada1 to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link

command-bot bot commented Mar 10, 2023 •
edited
Loading

@ggwpez https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2508368 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 39-edcd45ac-0d0d-4228-b187-0e18a123c0bf to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link

command-bot bot commented Mar 10, 2023

@ggwpez Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2508367 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2508367/artifacts/download.

@command-bot
Copy link

command-bot bot commented Mar 10, 2023

@ggwpez Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2508368 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2508368/artifacts/download.

@ggwpez
Copy link
Member Author

ggwpez commented Mar 13, 2023 •
edited
Loading

Looks much better now (SWC link). Note that it is using the Asymptotic mode to mostly ignore the base-weight, since for the contracts it does not really matter.
image

On Exact-worst it shows the ones which have increase base-weight; as said, should not matter here.
image

@ggwpez ggwpez marked this pull request as ready for review March 13, 2023 13:52
@athei athei mentioned this pull request Mar 14, 2023
Merged
@ggwpez
Copy link
Member Author

ggwpez commented Mar 15, 2023

Closing in favour of #13595

@ggwpez ggwpez closed this Mar 15, 2023
@ggwpez ggwpez deleted the oty-bump-contract-bench-reps branch April 24, 2023 12:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@athei athei athei left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ggwpez @athei