Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vm] benchmark transaction scripts gas consumption #1898

Merged
merged 3 commits into from
Dec 30, 2020
Merged

Conversation

guangyuz
Copy link
Contributor

No description provided.

@guangyuz guangyuz changed the title [vm] add functional tests for transaction scripts gas consumption [vm] benchmark transaction scripts gas consumption Dec 29, 2020
@github-actions
Copy link

Benchmark for 50a9275

Click to view benchmark
Test PR Benchmark Master Benchmark %
accumulator_append 1544.4±322.27µs 1504.5±161.88µs +2.65%
block_apply/block_apply_10 428.6±19.07ms 436.0±43.75ms -1.70%
block_apply/block_apply_1000 49.1±8.27s 46.7±5.17s +5.14%
get_with_proof/db_store 49.1±2.13µs 44.5±1.27µs +10.34%
get_with_proof/mem_store 31.8±0.75µs 28.8±0.76µs +10.42%
put_and_commit/db_store/1 739.5±42.37µs 764.8±120.81µs -3.31%
put_and_commit/db_store/10 2.3±0.34ms 2.3±0.23ms 0.00%
put_and_commit/db_store/100 14.9±2.34ms 13.4±0.84ms +11.19%
put_and_commit/db_store/5 1427.2±98.90µs 1528.9±513.15µs -6.65%
put_and_commit/db_store/50 7.7±1.07ms 7.5±0.50ms +2.67%
put_and_commit/mem_store/1 121.8±8.64µs 110.6±6.98µs +10.13%
put_and_commit/mem_store/10 1078.1±61.71µs 969.6±54.89µs +11.19%
put_and_commit/mem_store/100 10.6±2.25ms 9.6±0.81ms +10.42%
put_and_commit/mem_store/5 544.2±37.47µs 500.2±30.40µs +8.80%
put_and_commit/mem_store/50 5.3±0.89ms 4.9±0.48ms +8.16%
query_block/query_block_in(10)_times(100) 1328.8±10.24µs 1255.1±8.83µs +5.87%
query_block/query_block_in(10)_times(1000) 13.3±0.11ms 12.5±0.17ms +6.40%
query_block/query_block_in(10)_times(10000) 132.5±0.82ms 125.5±0.90ms +5.58%
query_block/query_block_in(1000)_times(100) 1346.6±11.08µs 1220.4±6.79µs +10.34%
query_block/query_block_in(1000)_times(1000) 13.4±0.03ms 12.1±0.08ms +10.74%
query_block/query_block_in(1000)_times(10000) 133.3±0.88ms 120.7±1.16ms +10.44%
storage_transaction 19.2±1.45ms 19.5±1.51ms -1.54%
vm/transaction_execution/1 373.3±24.69ms 368.8±4.27ms +1.22%
vm/transaction_execution/10 131.6±7.43ms 118.7±1.26ms +10.87%
vm/transaction_execution/20 113.9±1.83ms 110.7±0.73ms +2.89%
vm/transaction_execution/5 151.4±11.62ms 142.9±0.81ms +5.95%
vm/transaction_execution/50 141.0±12.88ms 132.1±1.47ms +6.74%

@codecov
Copy link

codecov bot commented Dec 30, 2020

Codecov Report

Merging #1898 (458eadf) into master (1152ca5) will decrease coverage by 0.13%.
The diff coverage is 3.84%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1898      +/-   ##
==========================================
- Coverage   29.59%   29.46%   -0.14%     
==========================================
  Files         432      432              
  Lines       44064    44085      +21     
  Branches    20311    20320       +9     
==========================================
- Hits        13040    12988      -52     
- Misses      17405    17545     +140     
+ Partials    13619    13552      -67     
Flag Coverage Δ
unittests 29.46% <3.84%> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
vm/functional-tests/src/evaluator.rs 5.43% <0.00%> (-0.34%) ⬇️
vm/vm-runtime/src/starcoin_vm.rs 26.29% <8.33%> (-0.47%) ⬇️
rpc/server/src/module/mod.rs 0.00% <0.00%> (-18.75%) ⬇️
vm/types/src/transaction/error.rs 3.77% <0.00%> (-15.10%) ⬇️
types/src/error/mod.rs 0.00% <0.00%> (-6.25%) ⬇️
vm/vm-runtime/src/data_cache.rs 62.96% <0.00%> (-3.71%) ⬇️
...2p/src/protocol/generic_proto/handler/notif_out.rs 34.94% <0.00%> (-3.23%) ⬇️
network-p2p/src/peer_info.rs 39.45% <0.00%> (-2.17%) ⬇️
cmd/starcoin/src/dev/sign_txn_helper.rs 14.12% <0.00%> (-1.86%) ⬇️
...rk-p2p/src/protocol/generic_proto/handler/group.rs 32.25% <0.00%> (-1.44%) ⬇️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1152ca5...458eadf. Read the comment docs.

@github-actions
Copy link

Benchmark for b3e2f9b

Click to view benchmark
Test PR Benchmark Master Benchmark %
accumulator_append 1525.3±321.82µs 1441.8±185.51µs +5.79%
block_apply/block_apply_10 390.1±18.25ms 381.6±5.27ms +2.23%
block_apply/block_apply_1000 40.2±0.29s 39.7±0.74s +1.26%
get_with_proof/db_store 45.3±0.48µs 45.3±0.77µs 0.00%
get_with_proof/mem_store 29.3±0.33µs 29.4±0.29µs -0.34%
put_and_commit/db_store/1 702.1±168.61µs 696.9±128.88µs +0.75%
put_and_commit/db_store/10 2.5±0.88ms 2.4±0.71ms +4.17%
put_and_commit/db_store/100 14.5±2.89ms 14.6±2.67ms -0.68%
put_and_commit/db_store/5 1512.7±236.51µs 1456.7±240.46µs +3.84%
put_and_commit/db_store/50 8.3±2.81ms 8.0±1.49ms +3.75%
put_and_commit/mem_store/1 109.4±6.59µs 108.9±6.55µs +0.46%
put_and_commit/mem_store/10 971.6±58.98µs 961.6±52.94µs +1.04%
put_and_commit/mem_store/100 9.6±0.77ms 9.6±0.75ms 0.00%
put_and_commit/mem_store/5 498.2±28.50µs 494.7±28.40µs +0.71%
put_and_commit/mem_store/50 4.8±0.38ms 4.8±0.37ms 0.00%
query_block/query_block_in(10)_times(100) 1225.7±6.37µs 1223.4±10.70µs +0.19%
query_block/query_block_in(10)_times(1000) 12.3±0.11ms 12.4±0.09ms -0.81%
query_block/query_block_in(10)_times(10000) 123.6±0.59ms 121.7±0.55ms +1.56%
query_block/query_block_in(1000)_times(100) 1188.4±3.52µs 1181.8±1.08µs +0.56%
query_block/query_block_in(1000)_times(1000) 11.9±0.06ms 12.0±0.07ms -0.83%
query_block/query_block_in(1000)_times(10000) 119.7±0.95ms 119.4±0.79ms +0.25%
storage_transaction 18.4±4.19ms 16.5±3.30ms +11.52%
vm/transaction_execution/1 361.4±2.54ms 360.4±0.99ms +0.28%
vm/transaction_execution/10 116.7±0.37ms 117.4±0.97ms -0.60%
vm/transaction_execution/20 110.3±0.63ms 110.2±0.64ms +0.09%
vm/transaction_execution/5 141.2±0.61ms 142.4±0.36ms -0.84%
vm/transaction_execution/50 130.7±2.97ms 129.3±0.35ms +1.08%

@guangyuz guangyuz merged commit a58bfb8 into master Dec 30, 2020
@guangyuz guangyuz deleted the gas_bench branch December 30, 2020 11:10
@guangyuz guangyuz mentioned this pull request Jan 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants