Skip to content
This repository was archived by the owner on Nov 30, 2021. It is now read-only.

Benchmarking Tooling #399

Closed
wants to merge 48 commits into from
Closed

Benchmarking Tooling #399

wants to merge 48 commits into from

Conversation

@araskachoi
Copy link
Contributor

@araskachoi araskachoi commented Jul 21, 2020 •
edited
Loading

example benchmark run:
in the ./docker/benchmark directory, run bash test.sh 5000

  • this will send 5000 transactions (will keep running til 5000 transactions have been sent)
  • will result in files that are generated
    • emintcli.log (logs of the output of emintcli process)
    • emintd.log (logs of the output of emintd process)
    • hashes.json (hashes of all the transactions that were sent)
    • receipts.json (receipt results of the transactions from chain)
    • resource.log (resource consumption logs of emintd and emintcli processes)
  • output will give metrics
    example output:
start time set:  1595299469
end time set:  1595299503
Average CPU Usage [emintd]:  30.83235294117647
Average RAM Usage [emintd]:  5.2823529411764705
Average CPU Usage [emintcli]:  37.126470588235286
Average RAM Usage [emintcli]:  2.667647058823529
1595299499
1595299469
TX per second (TPS):  166.66666666666666
Blocks with Tx:  [0x3 0x4 0x5 0x6 0x7 0x8 0x9]
Block Timestamps:  [1595299469 1595299474 1595299479 1595299484 1595299489 1595299494 1595299499 1595299464]
Transactions:  [801 729 729 724 738 753 526]
Total Transactions:  5000

PLEASE DO NOT MERGE!

this PR has a change in the antehandler where it will bypass the exactness (ordering) of the nonce. This is required for sending transactions rapidly.
The out-of-sequence nonce checking has been included in https://github.com/ChainSafe/ethermint/pull/550

Closes: #XXX

Description

For contributor use:

  • Targeted PR against correct branch (see CONTRIBUTING.md)
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Code follows the module structure standards.
  • Wrote unit and integration tests
  • Updated relevant documentation (docs/) or specification (x/<module>/spec/)
  • Added relevant godoc comments.
  • Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer

For admin use:

  • Added appropriate labels to PR (ex. WIP, R4R, docs, etc)
  • Reviewers assigned
  • Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)

@codecov
Copy link

codecov bot commented Jul 21, 2020 •
edited
Loading

Codecov Report

Merging #399 into development will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##           development     #399   +/-   ##
============================================
  Coverage        70.71%   70.71%           
============================================
  Files               42       42           
  Lines             2339     2339           
============================================
  Hits              1654     1654           
  Misses             541      541           
  Partials           144      144

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e01da9...0427010. Read the comment docs.

fedekunze
fedekunze reviewed Aug 12, 2020
View reviewed changes
Copy link
Contributor

@fedekunze fedekunze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Will test tomorrow morning. Pending lint fixes

docker/benchmarking/benchmark/geth-genesis.go
"github.com/urfave/cli"
)

type Genesis struct {
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit add godoc comment

docker/benchmarking/benchmark/geth-genesis.go Outdated
)

type Genesis struct {
Config struct {
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of the struct and pass it as a field

docker/benchmarking/benchmark/geth-genesis.go
Difficulty string `json:"difficulty"`
GasLimit string `json:"gasLimit"`
ExtraData string `json:"extraData"`
Alloc interface{} `json:"alloc"`
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

docker/benchmarking/benchmark/geth-genesis.go
ExtraData string `json:"extraData"`
Alloc interface{} `json:"alloc"`
}
type Balance struct {
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto godoc

docker/benchmarking/benchmark/geth-genesis.go Outdated
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
var genesis Genesis
json.Unmarshal(byteValue, &genesis)
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check error

docker/benchmarking/benchmark/geth-genesis.go Outdated
return err
}

err = ioutil.WriteFile("genesis.json", file, 0644)
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

address lint issue

docker/benchmarking/benchmark/tx.go
"github.com/urfave/cli"
)

type Request struct {
Copy link
Contributor

@fedekunze fedekunze Aug 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto godoc comments in this file

fedekunze
fedekunze reviewed Sep 1, 2020
View reviewed changes
Copy link
Contributor

@fedekunze fedekunze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @araskachoi do you think we can address the final comments so that we can merge this PR?

docker/benchmarking/emint/init.sh Outdated
for i in $(seq 1 $ACCTS)
do
ethermintcli keys add $TESTKEY$i --algo "eth_secp256k1"
ethermintd add-genesis-account $(ethermintcli keys show $TESTKEY$i -a) 1000000000000000000photon,1000000000000000000stake
Copy link
Contributor

@fedekunze fedekunze Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these need to be aphoton now

@github-actions
Copy link

github-actions bot commented Nov 22, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days-before-close if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stale PRs that will be closed if no further action occurs label Nov 22, 2020
@github-actions github-actions bot closed this Nov 29, 2020
@fedekunze
Copy link
Contributor

fedekunze commented Nov 30, 2020

@araskachoi should we reopen this PR? feel free to do so and ping me once it's R4R 👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@noot noot Awaiting requested review from noot

@J-Thompson12 J-Thompson12 Awaiting requested review from J-Thompson12

1 more reviewer

@fedekunze fedekunze fedekunze left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

stale stale PRs that will be closed if no further action occurs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@araskachoi @fedekunze @J-Thompson12