Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve block generation performance #1251

Merged
merged 5 commits into from
Aug 29, 2018
Merged

Improve block generation performance #1251

merged 5 commits into from
Aug 29, 2018

Conversation

abitmore
Copy link
Member

@abitmore abitmore commented Aug 13, 2018 •
edited
Loading

Improved block generating performance by caching public keys extracted from transaction signatures which reduces number of ECDSA verification calls.

Side effect: due to additional data copy, replay is slightly slower, profiling data says the difference is around 4%.Update: latest code avoided the performance penalty.

Perhaps todo:

  • extract signing keys for popped transactions in the first place. Update: it's done implicitly.

@pmconrad
Copy link
Contributor

IMO at this time 4% replay time hurts more than block generation performance.

@abitmore
Copy link
Member Author

abitmore commented Aug 19, 2018 •
edited
Loading

To improve replay performance, we need to change transactions field in block struct from vector<processed_transaction> to vector<processed_transaction_with_signees>, but this will change serialization, so will break both on-disk block database storage and p2p. A potential solution is to remove the added signees field from FC_REFLECT. Thoughts?

@abitmore abitmore changed the title Improve block generating performance Improve block generation performance Aug 19, 2018
@abitmore
Copy link
Member Author

abitmore commented Aug 19, 2018 •
edited
Loading

Update: it's even slower to replay after changed transactions in signed_block struct to processed_transaction_with_signees type. :-/ Code is here: abitmore@5b45f82

By the way, I tried to extract trx signees before pushing a block with this commit: abitmore@7309e85, don't know if it's worthwhile. With the patch, I assume that the performance will be better when there is frequent chain reorganization, but perhaps will have reduced performance under normal conditions due to additional data copy, although both affect validation nodes only (nodes that configured with a witness_id or started with --force-validate).

Thoughts?

@abitmore
Copy link
Member Author

IMO at this time 4% replay time hurts more than block generation performance.

Yes, at this time the chain is not under any meaningful pressure.

In the long run, faster block generation means higher throughput, aka higher TPS. This PR is the first step towards multi-threaded signature verification. If we're going to do another stress testing, I think we need something like this.

@abitmore
Copy link
Member Author

abitmore commented Aug 21, 2018 •
edited
Loading

I guess the performance loss is due to additional object construction when applying transactions. I guess performance can be restored by moving the signees field into signed_transaction struct directly. Need to revert quite some code, so I'll rebase.

@abitmore
Copy link
Member Author

Rebased. The code is much cleaner now, and replay performance is fine. @pmconrad please review.

pmconrad
pmconrad reviewed Aug 23, 2018
View reviewed changes
Copy link
Contributor

@pmconrad pmconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good first steps.

libraries/app/api.cpp Outdated Show resolved Hide resolved
libraries/app/api.cpp Outdated Show resolved Hide resolved
libraries/chain/include/graphene/chain/protocol/transaction.hpp Outdated Show resolved Hide resolved
libraries/chain/protocol/transaction.cpp Outdated Show resolved Hide resolved
libraries/chain/protocol/transaction.cpp Outdated Show resolved Hide resolved
@abitmore
Dereference chain_database shared_ptr
8ae885f
@abitmore
Updated transaction::signees to mutable
1124800 
and
* updated get_signature_keys() to return a const reference,
* get_signature_keys() will update signees on first call,
* modified test cases and wallet.cpp accordingly,
* no longer construct a new signed_transaction object before pushing
@abitmore
No longer extract public keys before pushing a trx
de65fc4 
and removed unused new added constructor and _get_signature_keys() function from signed_transaction struct
@abitmore
Copy link
Member Author

Updated signees to mutable, in the meanwhile reverted a lot of code (again).

@abitmore abitmore merged commit 0dbf181 into develop Aug 29, 2018
@abitmore abitmore deleted the trx-signees branch August 29, 2018 00:23
@TheTaconator TheTaconator mentioned this pull request Oct 15, 2018
Closed
@pmconrad pmconrad mentioned this pull request Feb 1, 2019
Open
24 tasks
@gladcow gladcow mentioned this pull request Sep 20, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pmconrad pmconrad pmconrad approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
201810 - Feature Release
Development

Successfully merging this pull request may close these issues.
2 participants
@abitmore @pmconrad