Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FullSync Future should stop when total terminal difficulty is reached #3423

Merged
merged 9 commits into from
Mar 2, 2022

Conversation

gezero
Copy link
Contributor

@gezero gezero commented Feb 14, 2022

Signed-off-by: Jiri Peinlich jiri.peinlich@gmail.com

The Full sync future should finish executing once the terminal total difficulty is reached. Note that we will not stop syncing exactly at the TTD but we will finish syncing the current batch of blocks.

@gezero gezero force-pushed the stop-fullsync-when-ttd-reached branch from 6a96b21 to 9eb1598 Compare February 16, 2022 14:43
Copy link
Contributor

@garyschulte garyschulte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Feel free to disregard opinionated suggestion to use streaming Optional.

@garyschulte
Copy link
Contributor

One other comment - we need to follow this up with a post-ttd sync method pretty soon, otherwise we are going to be unable to sync kintsugi. Backward sync alone won't work for networks that have a long post-merge history

@gezero gezero force-pushed the stop-fullsync-when-ttd-reached branch 7 times, most recently from 6d0d7bc to 622e7f7 Compare February 22, 2022 13:30
@gezero gezero changed the base branch from merge to main February 22, 2022 13:30
@gezero
Copy link
Contributor Author

gezero commented Feb 22, 2022

I am only going to merge the changes here, when I will have the follow-up sync ready.

@gezero gezero force-pushed the stop-fullsync-when-ttd-reached branch 6 times, most recently from 61706df to bb87943 Compare February 23, 2022 15:42
Copy link
Contributor

@garyschulte garyschulte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with this PR on devnet4 with good, but mixed results regarding the initial full-sync.

@gezero gezero force-pushed the stop-fullsync-when-ttd-reached branch 2 times, most recently from 300361c to 274293b Compare March 1, 2022 13:57
@garyschulte garyschulte enabled auto-merge (squash) March 1, 2022 21:04
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
in the future we want to be able to specify a specific block hash as a termination condition. The FullSyncTerminationCondition interface should allow it

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
@gezero gezero force-pushed the stop-fullsync-when-ttd-reached branch from ae9a715 to aec582b Compare March 2, 2022 13:24
@sonarcloud
Copy link

sonarcloud bot commented Mar 2, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

81.2% 81.2% Coverage
0.0% 0.0% Duplication

Copy link
Contributor

@garyschulte garyschulte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes #3268 . Remaining sync issues have been split into separate issues

@garyschulte garyschulte merged commit 605ff69 into hyperledger:main Mar 2, 2022
frankisawesome pushed a commit to frankisawesome/besu that referenced this pull request Mar 16, 2022
…hyperledger#3423)

* FullSync Future should stop when total terminal difficulty is reached

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
frankisawesome added a commit that referenced this pull request Mar 16, 2022
* Prepare fo release 22.1.2

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* remove orion refs (#3494)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Handle TTD and mining pre and post merge  (#3486)

* handle TTD and mining pre and post merge without creating a regression in behavior for clique networks

Signed-off-by: garyschulte <garyschulte@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Reject locally-sourced transactions below the minimum gas price when not mining (#3397)

* Refactor TransactionPool to accept MiningParameters
* Check for zero GasPrice Frontier Transactions
* if you are not mining your node could fill up with pending transactions.
* make low-or-no-gas transactions viable for local transactions

Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: garyschulte <garyschulte@gmail.com>

* 3378 ws engine (#3493)

* implements execution engine APIs over websockets

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* QBFT Migration: allow active mining coordinator to be shutdown (#3438)

Call the shutdown task async to allow the BftProcessor CountDownLatch to complete.
Start the next coordinator only once the previous shutdown is complete.

Fixes #3003

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* to remove deprecated proto dependency, needed to add jaeger-proto dependency (#3485)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Rename field random to prevRandao (#3495)

* rename field random to prevRandao

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* Improve contract validation errors (#3499)

For each contract validation error return a relevant error instead of
the incorrect "INSUFFICIENT_GAS" error.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* Update ef bootnodes (#3502)

* update ef bootnodes

Signed-off-by: Frank Li <b439988l@gmail.com>

* fix typo in bootnode

Signed-off-by: Frank Li <b439988l@gmail.com>

* update mainnet config

Signed-off-by: Frank Li <b439988l@gmail.com>

* Add GitHub action to verify gradle-wrapper.jar (#3503)

Checks that the gradle wrapper jar is from an official release.
See https://github.com/gradle/wrapper-validation-action for more details.

Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>

* Bugfix/terminal block check (#3501)

* speculative use of backward sync in forkchoiceupdated
* reorg newPayload tests to not return INVALID_TERMINAL_BLOCK for as-yet-unknown parent hashes

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Refactor trace api methods (#3510)

* abstract trace classes

Signed-off-by: Frank Li <b439988l@gmail.com>

* add spdx headers

Signed-off-by: Frank Li <b439988l@gmail.com>

* FullSync Future should stop when total terminal difficulty is reached (#3423)

* FullSync Future should stop when total terminal difficulty is reached

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>

* [MINOR] typo in resource class  (#3516)

* medium

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* typo

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* debug_accountAt - fixed issue with contract code (#3518)

* fixed bug with contract code > 32 bytes

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* move Vijay to emeritus (#3514)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* trace_get  (#3477)

* Initial commit for trace_get implementation. No tests.
* test files for trace_get

Signed-off-by: Mark Terry <mark.terry@consensys.net>
Signed-off-by: Frank Li <b439988l@gmail.com>
Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Co-authored-by: Frank Li <b439988l@gmail.com>
Signed-off-by: Frank Li <b439988l@gmail.com>

* [MINOR] addressed some sonar code smells (#3520)

* addressed some sonar code smells

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* updated several dependencies (#3523)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Log initial sync stats every 10 secs at max to avoid spamming logs (#3525)

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Updated dependencies including log4j (#3529)

* updated some more dependencies

* changed errorprone back

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* upgrade vertx (#3530)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* trace_rawTransaction (#3459)

* Json RPC endpoint and test files for trace_rawTransaction
* cache the block number to make sure we are using the same block

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Co-authored-by: Frank Li <b439988l@gmail.com>
Co-authored-by: MadelineMurray <43356962+MadelineMurray@users.noreply.github.com>

* Feature/trace callmany (#3468)

* unit test + working code + test spec files

Signed-off-by: Frank Li <b439988l@gmail.com>

Signed-off-by: Stefan Pingel <stefan.pingel@consensys.net>

Co-authored-by: Frank Li <b439988l@gmail.com>
Signed-off-by: Frank Li <b439988l@gmail.com>

* add trace_callMany to CHANGELOG (#3531)

Signed-off-by: Stefan Pingel <stefan.pingel@consensys.net>
Signed-off-by: Frank Li <b439988l@gmail.com>

* 3465 jwt auth (#3508)

* refactored auth to allow variant JWT rules to be adopted by different stacks
* configures JWT secret from a file, or creates a temporary one
* jwt auth is optional for engine apis
* test coverage of new cli options

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* [Issue 3115] Support mining beneficiary transitions (#3505)

Signed-off-by: Meredith Baxter <meredith.baxter@palm.io>
Signed-off-by: Frank Li <b439988l@gmail.com>

* MERGE: engine_exchangeTransitionConfigurationV1 API (#3473)

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: garyschulte <garyschulte@gmail.com>

* docker executor (#3535)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

Co-authored-by: Frank Li <b439988l@gmail.com>

* Trace APIs: add revert reason (#3539)

* added revertReason

* added test file for trace_transaction with revertReason

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Updated netty (#3538)

* updated netty

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* check types before providing a PowMiningCoordinator (#3548)

Signed-off-by: Justin Florentine <justin+github@florentine.us>

* add kiln network, add network option for fast-sync capable (#3547)

* add kiln network, add network option for fast-sync capable

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Richer return information for Precompiled Contracts (#3546)

Allow precompiled contracts to return richer information instead of
results/fail. System/precompile contracts can now revert, fail, refund
gas, etc. instead of just succeed or fail.

Signed-off-by: Danno Ferrin <danno.ferrin@gmail.com>

* Refactor graphqlcontext since getContext() is deprecated (#3536)

* refactor graphqlcontext since getContext() is deprecated

Signed-off-by: Sandra Wang <yx97.wang@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Always show privacyGroupId on PrivateTransactionReceipt (#3544)

- This is to provide a clearer way of knowing which
privacyGroupId a private transaction ended up in.

Signed-off-by: Antony Denyer <git@antonydenyer.co.uk>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Acceptance test for execution engine apis (#3533)

* added acceptance test for execution engine apis

* removed check of payload id length

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* fix: AuthProvider deprecated, refactor TomlAuth  with AuthenticationProvider (#3550)

Signed-off-by: Sandra Wang <yx97.wang@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* modified batch request test to avoid async complete issue (#3556)

Signed-off-by: garyschulte <garyschulte@gmail.com>
Signed-off-by: Frank Li <b439988l@gmail.com>

* Stratum mining errata (#3554)

* couple bits of additional plumbing to ensure transitionCoordinator correctly routes stratum ethash observers

Signed-off-by: garyschulte <garyschulte@gmail.com>

* ignore flaky test (#3566)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* remove requirement for CL to have zero blocknumber for TransitionConfiguration (#3565)

Signed-off-by: garyschulte <garyschulte@gmail.com>

* added trace logging of parameter RPC calls (#3553)

* added trace logging of parameter RPC calls

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Revert "upgrade vertx (#3530)" (#3567)

This reverts commit 9d2d87e.

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* added extra info to issue template (#3551)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Added debug logging to header download step. (#3568)

Signed-off-by: Mark Terry <mark.terry@consensys.net>
Signed-off-by: Frank Li <b439988l@gmail.com>

* [#2766]fix: clean up deprecation warnings (#3511)

* clean up deprecation warnings
*Quantity Interface getValue()
* LegacyPrivateStateStorage
* GraphQLDataFetcherContext

Signed-off-by: Sandra Wang <yx97.wang@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* updated TTD for Kiln (#3573)

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* add execution engine endpoints to docker ports (#3575)

* add execution engine endpoints to docker ports

Signed-off-by: garyschulte <garyschulte@gmail.com>

* Warn if consensus client doesn't call transition configuration endpoint within 120 seconds (#3569)

* add qos timer to engine_exchangeTransitionConfiguration

Signed-off-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Justin Florentine <justin+github@florentine.us>

* updated tests to ensure hex values are returned (#3564)

Signed-off-by: Justin Florentine <justin+github@florentine.us>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* [MINOR] corrections to changelog and typos (#3580)

* typos
* 3379 was the pr to merge branch
* added link for 3202

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* [MINOR] xl executor for quorum ATs (nightly) (#3571)

* xl executor

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* WebSocket batch test - remove Ignore (#3579)

* remove ignore of batch websocket test

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* Added --version flag to subcommands [#2762] (#3577)

* added --version flag to subcommands (#2762)

Signed-off-by: Davide Savazzi <davide@davidesavazzi.net>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Signed-off-by: Frank Li <b439988l@gmail.com>

* Add headings to Besu help (#3559)

* Headings are bold and cyan
* grouping graphQL related options
* Json RPC Http Option Group
Json RPC Websocket Option Group

Signed-off-by: Sandra Wang <yx97.wang@gmail.com>

Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* update picocli (#3552)

Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

* change gradle version

Signed-off-by: Frank Li <b439988l@gmail.com>

Co-authored-by: Danno Ferrin <danno.ferrin@gmail.com>
Co-authored-by: Sally MacFarlane <sally.macfarlane@consensys.net>
Co-authored-by: garyschulte <garyschulte@gmail.com>
Co-authored-by: Antony Denyer <git@antonydenyer.co.uk>
Co-authored-by: Justin Florentine <justin+github@florentine.us>
Co-authored-by: Simon Dudley <simon.dudley@consensys.net>
Co-authored-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Co-authored-by: Adrian Sutton <adrian.sutton@consensys.net>
Co-authored-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Co-authored-by: mark-terry <36909937+mark-terry@users.noreply.github.com>
Co-authored-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Co-authored-by: MadelineMurray <43356962+MadelineMurray@users.noreply.github.com>
Co-authored-by: Stefan Pingel <16143240+pinges@users.noreply.github.com>
Co-authored-by: mbaxter <meredith.baxter@palm.io>
Co-authored-by: Sandra Wang <yx97.wang@gmail.com>
Co-authored-by: Davide Savazzi <davide.savazzi@gmail.com>
garyschulte pushed a commit to garyschulte/besu that referenced this pull request May 2, 2022
…hyperledger#3423)

* FullSync Future should stop when total terminal difficulty is reached

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
eum602 pushed a commit to lacchain/besu that referenced this pull request Nov 3, 2023
…hyperledger#3423)

* FullSync Future should stop when total terminal difficulty is reached

Signed-off-by: Jiri Peinlich <jiri.peinlich@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants