Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check reorg refactor #3549

Closed
ARR552 opened this issue Apr 9, 2024 · 0 comments
Closed

Check reorg refactor #3549

ARR552 opened this issue Apr 9, 2024 · 0 comments
Assignees
@ARR552
Milestone
v0.6.6

Comments

@ARR552
Copy link
Contributor

ARR552 commented Apr 9, 2024 •
edited
Loading

Instead of checking the reorg in a different query, it should be done in the same query that reads the new data.
Old way:

  • check reorg for latest synced block
  • get new data from lastes synced block + 1

New way:

  • get new data from latest synced block

The initial block should contain the same info that the node synced previously. If not, reorg detected
@ARR552 ARR552 self-assigned this Apr 9, 2024
@ARR552 ARR552 added this to the v0.6.6 milestone Apr 9, 2024
@ARR552 ARR552 mentioned this issue Apr 10, 2024
Merged
ARR552 added a commit that referenced this issue Apr 15, 2024
@ARR552 @tclemos
Feature/#3549 reorgs improvement (#3553)
32a7750 
* New reorg function

* mocks

* linter

* Synchronizer tests

* new elderberry smc docker image

* new image

* logs

* fix json rpc

* fix

* Test sync from empty block

* Regular reorg case tested

* linter

* remove empty block + fix LatestSyncedBlockEmpty

* Improve check reorgs when no block is received during the call

* fix RPC error code for eth_estimateGas and eth_call for reverted tx and no return value; fix e2e test;

* fix test

* Extra unit test

* fix reorg until genesis

* disable parallel synchronization

---------

Co-authored-by: tclemos <thiago@polygon.technology>
@ARR552 ARR552 closed this as completed Apr 15, 2024
ARR552 added a commit that referenced this issue Apr 23, 2024
@ARR552 @tclemos
Feature/#3549 reorgs improvement (#3553)
01c456c 
* New reorg function

* mocks

* linter

* Synchronizer tests

* new elderberry smc docker image

* new image

* logs

* fix json rpc

* fix

* Test sync from empty block

* Regular reorg case tested

* linter

* remove empty block + fix LatestSyncedBlockEmpty

* Improve check reorgs when no block is received during the call

* fix RPC error code for eth_estimateGas and eth_call for reverted tx and no return value; fix e2e test;

* fix test

* Extra unit test

* fix reorg until genesis

* disable parallel synchronization

---------

Co-authored-by: tclemos <thiago@polygon.technology>
ARR552 added a commit that referenced this issue Apr 25, 2024
@ARR552 @tclemos
Cherrypick/reorgs (#3588)
c7d7834 
* change number migration

* add column checked on state.block

* if no unchecked blocks  return ErrNotFound

* migration set to checked all but the block with number below max-1000

* add column checked on state.block (#3543)

* add column checked on state.block

* if no unchecked blocks  return ErrNotFound

* migration set to checked all but the block with number below max-1000

* Feature/#3549 reorgs improvement (#3553)

* New reorg function

* mocks

* linter

* Synchronizer tests

* new elderberry smc docker image

* new image

* logs

* fix json rpc

* fix

* Test sync from empty block

* Regular reorg case tested

* linter

* remove empty block + fix LatestSyncedBlockEmpty

* Improve check reorgs when no block is received during the call

* fix RPC error code for eth_estimateGas and eth_call for reverted tx and no return value; fix e2e test;

* fix test

* Extra unit test

* fix reorg until genesis

* disable parallel synchronization

---------

Co-authored-by: tclemos <thiago@polygon.technology>

* migrations

* Fix + remove empty blocks

* unit test

* linter

* Fix + remove empty blocks (#3564)

* Fix + remove empty blocks

* unit test

* linter

* Fix/#3565 reorg (#3566)

* fix + logs

* fix loop

* Revert "fix + logs"

This reverts commit 39ced69.

* fix L1InfoRoot when an error happens during the process of the L1 information (#3576)

* fix

* Comments + mock

* avoid error from some L1providers when fromBlock is higher than toBlock

* Revert some changes

* comments

* add L2BlockModulus to L1check

* doc

* fix dbTx = nil

* fix unit tests

* config

* fix sync unit test

* linter

* fix config param typo

* synchronizer:  check l1blocks (#3546)

* wip

* run on background L1block checker

* fix lint and documentation

* fix conflict

* add unittest

* more unittest

* fix lint

* increase timeout for async unittest

* fix unittest

* rename GetResponse for GetResult and fix uniitest

* add a second gorutines for check the newest blocks

* more unittest

* add unittest and run also preCheck on launch

* by default Precheck from FINALIZED and SAFE

* fix unittest, apply PR comments

* changes suggested by ARR552 in integration method

* fix documentation

* import new network-l1-mock from PR#3553

* import new network-l1-mock from PR#3553

* import new network-l1-mock from PR#3553

* import new network-l1-mock from PR#3553

* fix unittest

* fix PR comments

* fix error

* checkReorgAndExecuteReset can't be call with lastEthBlockSynced=nil

* add parentHash to error

* fix error

* merge 3553 fix unittest

* fix unittest

* fix wrong merge

* adapt parallel reorg detection to flow

* fix unit tests

* fix log

* allow use sync parallel mode

---------

Co-authored-by: Alonso <ARR551@protonmail.com>

* linter

* comment check

---------

Co-authored-by: tclemos <thiago@polygon.technology>
Stefan-Ethernal pushed a commit to 0xPolygon/cdk-validium-node that referenced this issue Apr 25, 2024
@ARR552 @tclemos @Stefan-Ethernal
Feature/0xPolygonHermez#3549 reorgs improvement (0xPolygonHermez#3553)
67cbd46 
* New reorg function

* mocks

* linter

* Synchronizer tests

* new elderberry smc docker image

* new image

* logs

* fix json rpc

* fix

* Test sync from empty block

* Regular reorg case tested

* linter

* remove empty block + fix LatestSyncedBlockEmpty

* Improve check reorgs when no block is received during the call

* fix RPC error code for eth_estimateGas and eth_call for reverted tx and no return value; fix e2e test;

* fix test

* Extra unit test

* fix reorg until genesis

* disable parallel synchronization

---------

Co-authored-by: tclemos <thiago@polygon.technology>
@joanestebanr joanestebanr mentioned this issue Apr 26, 2024
Closed
Stefan-Ethernal pushed a commit to 0xPolygon/cdk-validium-node that referenced this issue May 21, 2024
@ARR552 @tclemos @Stefan-Ethernal
Feature/0xPolygonHermez#3549 reorgs improvement (0xPolygonHermez#3553)
d9ec8f0 
* New reorg function

* mocks

* linter

* Synchronizer tests

* new elderberry smc docker image

* new image

* logs

* fix json rpc

* fix

* Test sync from empty block

* Regular reorg case tested

* linter

* remove empty block + fix LatestSyncedBlockEmpty

* Improve check reorgs when no block is received during the call

* fix RPC error code for eth_estimateGas and eth_call for reverted tx and no return value; fix e2e test;

* fix test

* Extra unit test

* fix reorg until genesis

* disable parallel synchronization

---------

Co-authored-by: tclemos <thiago@polygon.technology>
Stefan-Ethernal added a commit to 0xPolygon/cdk-validium-node that referenced this issue May 22, 2024
@Stefan-Ethernal @agnusmor
@tclemos @ARR552 @joanestebanr @dpunish3r
Merge upstream v0.6.6 changes (#134)
404088f 
* check GER and index of synced L1InfoRoot matches with sc values (0xPolygonHermez#3551)

* apply txIndex fix to StoreTransactions; add migration to fix wrong txIndexes (0xPolygonHermez#3556)

* Feature/0xPolygonHermez#3549 reorgs improvement (0xPolygonHermez#3553)

* New reorg function

* mocks

* linter

* Synchronizer tests

* new elderberry smc docker image

* new image

* logs

* fix json rpc

* fix

* Test sync from empty block

* Regular reorg case tested

* linter

* remove empty block + fix LatestSyncedBlockEmpty

* Improve check reorgs when no block is received during the call

* fix RPC error code for eth_estimateGas and eth_call for reverted tx and no return value; fix e2e test;

* fix test

* Extra unit test

* fix reorg until genesis

* disable parallel synchronization

---------

Co-authored-by: tclemos <thiago@polygon.technology>

* Fix adding tx that matches with tx that is being processed (0xPolygonHermez#3559)

* fix adding  tx that matches (same addr and nonce) tx that is being processing

* fix generate mocks

* fix updateCurrentNonceBalance

* synchronizer:  check l1blocks (0xPolygonHermez#3546)

* wip

* run on background L1block checker

* fix lint and documentation

* fix conflict

* add unittest

* more unittest

* fix lint

* increase timeout for async unittest

* fix unittest

* rename GetResponse for GetResult and fix uniitest

* add a second gorutines for check the newest blocks

* more unittest

* add unittest and run also preCheck on launch

* by default Precheck from FINALIZED and SAFE

* fix unittest, apply PR comments

* changes suggested by ARR552 in integration method

* fix documentation

* import new network-l1-mock from PR#3553

* import new network-l1-mock from PR#3553

* import new network-l1-mock from PR#3553

* import new network-l1-mock from PR#3553

* fix unittest

* fix PR comments

* fix error

* checkReorgAndExecuteReset can't be call with lastEthBlockSynced=nil

* add parentHash to error

* fix error

* merge 3553 fix unittest

* fix unittest

* fix wrong merge

* adapt parallel reorg detection to flow

* fix unit tests

* fix log

* allow use sync parallel mode

---------

Co-authored-by: Alonso <ARR551@protonmail.com>

* Fix + remove empty blocks (0xPolygonHermez#3564)

* Fix + remove empty blocks

* unit test

* linter

* Fix/0xPolygonHermez#3565 reorg (0xPolygonHermez#3566)

* fix + logs

* fix loop

* Revert "fix + logs"

This reverts commit 39ced69.

* fix L1InfoRoot when an error happens during the process of the L1 information (0xPolygonHermez#3576)

* fix

* Comments + mock

* avoid error from some L1providers when fromBlock is higher than toBlock

* Revert some changes

* comments

* add L2BlockModulus to L1check

* doc

* fix dbTx = nil

* fix unit tests

* added logs to analyze blocking issue when storing L2 block

* add debug logs for datastreamer

* fix 0xPolygonHermez#3581 synchronizer panic synchronizing from trusted node (0xPolygonHermez#3582)

* synchronized: 0xPolygonHermez#3583  stop sync from l2 after no closed batch (0xPolygonHermez#3584)

* stop processing trusted Node after first open batch

* Update datastream lib to the latest version with additional debug info

* update dslib client interface

* Update the diff

* Fix non-e2e tests

* Update the docker image for the mock L1 network

* Update the diff

* Fix typo in the comment

* Use the Geth v1.13.11 Docker image and update the genesis spec

* Update the diff

---------

Co-authored-by: agnusmor <100322135+agnusmor@users.noreply.github.com>
Co-authored-by: Thiago Coimbra Lemos <tclemos@users.noreply.github.com>
Co-authored-by: Alonso Rodriguez <ARR552@users.noreply.github.com>
Co-authored-by: tclemos <thiago@polygon.technology>
Co-authored-by: Joan Esteban <129153821+joanestebanr@users.noreply.github.com>
Co-authored-by: Alonso <ARR551@protonmail.com>
Co-authored-by: agnusmor <agnusmor@gmail.com>
Co-authored-by: dPunisher <dpunish3r@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ARR552 ARR552

Labels
None yet
Projects
None yet
Milestone
v0.6.6
Development

No branches or pull requests
1 participant
@ARR552