Skip to content

Commit

Permalink
State-only checkpoint state startup (#4251)
Browse files Browse the repository at this point in the history
Currently, we require genesis and a checkpoint block and state to start
from an arbitrary slot - this PR relaxes this requirement so that we can
start with a state alone.

The current trusted-node-sync algorithm works by first downloading
blocks until we find an epoch aligned non-empty slot, then downloads the
state via slot.

However, current
[proposals](ethereum/beacon-APIs#226) for
checkpointing prefer finalized state as
the main reference - this allows more simple access control and caching
on the server side - in particular, this should help checkpoint-syncing
from sources that have a fast `finalized` state download (like infura
and teku) but are slow when accessing state via slot.

Earlier versions of Nimbus will not be able to read databases created
without a checkpoint block and genesis. In most cases, backfilling makes
the database compatible except where genesis is also missing (custom
networks).

* backfill checkpoint block from libp2p instead of checkpoint source,
when doing trusted node sync
* allow starting the client without genesis / checkpoint block
* perform epoch start slot lookahead when loading tail state, so as to
deal with the case where the epoch start slot does not have a block
* replace `--blockId` with `--state-id` in TNS command line
* when replaying, also look at the parent of the last-known-block (even
if we don't have the parent block data, we can still replay from a
"parent" state) - in particular, this clears the way for implementing
state pruning
* deprecate `--finalized-checkpoint-block` option (no longer needed)
  • Loading branch information
arnetheduck authored Nov 2, 2022
1 parent aff9147 commit d839b9d
Show file tree
Hide file tree
Showing 14 changed files with 547 additions and 522 deletions.
7 changes: 6 additions & 1 deletion AllTests-mainnet.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,11 @@ OK: 2/2 Fail: 0/2 Skip: 0/2
+ roundtrip OK
```
OK: 2/2 Fail: 0/2 Skip: 0/2
## Starting states
```diff
+ Starting state without block OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1
## Sync committee pool
```diff
+ Aggregating votes OK
Expand Down Expand Up @@ -598,4 +603,4 @@ OK: 1/1 Fail: 0/1 Skip: 0/1
OK: 9/9 Fail: 0/9 Skip: 0/9

---TOTAL---
OK: 331/336 Fail: 0/336 Skip: 5/336
OK: 332/337 Fail: 0/337 Skip: 5/337
15 changes: 15 additions & 0 deletions beacon_chain/beacon_chain_db.nim
Original file line number Diff line number Diff line change
Expand Up @@ -1101,6 +1101,21 @@ proc getState*(
db.immutableValidators, db.statesNoVal[T.toFork], key.data, output,
rollback)

proc getState*(
db: BeaconChainDB, fork: BeaconStateFork, state_root: Eth2Digest,
state: var ForkedHashedBeaconState, rollback: RollbackProc): bool =
if state.kind != fork:
# Avoid temporary (!)
state = (ref ForkedHashedBeaconState)(kind: fork)[]

withState(state):
if not db.getState(state_root, forkyState.data, rollback):
return false

forkyState.root = state_root

true

proc getStateRoot(db: BeaconChainDBV0,
root: Eth2Digest,
slot: Slot): Opt[Eth2Digest] =
Expand Down
13 changes: 10 additions & 3 deletions beacon_chain/conf.nim
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,7 @@ type
name: "finalized-checkpoint-state" .}: Option[InputFile]

finalizedCheckpointBlock* {.
hidden
desc: "SSZ file specifying a recent finalized block"
name: "finalized-checkpoint-block" .}: Option[InputFile]

Expand Down Expand Up @@ -763,11 +764,17 @@ type
name: "trusted-node-url"
.}: string

blockId* {.
desc: "Block id to sync to - this can be a block root, slot number, \"finalized\" or \"head\""
defaultValue: "finalized"
stateId* {.
desc: "State id to sync to - this can be \"finalized\", a slot number or state hash or \"head\""
defaultValue: "finalized",
name: "state-id"
.}: string

blockId* {.
hidden
desc: "Block id to sync to - this can be a block root, slot number, \"finalized\" or \"head\" (deprecated)"
.}: Option[string]

backfillBlocks* {.
desc: "Backfill blocks directly from REST server instead of fetching via API"
defaultValue: true
Expand Down
Loading

0 comments on commit d839b9d

Please sign in to comment.