Persist light client updates #5545

eserilev · 2024-04-09T21:01:26Z

Issue Addressed

Partially #3651

Proposed Changes

This PR aims to persist light client updates. Once persisted, this will allow lighthouse nodes to serve them over RPC.

beacon_node/beacon_chain/src/light_client_server_cache.rs

…rsist-light-client-updates

dapplion

Overall looks good! Two points:

We should run the specs tests for the is_better_update function, see here https://github.com/ethereum/consensus-specs/blob/dev/tests/formats/light_client/update_ranking.md

The function under which updates are computed only gets triggered for recent blocks. See

lighthouse/beacon_node/beacon_chain/src/beacon_chain.rs

Lines 3944 to 3948 in ffe29c0

    
           // Do not trigger light_client server update producer for old blocks, to extra work 
        
           // during sync. 
        
           if self.config.enable_light_client_server 
        
               && block_delay_total < self.slot_clock.slot_duration() * 32 
        
           {

Now that the LC server computes recent updates AND full updates, the latter should include historical updates. Therefore we should be producing updates during forward range sync. Also for backfill sync, but that's a feature for another PR :)

beacon_node/beacon_chain/src/light_client_server_cache.rs

dapplion · 2024-07-28T20:05:08Z

Same CI failures

FAIL [   4.186s] beacon_chain::beacon_chain_tests store_tests::light_client_updates_test

FAIL [   1.187s] ef_tests::tests light_client_update

And the basic simulator

…rilev/lighthouse into persist-light-client-updates

eserilev · 2024-08-01T01:02:10Z

Since the fork epoch boundaries in basic-sim aren't multiples of 256, the light client periods dont align with the fork boundaries. We end up getting some BeaconStateError::IncorrectStateVariant when constructing light client updates

I've gotten the basic-sim tests to pass by moving the light client update logic to the bottom of recompute_and_cache_updates. This prevents the light client update stuff from affecting optimistic and finality update

I've also completely removed any checks related to light client updates in basic-sim. I've kept the test vectors related to optimistic and finality updates. I think we have enough light client update test coverage in other places to live with this for now...

dapplion

Pushed a commit with:

Clean up two info statements likely there from debugging
Remove pub where possible
A few cosmetic nits

PR is good to go and passing tests :)

jimmygchen

LGTM too!

jimmygchen · 2024-08-09T07:34:00Z

@mergify queue

mergify · 2024-08-09T07:34:05Z

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 3913ea4

* persist light client updates * update beacon chain to serve light client updates * resolve todos * cache best update * extend cache parts * is better light client update * resolve merge conflict * initial api changes * add lc update db column * fmt * added tests * add sim * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * fix some weird issues with the simulator * tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * test changes * merge conflict * testing * started work on ef tests and some code clean up * update tests * linting * noop pre altair, were still failing on electra though * allow for zeroed light client header * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * merge unstable * remove unwraps * remove unwraps * Update light_client_update.rs * merge unstable * move functionality to helper methods * refactor is best update fn * refactor is best update fn * improve organization of light client server cache logic * fork diget calc, and only spawn as many blcoks as we need for the lc update test * fetch lc update from the cache if it exists * fmt * Fix beacon_chain tests * Add debug code to update ranking_order ef test * Fix compare code * merge conflicts * fix test * Merge branch 'persist-light-client-updates' of https://github.com/eserilev/lighthouse into persist-light-client-updates * Use blinded blocks for light client proofs * fix ef test * merge conflicts * fix lc update check * Lint * resolve merge conflict * Merge branch 'persist-light-client-updates' of https://github.com/eserilev/lighthouse into persist-light-client-updates * revert basic sim * small fix * revert sim * Review PR * resolve merge conflicts * Merge branch 'unstable' into persist-light-client-updates

eserilev added 4 commits April 2, 2024 22:45

persist light client updates

914bb1e

update beacon chain to serve light client updates

c308768

resolve todos

e332b00

cache best update

3a93933

eserilev added light-client work-in-progress PR is a work-in-progress labels Apr 9, 2024

dapplion reviewed Apr 10, 2024

View reviewed changes

beacon_node/beacon_chain/src/light_client_server_cache.rs Outdated Show resolved Hide resolved

eserilev added 12 commits April 10, 2024 14:06

extend cache parts

b9bdd88

is better light client update

9671423

resolve merge conflict

5861417

initial api changes

6e0fa26

add lc update db column

c8af8fb

fmt

45a5d8d

added tests

2545655

add sim

3029f2c

Merge branch 'unstable' of https://github.com/sigp/lighthouse into pe…

fad083a

…rsist-light-client-updates

fix some weird issues with the simulator

49a7e61

tests

00d4fd0

Merge branch 'unstable' of https://github.com/sigp/lighthouse into pe…

4b8683c

…rsist-light-client-updates

eserilev added the ready-for-review The code is ready for review label May 27, 2024

eserilev marked this pull request as ready for review May 27, 2024 18:03

jimmygchen removed the work-in-progress PR is a work-in-progress label May 28, 2024

eserilev added 2 commits May 29, 2024 14:31

test changes

6f12133

merge conflict

9464518

dapplion reviewed May 30, 2024

View reviewed changes

beacon_node/beacon_chain/src/light_client_server_cache.rs Outdated Show resolved Hide resolved

beacon_node/beacon_chain/src/light_client_server_cache.rs Outdated Show resolved Hide resolved

beacon_node/beacon_chain/src/light_client_server_cache.rs Outdated Show resolved Hide resolved

eserilev added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels May 30, 2024

eserilev added 3 commits May 31, 2024 13:45

testing

22e9201

started work on ef tests and some code clean up

1946100

update tests

2b1e47d

dapplion and others added 4 commits July 5, 2024 10:24

Fix beacon_chain tests

eb19b0d

Add debug code to update ranking_order ef test

bcda61b

Fix compare code

e8691db

merge conflicts

50a29c5

eserilev and others added 12 commits July 29, 2024 15:58

fix test

b8741af

Merge branch 'persist-light-client-updates' of https://github.com/ese…

8b332e0

…rilev/lighthouse into persist-light-client-updates

Use blinded blocks for light client proofs

a1f2408

fix ef test

42e5f23

merge conflicts

dc3a909

fix lc update check

25985d9

Lint

d7d097c

resolve merge conflict

81394c4

Merge branch 'persist-light-client-updates' of https://github.com/ese…

56afbdd

…rilev/lighthouse into persist-light-client-updates

revert basic sim

42dec62

small fix

2f0837a

revert sim

e6555f3

Review PR

1835b29

dapplion approved these changes Aug 1, 2024

View reviewed changes

eserilev and others added 2 commits August 2, 2024 09:25

resolve merge conflicts

13e3c42

Merge branch 'unstable' into persist-light-client-updates

cd32a28

jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Aug 9, 2024

jimmygchen approved these changes Aug 9, 2024

View reviewed changes

mergify bot merged commit 3913ea4 into sigp:unstable Aug 9, 2024
28 checks passed

eserilev mentioned this pull request Sep 3, 2024

Add beacon API support for light client updates by range #4895

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persist light client updates #5545

Persist light client updates #5545

eserilev commented Apr 9, 2024

dapplion left a comment

dapplion commented Jul 28, 2024

eserilev commented Aug 1, 2024 •

edited

Loading

dapplion left a comment

jimmygchen left a comment

jimmygchen commented Aug 9, 2024

mergify bot commented Aug 9, 2024 •

edited

Loading

	// Do not trigger light_client server update producer for old blocks, to extra work
	// during sync.
	if self.config.enable_light_client_server
	&& block_delay_total < self.slot_clock.slot_duration() * 32
	{

Persist light client updates #5545

Persist light client updates #5545

Conversation

eserilev commented Apr 9, 2024

Issue Addressed

Proposed Changes

dapplion left a comment

Choose a reason for hiding this comment

dapplion commented Jul 28, 2024

eserilev commented Aug 1, 2024 • edited Loading

dapplion left a comment

Choose a reason for hiding this comment

jimmygchen left a comment

Choose a reason for hiding this comment

jimmygchen commented Aug 9, 2024

mergify bot commented Aug 9, 2024 • edited Loading

✅ The pull request has been merged automatically

eserilev commented Aug 1, 2024 •

edited

Loading

mergify bot commented Aug 9, 2024 •

edited

Loading