Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EOF+shandong #5

Draft
wants to merge 71 commits into
base: eof
Choose a base branch
from
Draft

EOF+shandong #5

wants to merge 71 commits into from

Conversation

meowsbits
Copy link

@meowsbits meowsbits commented Dec 1, 2022


This PR makes the needed changes in order we can sync the Shandong testnet using the ConsenSys Teku Ethereum 2.0 Beacon Chain.

We were able to sync up to the latest block which at the time of editing is 211430.

Setup

Running an Erigon instance on a DO VPS with systemd config:

root@ubuntu-ny1-erigon:/etc/systemd/system# cat /etc/systemd/system/erigon_shandong.service
[Unit]
Description=Erigon Shandong

[Service]
Type=simple
User=root
Restart=always
RestartSec=3
TimeoutStopSec=10min
ExecStart=/bin/sh -c '/root/etccooperative-erigon+shandong/build/bin/erigon \
        --chain shandong \
        --datadir /mnt/volume_nyc1_02/erigon/shandong \
        --log.console.verbosity 5 \
        --private.api.addr 127.0.0.1:9998 \
        --port 30344 \
        --authrpc.port 8558 \
        --http.port 8544 \
        --http.api=engine,eth,erigon,web3,net \
        --externalcl \
        --authrpc.port 8558 \
        --authrpc.jwtsecret=/mnt/volume_nyc1_02/erigon/shandong/jwtsecret \
        --torrent.port 42086 \
        --lightclient.discovery.port 4004 \
        --sentinel.port 7778 \
        '

[Install]
WantedBy=default.target

Also running Teku instance with systemd config:

[Unit]
Description=Teku Shandong

[Service]
Type=simple
User=root
Restart=always
RestartSec=3
TimeoutStopSec=10min
ExecStart=/bin/sh -c '/root/teku/teku-22.11.0/bin/teku \
        --initial-state=https://raw.githubusercontent.com/ethereumjs/consensus-deployment-ansible/master/shandong-testnet/custom_config_data/genesis.ssz \
        --network=/root/teku/config.yaml \
        --data-base-path=/mnt/volume_nyc1_02/teku \
        --data-storage-mode=archive \
        --ee-endpoint=http://localhost:8558 \
        --ee-jwt-secret-file "/mnt/volume_nyc1_02/erigon/shandong/jwtsecret" \
        --p2p-discovery-bootnodes=enr:-LK4QHkdCND7lcPwqP0oP8EvjtyEIEwlufo4Q2WLU7lfnE7wXaiPFYqrxG2ve0yjwobsv-JivPPnPgM5FXF9_AUe2JIGh2F0dG5ldHOIAAAAAAAAAACEZXRoMpA6j89cITN5Av__________gmlkgnY0gmlwhC5lfi2Jc2VjcDI1NmsxoQO2iyKHl53XEZpkmqwzrNde8tJtHBG1juKX6GQ8maqYAIN0Y3CCIyiDdWRwgiMo,enr:-LK4QFUme0A5wcehaAVkgo3wILst__VwT-CS90IAHRf81EEDewxXYOY3tGH0kYg8jm3dRap-ebt9W2YpYxK4RhICoc4Gh2F0dG5ldHOIAAAAAAAAAACEZXRoMpA6j89cITN5Av__________gmlkgnY0gmlwhLKAy_OJc2VjcDI1NmsxoQIioMWqai_HMbtalAFqTa97lLgjfA_D9NBt9BenWmKjDIN0Y3CCIyiDdWRwgiMo,enr:-LK4QClQvVrrQ9Jm0mOUX8I9vu-anp-dgD9FSiW8Ep0uR6pEZh4t8iMljhXnE2q1UjL2rHAJeIxlrdbwcn1wjeLaamwGh2F0dG5ldHOIAAAAAAAAAACEZXRoMpA6j89cITN5Av__________gmlkgnY0gmlwhI5draqJc2VjcDI1NmsxoQKZ1U-C4IWnkiu6EvbIls9iRazxW5RZej-htHgwNf3Ef4N0Y3CCIyiDdWRwgiMo,enr:-LK4QGMlUKIzZVYqB2uIsizLIaKrPlHrGyZFCg5ond0soaGGOdsV9oR_50PAnOTE_6GZN6p_uqqkvGtnXPyhKEiizbYGh2F0dG5ldHOIAAAAAAAAAACEZXRoMpA6j89cITN5Av__________gmlkgnY0gmlwhKRcrjiJc2VjcDI1NmsxoQIO0t2j7TMxczat4kjQJaFikgg3mNCMQmgUX99zotTV5YN0Y3CCIyiDdWRwgiMo \
        '

[Install]
WantedBy=default.target

Code diff at erigon for syncing

For syncing we had to temporary disable the CheckpointSyncEndpoint logic

git diff
diff --git a/eth/backend.go b/eth/backend.go
index 1151a7514..5383c9191 100644
--- a/eth/backend.go
+++ b/eth/backend.go
@@ -59,7 +59,7 @@ import (
        "google.golang.org/protobuf/types/known/emptypb"

        "github.com/ledgerwatch/erigon/cl/clparams"
-       clcore "github.com/ledgerwatch/erigon/cmd/erigon-cl/core"
+       //clcore "github.com/ledgerwatch/erigon/cmd/erigon-cl/core"
        "github.com/ledgerwatch/erigon/cmd/lightclient/lightclient"
        "github.com/ledgerwatch/erigon/cmd/rpcdaemon/cli"
        "github.com/ledgerwatch/erigon/cmd/rpcdaemon/commands"
@@ -519,16 +519,16 @@ func New(stack *node.Node, config *ethconfig.Config, logger log.Logger) (*Ethere
                if err != nil {
                        return nil, err
                }
-               bs, err := clcore.RetrieveBeaconState(ctx,
-                       clparams.GetCheckpointSyncEndpoint(clparams.NetworkType(config.NetworkID)))
+               //bs, err := clcore.RetrieveBeaconState(ctx,
+               //      clparams.GetCheckpointSyncEndpoint(clparams.NetworkType(config.NetworkID)))

-               if err != nil {
-                       return nil, err
-               }
+               //if err != nil {
+               //      return nil, err
+               //}

-               if err := lc.BootstrapCheckpoint(ctx, bs.FinalizedCheckpoint.Root); err != nil {
-                       return nil, err
-               }
+               //if err := lc.BootstrapCheckpoint(ctx, bs.FinalizedCheckpoint.Root); err != nil {
+               //      return nil, err
+               //}

                go lc.Start()
        }

meowsbits and others added 20 commits December 1, 2022 11:23
…figuration parameters

Date: 2022-12-01 11:23:57-08:00
Signed-off-by: meows <b5c6@protonmail.com>
…etwork to configuration switches

Use: --chain=shandong

Date: 2022-12-01 11:50:44-08:00
Signed-off-by: meows <b5c6@protonmail.com>
Date: 2022-12-01 12:12:35-08:00
Signed-off-by: meows <b5c6@protonmail.com>
…nfigs for CL for Shandong

Date: 2022-12-01 13:01:30-08:00
Signed-off-by: meows <b5c6@protonmail.com>
Date: 2022-12-01 13:29:51-08:00
Signed-off-by: meows <b5c6@protonmail.com>
Date: 2022-12-01 13:35:08-08:00
Signed-off-by: meows <b5c6@protonmail.com>
Date: 2022-12-01 13:39:54-08:00
Signed-off-by: meows <b5c6@protonmail.com>
… oracle

Date: 2022-12-01 13:48:49-08:00
Signed-off-by: meows <b5c6@protonmail.com>
The rand.Int call below for the lookup panics
because len(checkpoints) is 0 and it uses an unchecked
len(checkpoints) -1. Not allowed.

Date: 2022-12-01 13:53:38-08:00
Signed-off-by: meows <b5c6@protonmail.com>
The testnet Shandong does not have beacon state
infrastructure, and the code as-is does not handle
this omission well.

So this patch modifies the logic to not fail
hard if the beacon state parameters are not
configured (or fail for some other reason).

Date: 2022-12-03 09:04:43-08:00
Signed-off-by: meows <b5c6@protonmail.com>
…estnet

Configuration types declare this feature as a
Shanghai fork feature, but Shandong testnet uses
a Shanghai configuration that does not have Withdrawals
configured.
So we need to hack it out.

Date: 2022-12-05 13:16:40-08:00
Signed-off-by: meows <b5c6@protonmail.com>
revitteth and others added 28 commits December 9, 2022 18:04
Co-authored-by: nanevardanyan <nanevardanyants@gmail.com>
…nSync (erigontech#6222)

Change from: 
```
begin(TxNoSync)
exec new block, index nee data
do limited pruning 
commit()
send notifications about new data arrival to other apps, they may start reading new data at this time (by new read transactions) 
```
Change to: 
```
begin(TxNoSync)
exec new block, index nee data
commit() // no fsync here
send notifications about new data arrival to other apps, they may start reading new data at this time (by new read transactions) 
begin()
do pruning 
commit() // fsync here
```

it allows notify earlier. Fsync (of all changes) on modern drives is
fast, but on cloud-drives it’s about 1sec in worst cases.
This ensures that the filter bad transactions can use the same context
each time, so increases an account nonce in the first batch of
transactions is committed to the simulation just in case the same
account appears in the next batch of transactions.

Also makes use of the new txpool candidate functionality to ensure we're
getting fresh transactions each time.
…#6266)

Fixes erigontech#4935

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
erigontech#6273)

… formatting when error

Fixes erigontech#5365

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
)

Works around a flaw in the upgrade logic of the system contracts. Since
they are updated directly, without first being self-destructed and then
re-created, the usual incarnation logic does not get activated, and all
historical records of the code of these contracts are retrieved as the
most recent version. This problem will not exist in erigon3, but until
then, a workaround will be used to access code of such contracts through
a special structure, `SystemContractCodeLookup`

Fixes erigontech#5865

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Fixes erigontech#4778

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
…uests (erigontech#6281)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
# Conflicts:
#	cmd/hack/hack.go
#	consensus/serenity/serenity.go
#	core/genesis.go
#	eth/backend.go
#	go.mod
#	params/bootnodes.go
#	params/config.go
#	params/networkname/network_name.go
#	turbo/node/node.go
@github-actions
Copy link

This PR is stale because it has been open for 40 days with no activity.

@github-actions github-actions bot added the Stale label Jan 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.