[Merge] Merge transition interop with Geth

[Nashatyrev]

Description

This is How-To issue on simulating Merge transition interop with Geth EL

1. Build and start Geth EE as described here

• NOTE-1: The Initialize the genesis: command which worked for us:

./build/bin/geth --datadir "~/ethereum/taunus" init genesis.json

instead of

./build/bin/geth --catalyst --http --ws -http.api "engine" --datadir "~/ethereum/taunus" init genesis.json

• NOTE-2: besides engine API we also need eth API, so make sure the APIs list option is --http.api "engine,eth"

2. DO NOT start mining for now
3. Generate consensus genesis state:

> teku genesis mock \
  --output-file ./local-genesis.ssz \ 
  --network=minimal \
  --Xnetwork-altair-fork-epoch=0 \
  --validator-count=256

4. Run a Teku node (with overridden TTD):

> teku \
  --eth1-endpoints=http://localhost:8545 \
  --ee-fee-recipient-address=0xfe3b557e8fb62b89f4916b721be55ceb828dbd73 \
  --Xinterop-enabled=true \
  --Xinterop-number-of-validators=256 \
  --Xinterop-owned-validator-start-index=0 \
  --Xinterop-owned-validator-count=256 \
  --network=minimal \
  --Xnetwork-altair-fork-epoch=0 \ 
  --Xnetwork-merge-fork-epoch=1 \ 
  --Xnetwork-merge-total-terminal-difficulty=10 \
  --p2p-enabled=false \
  --initial-state ./local-genesis.ssz

Now Teku runs Altair fork until epoch 1, then forked to Merge and starts monitoring head PoW blocks from EL

5. Start mining in Geth console: miner.start()
   On block 5-8 the total difficulty would reach 10 and the EL should stop mining and start accepting execution payloads from CL. CL on reaching TTD would start producing execution payloads

Fixes

There were a couple of issues resolved make this working: 706ca1f

[Nashatyrev]

The issue is fixed already

[ajsutton]

I had a couple of issues following this:

1. I think the instructions for geth should point to https://notes.ethereum.org/_UH57VUPRrC-re3ubtmo2w which has terminalTotalDifficulty of 10 rather than 0 in the doc currently linked. Otherwise geth can't produce any blocks because it doesn't have the private key for the clique signer and is already at the TTD of 0 for the genesis block. However I still used the geth version from the PR linked in the current doc so it's a bit unclear...
2. The geth instructions only enable the engine JSON RPC APIs but we also need eth_getBlockByNumber so I had to start geth with:

./build/bin/geth --catalyst --http --ws --http.api "engine,eth" --datadir /tmp/geth --allow-insecure-unlock --unlock "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" --password "" --nodiscover console

But then I still fail to validate blocks because I get either:

13:23:41.014 INFO  - Validator   *** Published block              Count: 1, Slot: 44, Root: 2dcd4c..bad9
13:23:41.249 ERROR - Internal error while importing block: 2dcd4c..bad9 (44)
java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: Invalid remote response: 4: unknown header

or divide by zero errors which should be fixed by the linked commit but that doesn't appear to be in the merge-interop branch yet, only in merge/master-teku.

[ajsutton]

Cherry-picked that commit and things are running smoothly now, but Teku continues producing empty execution payloads because it hasn't been told the total terminal difficulty was 10:

13:42:27.007 INFO  - Produce pre-merge block: pow_block.total_difficulty(114) < transition_total_difficulty(115792089237316195423570985008687907853269984665640564039457584007913129638912), PoW blocks left ~57896044618658097711785492504343953926634992332820282019728792003956564819400

Looks like we need an easy way to override total terminal difficulty. Bingo - overriding TTD gets it working for me too.

So I wrote two scripts that make it easy to spin things up:

For starting geth (assumes it's running from within the root of geth's source tree, checked out with the merge PR and geth already built, with genesis.json and sk.json from https://notes.ethereum.org/_UH57VUPRrC-re3ubtmo2w in the current dir:

#!/bin/bash
set -euo pipefail

rm -rf /tmp/geth

./build/bin/geth --catalyst --datadir /tmp/geth init genesis.json
./build/bin/geth --catalyst --http --ws -http.api "engine" --datadir /tmp/geth account import sk.json
./build/bin/geth --catalyst --http --ws --http.api "engine,eth" --datadir /tmp/geth --allow-insecure-unlock --unlock "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" --password "" --nodiscover console

Just need to run miner.start() to start the miner once Teku has reached epoch 1 and is waiting for TTD to be reached.

And to start Teku, from the root of a checked out teku on merge-interop branch (with commit above cherry-picked in):

#!/bin/bash
set -euo pipefail

rm -rf /tmp/teku
rm -rf local-genesis.ssz

./build/install/teku/bin/teku genesis mock --output-file local-genesis.ssz --network minimal --Xnetwork-altair-fork-epoch 0 --validator-count 256
./build/install/teku/bin/teku \
  --log-destination CONSOLE \
  --eth1-endpoints http://127.0.0.1:8545 \
  --ee-fee-recipient-address=0xfe3b557e8fb62b89f4916b721be55ceb828dbd73 \
  --Xinterop-enabled=true \
  --Xinterop-number-of-validators=256 \
  --Xinterop-owned-validator-start-index=0 \
  --Xinterop-owned-validator-count=256 \
  --network=minimal \
  --Xnetwork-altair-fork-epoch=0 \
  --Xnetwork-merge-fork-epoch=1 \
  --Xnetwork-merge-total-terminal-difficulty=10 \
  --p2p-enabled=false \
  --rest-api-enabled \
  --initial-state ./local-genesis.ssz --data-path /tmp/teku

I'll raise a PR shortly to add support for the --Xnetwork-merge-total-terminal-difficulty=10 option in there.

[Nashatyrev]

@ajsutton thanks for corrections, I've updated the issue description. Sorry for those issues, was writing this a bit in a hurry

./build/bin/geth --catalyst --datadir /tmp/geth init genesis.json

Are you sure this worked for you with --catalyst option? Cause I think me and @tbenr had failed with it

[MariusVanDerWijden]

You don't need the --catalyst flag when initializing the genesis, you only need it later for execution

[ajsutton]

Are you sure this worked for you with --catalyst option? Cause I think me and @tbenr had failed with it
Sorry, thought I posted a reply to this. Those scripts are an exact copy of what I ran so yeah it works with --catalyst though as Marius says it's not necessary.