Cannot produce blocks after genesis in private net

[mrtestyboy781]

🐞 Bug Report

Description

Cannot create blocks in beacon-chain after genesis activation in private testnet setup.

Has this worked before in a previous version?

Never attempted before.

🔬 Minimal Reproduction

ETH1 side

1. Launch ETH1 Auro chain node (PoA)
2. Deploy ETH 2.0 deposit contract to ETH1 chain
3. Generated 16 validator keys with deposit-cli

deposit --num_validators 16 --mnemonic_language english --chain medalla

4. Generate 16 validator deposits of 32 ETH to contract

ETH2 side

1. Build prysm with bazel build --define ssz=minimal //beacon-chain:beacon-chain
2. Modify minimal chain config to have 16 validators, and update genesis_fork, deposit contract, etc. See modified chain config
3. Launch 2 beacon-chain nodes

• bazel run --define=ssz=minimal //beacon-chain:beacon-chain -- --chain-id 8995 --contract-deployment-block 3 --deposit-contract 0xab98823DD9f56dFB9f1459072631bDB1FF2Eb0Ea --network-id 8995 --datadir /Users/anon/Library/Eth2-PoA --http-web3provider http://127.0.0.1:8545 --no-discovery --chain-config-file /Users/anon/eth2-integration/config/beacon-chain-testnet.yaml --enable-debug-rpc-endpoints --force-clear-db --min-sync-peers 0 --p2p-local-ip=127.0.0.1 --verbosity=trace
• bazel-3.5.0 run --define=ssz=minimal //beacon-chain:beacon-chain -- --chain-id 8995 --contract-deployment-block 3 --deposit-contract 0xab98823DD9f56dFB9f1459072631bDB1FF2Eb0Ea --network-id 8995 --datadir /Users/anon/Library/Eth2-PoA-2 --http-web3provider http://127.0.0.1:8545 --no-discovery --chain-config-file /Users/anon/eth2-integration/config/beacon-chain-testnet.yaml --enable-debug-rpc-endpoints --force-clear-db --min-sync-peers 0 --p2p-tcp-port 13001 --p2p-udp-port 12001 --grpc-gateway-port 3501 --rpc-port 4001 --peer /ip4/127.0.0.1/tcp/13000/p2p/16Uiu2HAmKa8hQh7yABY8VrGoKpNdb69NEQkrRaRvyH8qibJh4hm1 --verbosity=trace --p2p-local-ip=127.0.0.1

🔥 Error

The beacon-chain will detect enough deposits to activate the beacon-chain, but the beacon-chain nodes are then unable to produce any blocks. It appears they are stuck trying to sync chain-tip from other peers, but neither has the first block.

See logs from the two nodes:
log1.txt
log2.txt

I am also curious if

🌍 Your Environment

Operating System:

Mac v10.15.4

What version of Prysm are you running? (Which release)

master @ a74cf5d

Anything else relevant (validator index / public key)?

I am curious if the beacon-chain can be configured to run standalone without a peer after genesis activation.

Also, is there a public example of someone else doing this? Being able to quickly test an entire validator deposit -> activation cycle will be very important

[mrtestyboy781]

Resolved by #7258