Validator not starting if started together with beacon chain

[Belkaar]

🐞 Bug Report

Description

If validator and beacon chain are started together with systemd (calling prysm.sh), the validator prints this:
level=info msg="Waiting for beacon chain start log from the ETH 1.0 deposit contract" prefix=validator
and stays there.

Restarting the validator resolves the issue, but the system does not boot to a functional state.

Has this worked before in a previous version?

Don't know

🔬 Minimal Reproduction

Guess:

• Start validator while beacon is in early startup

🔥 Error

🌍 Your Environment

Operating System:

  
Debian 10
  

What version of Prysm are you running? (Which release)

  
alpha.19
  

Anything else relevant (validator index / public key)?

[j8sr]

How long did you wait after starting up both? After the beacon chain syncs to the most current node the validator will start on its own.
level=info msg="Waiting for beacon chain start log from the ETH 1.0 deposit contract" prefix=validator
is expected behaviour for the validator during beacon chain startup.

[heueristik]

I can confirm this. This seems to happen even if the beacon node was synced before.
I repeatedly got

Aug 06 01:52:44 ethnode prysm-bn[767]: time="2020-08-06 01:52:44" level=info msg="Waiting for enough suitable peers before syncing" prefix=initial-sync required=3 suitable=0
Aug 06 01:52:47 ethnode prysm-vc[768]: time="2020-08-06 01:52:47" level=info msg="Waiting for beacon node to sync to latest chain head" prefix=validator

for 5 minutes.
Stopping the validator, restarting the beacon, and starting the validator helped.

[fireagainsmile]

I built a docker image using prysm script, run a validator and a beacon chain in two containers on the same machine.

use API query beacon-chain status:
#curl localhost:3500/eth/v1alpha1/node/syncing {"syncing":false}

restarted several times, always got this

time="2020-08-06 10:52:49" level=info msg="(wallet path) /root/.eth2validators/prysm-wallet-v2" prefix=accounts-v2
time="2020-08-06 10:52:49" level=info msg="(wallet directory) /root/.eth2validators/prysm-wallet-v2" prefix=accounts-v2
time="2020-08-06 10:52:49" level=warning msg="No keys found; nothing to validate" prefix=node
time="2020-08-06 10:52:49" level=info msg="Checking DB" databasePath="/root/.eth2" prefix=node
time="2020-08-06 10:52:49" level=info msg="Starting validator node" prefix=node version="Prysm/v1.0.0-alpha.19/0d118df0343bf0e268e9fb4f2d5eb60156519c11. Built at: 2020-08-05 14:27:07+00:00"
time="2020-08-06 10:52:49" level=warning msg="You are using an insecure gRPC connection. If you are running your beacon node and validator on the same machines, you can ignore this message. If you want to know how to enable secure connections, see: https://docs.prylabs.network/docs/prysm-usage/secure-grpc" prefix=validator
time="2020-08-06 10:52:49" level=info msg="Waiting for beacon chain start log from the ETH 1.0 deposit contract" prefix=validator

[hoangong]

I have the same problem. My setup is in kubernetes and so the order of starting up is not guarantee. I had this issue times to times and had to restart the validator to fix.

[0xKiwi]

This seems like a common issue, maybe it's affected by accounts-v2 decrypting? @rauljordan

[WMEth]

I believe this is a duplicate of #6325. This problem has been around since at least June 20th so I think the cause predates accounts-v2.