Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redundant eth1 (Eth1Provider) logs vs execution logging when (dis)connected #6128

Open
philknows opened this issue Nov 25, 2023 · 4 comments
Labels
good first issue Issues that are suitable for first-time contributors. help wanted The author indicates that additional help is wanted. meta-bug Issues that identify a bug and require a fix. prio-low This is nice to have. scope-logging Issue about logs: hygiene, format issues, improvements.

Comments

@philknows
Copy link
Member

Describe the bug

When a connected execution client goes offline, Lodestar detects the disconnection, but prints out the two following logs:

Nov-25 21:04:33.334[execution]       error: Execution client went offline oldState=SYNCED, newState=OFFLINE
Nov-25 21:04:33.558[network]         error: Error receiving block slot=420322, peer=16Uiu2HAmC7eA5UzuYxRrUwshzEZaTkh2ByPuES7fpcBAhFVV3AuN, code=BLOCK_ERROR_EXECUTION_ERROR, execStatus=UNAVAILABLE, errorMessage=Request to http://localhost:8551 failed, reason: connect ECONNREFUSED 127.0.0.1:8551
Error: BLOCK_ERROR_EXECUTION_ERROR
    at verifyBlockExecutionPayload (file:///usr/app/packages/beacon-node/src/chain/blocks/verifyBlocksExecutionPayloads.ts:366:25)
    at verifyBlocksExecutionPayload (file:///usr/app/packages/beacon-node/src/chain/blocks/verifyBlocksExecutionPayloads.ts:157:28)
    at async Promise.all (index 0)
    at BeaconChain.verifyBlocksInEpoch (file:///usr/app/packages/beacon-node/src/chain/blocks/verifyBlock.ts:91:70)
    at BeaconChain.processBlocks (file:///usr/app/packages/beacon-node/src/chain/blocks/index.ts:75:68)
    at JobItemQueue.runJob (file:///usr/app/packages/beacon-node/src/util/queue/itemQueue.ts:101:22)
Nov-25 21:05:03.219[eth1]            error: Eth1Provider faced error state=OFFLINE, lastErrorAt=12:12:06 AM - Request to http://localhost:8551 failed, reason: connect ECONNREFUSED 127.0.0.1:8551
FetchError: Request to http://localhost:8551 failed, reason: connect ECONNREFUSED 127.0.0.1:8551
    at wrappedFetch (file:///usr/app/packages/api/src/utils/client/fetch.ts:10:11)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at JsonRpcHttpClient.fetchJsonOneUrl (file:///usr/app/packages/beacon-node/src/eth1/provider/jsonRpcHttpClient.ts:276:19)
    at JsonRpcHttpClient.fetchJson (file:///usr/app/packages/beacon-node/src/eth1/provider/jsonRpcHttpClient.ts:229:16)
    at wrapWithEvents.payload.payload (file:///usr/app/packages/beacon-node/src/eth1/provider/jsonRpcHttpClient.ts:147:37)
    at JsonRpcHttpClient.wrapWithEvents (file:///usr/app/packages/beacon-node/src/eth1/provider/jsonRpcHttpClient.ts:208:24)
    at Eth1Provider.getBlockNumber (file:///usr/app/packages/beacon-node/src/eth1/provider/eth1Provider.ts:186:28)
    at Eth1DepositDataTracker.update (file:///usr/app/packages/beacon-node/src/eth1/eth1DepositDataTracker.ts:197:32)
    at Eth1DepositDataTracker.runAutoUpdate (file:///usr/app/packages/beacon-node/src/eth1/eth1DepositDataTracker.ts:169:29)

We get errors from eth1, execution, and network relating to the same issue. When execution is online again, we receive the following logs:

Nov-25 21:05:25.577[eth1]             info: Eth1Provider is back online oldState=OFFLINE, newState=ONLINE
Nov-25 21:05:29.186[execution]        info: Execution client became online oldState=OFFLINE, newState=ONLINE

Expected behavior

There shouldn't be an overwhelming amount, nor redundant logging for users when an execution client goes offline. We should aim to just keep the execution logging and disregard the Eth1Provider logs unless it is necessary for something else.

Steps to reproduce

  1. Run Lodestar and an execution client locally (mine was run with Nethermind)
  2. Shut off execution client
  3. Observe logs
  4. Start execution client
  5. Observe logs

Additional context

No response

Operating system

Linux

Lodestar version or commit hash

unstable

@philknows philknows added scope-ux Issues for CLI UX or general consumer UX. meta-bug Issues that identify a bug and require a fix. labels Nov 25, 2023
@nflaig
Copy link
Member

nflaig commented Dec 1, 2023

Error logs for eth1 provider where just added in v1.12.0

Not sure if those were necessary, maybe @nazarhussain can clarify. If we expect those to always be set to same execution client (which is pretty much the case) I guess it is quite redundant.

In that case, could also consider removing eth1 provider urls log

this.logger?.info("Eth1 provider", {urls: providerUrls.toString()});

@nazarhussain
Copy link
Contributor

Not sure if those were necessary, maybe @nazarhussain can clarify. If we expect those to always be set to same execution client (which is pretty much the case) I guess it is quite redundant.

It's not necessary if eth1 and execution are pointing to same provider url. If not then each provider can have different status.

@nflaig
Copy link
Member

nflaig commented Dec 5, 2023

It's not necessary if eth1 and execution are pointing to same provider url. If not then each provider can have different status.

Does this mean we close this issue? I think it has a good point as in practice those point to the same EL which makes the logs redundant.

@philknows
Copy link
Member Author

Ah interesting. I guess if people wanted to have multiple ELs connected to a beacon node, this would make sense to track all of the provider statuses. If they're the same (which they generally are), can we suppress? It's super minor, but I like cleaner logs for a cleaner experience.

@philknows philknows added good first issue Issues that are suitable for first-time contributors. help wanted The author indicates that additional help is wanted. labels Dec 22, 2023
@philknows philknows added prio-low This is nice to have. scope-logging Issue about logs: hygiene, format issues, improvements. and removed scope-ux Issues for CLI UX or general consumer UX. labels Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Issues that are suitable for first-time contributors. help wanted The author indicates that additional help is wanted. meta-bug Issues that identify a bug and require a fix. prio-low This is nice to have. scope-logging Issue about logs: hygiene, format issues, improvements.
Projects
None yet
Development

No branches or pull requests

3 participants