Lodestar CL < > Teku VC incompatibility

[barnabasbusa]

Describe the bug

We are in the process to test cross beacon <> validator client compatibility, and found a bug when testing lodestar CL with teku VC.

Teku reports:

11:52:23.395 INFO  - Loaded validator key 60 of 64.
11:52:23.403 INFO  - Loaded 64 Validators: 96947de, 81b6765, a1d9840, a62c020, a0485d7, 84a687f, ad9222d, 9763dde, b27ad13, a804e4f, 8419cf0, 8dfa86c, 8723142, a03c2a8, b63f327, 84d08d5, a75ca94, 81fa222, 996323a, b5e898a, b222557, ae53027, 8fda66b, a4ee6d3, ae940a0, 86e0147, b72cb10, 996d10c, 8d46e9a, b570dde, 8aa5bbe, 8de5a62, 930743b, 958c269, 8d8985e, a759f6b, 8a8bb29, a2e2d83, 91709ee, ac69ae9, 84dc37c, 9918433, b09cb15, a8fa358, a54fe5c, 8725b32, af61f26, ab40dc1, 8d028a0, 99d83a0, b24391a, 81ea9f7, 8aec512, a35c600, 8c0d15b, afa10af, aaddb0c, ab64f90, abd1267, ab72cbc, aaf6c12, 896a51e, a1584df, af89ab0
11:52:30.013 WARN  - Validator   *** Slashing protection not loaded for validators: 8d028a0, 9763dde, b27ad13, 8d46e9a, aaddb0c, b570dde, b222557, 930743b, 8d8985e, abd1267, 8fda66b, 8aec512, 8a8bb29, 8dfa86c, b72cb10, a1584df, 8c0d15b, b5e898a, 81ea9f7, 96947de… (64 total)
11:53:23.799 ERROR - Validator   *** Error while connecting to beacon node event stream com.launchdarkly.eventsource.StreamIOException: java.net.SocketTimeoutException: timeout (See log file for full stack trace)

Lodestar reports:

nothing worth mentioning

Snooper between cl <> vc reports:

Apr  5 12:06:14.938 2024 RESPONSE (status 200 OK)
{
  "data": [],
  "execution_optimistic": false
}

Apr  5 12:06:19.940 2024 REQUEST /eth/v1/beacon/states/head/validators?id=0x96947de9e6068c22a7716656a2755a9551b0b66c2d1a741bf84a088fe1e84...
null

Apr  5 12:06:19.942 2024 RESPONSE (status 200 OK)
{
  "data": [],
  "execution_optimistic": false
}

Expected behavior

I would expect all client combinations would work.

Steps to reproduce

config.yaml:

participants:
  - cl_type: lodestar
    cl_image: chainsafe/lodestar:next
    vc_type: teku
additional_services:
  - dora
snooper_enabled: true
global_log_level: debug

kurtosis run github.com/kurtosis-tech/ethereum-package --args-file config.yaml

Additional context

Current BN <> VC Compatibility list tracker

Operating system

Linux

Lodestar version or commit hash

Version: v1.17.0/898cd90

cc: @pk910

[nflaig]

Related to #5693 (comment), we need to include finalized field in all required responses as noted in #5693.

Edit: There are further issues that need to be fixed, see Consensys/teku#8118 (comment)

[nflaig]

11:53:23.799 ERROR - Validator *** Error while connecting to beacon node event stream com.launchdarkly.eventsource.StreamIOException: java.net.SocketTimeoutException: timeout (See log file for full stack trace)

This issue is due to snooper, disabling it fixes all remaining issues with Teku VC, see Consensys/teku#8118 (comment)