slashing-protection: false-positive/noisy triggering for attestation(s)

[iurii-ssv]

I've noticed ❗ could not handle message shows in logs on stage quite a bit,

there is not enough data in the log to tell for sure, but based on the "msg_round": 10, it's probably just a message that arrived late (so slashing protector detects that and rejects attestation):

{
  "level": "\u001b[35mDEBUG\u001b[0m",
  "time": "2024-10-29T08:58:32.512368Z",
  "name": "Controller.Validator",
  "msg": "❗ could not handle message",
  "pubkey": "b5ca638a9f82da289a68e416c99636986a24585063aa145929a49737d324408b08ab15af716b89b9806e3875226b1e06",
  "beacon_role": "ATTESTER",
  "msg_height": 2857481,
  "msg_round": 10,
  "consensus_msg_type": 0,
  "signers": [
    420
  ],
  "msg_type": "consensus",
  "error": "failed processing consensus message: could not process msg: invalid signed message: proposal not justified: proposal fullData invalid: slashable attestation (HighestAttestationVote), not signing",
  "errorVerbose": "slashable attestation (HighestAttestationVote), not signing\ngithub.com/ssvlabs/ssv/ekm.(*ethKeyManagerSigner).IsAttestationSlashable\n\t/go/src/github.com/ssvlabs/ssv/ekm/eth_key_manager_signer.go:242\ngithub.com/ssvlabs/ssv/ekm.(*GenesisKeyManagerAdapter).IsAttestationSlashable\n\t/go/src/github.com/ssvlabs/ssv/ekm/genesis_adapter.go:56\ngithub.com/ssvlabs/ssv/operator/validator.SetupGenesisRunners.AttesterValueCheckF.func2\n\t/go/pkg/mod/github.com/ssvlabs/ssv-spec-pre-cc@v0.0.0-20240725052506-c48532da6a63/ssv/value_check.go:77
...

two things we probably want to do about it:

• we want to have "the root cause" in the log message (expected vs actual) to be able to tell definitively what actually happened (when debugging issues like this) actually this is what HighestAttestationVote part is telling us already in the log from above
• we don't want to log false-positives as "error", so we need to add some rules that will classify messages that trigger slashing protections into two categories 1) OK to ignore 2) can't ignore, must log (perhaps we cant really tell the difference between 1 and 2 - in which case we shouldn't be reporting it as error)

^ related to this is root-validation that we currently don't perform - as mentioned in a comment here