Bug causing crash: Integer divide by zero

[hjawhar]

System information

Erigon version: 2.60.8 (Error is still occurring for a while now for multiple releases since 2.60.2)
I believe someone also reported the error previously: #11099
I updated to 2.60.10 and hoping that it doesn't occur again

OS & Version: Linux

Commit hash: dec9d20

Erigon Command (with flags/config):
./erigon --datadir="/home/mainnet" --chain=mainnet --port=30303 --http --http.api=eth,debug,net,trace,web3,erigon --http.addr=0.0.0.0 --http.vhosts=* --http.port=8545 --http.corsdomain=* --authrpc.addr=0.0.0.0 --authrpc.vhosts=* --authrpc.port=8551 --private.api.addr=127.0.0.1:9090 --ws --torrent.port=42069 --healthcheck --internalcl

Consensus Layer: internal

Consensus Layer Command (with flags/config): -

Chain/Network: Ethereum - Mainnet

Expected behaviour

Node was running fine for a while and suddenly crashed (?)

Actual behaviour

panic: runtime error: integer divide by zero

goroutine 16980119929 [running]:
github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange.processRewardsAndPenaltiesPostAltair({0x32e4ea0, 0xc227dd9320}, {0xc5923ce000, 0x106da7, 0x0?}, {0xc25c255450, 0x3, 0x3})
        github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange/process_rewards_and_penalties.go:46 +0x697
github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange.ProcessRewardsAndPenalties({0x32e4ea0, 0xc227dd9320}, {0xc5923ce000, 0x106da7, 0x19661b}, {0xc25c255450, 0x3, 0x3})
        github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange/process_rewards_and_penalties.go:230 +0xf9
github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange.ProcessEpoch({0x32e4ea0, 0xc227dd9320})
        github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange/process_epoch.go:47 +0x17c
github.com/ledgerwatch/erigon/cl/transition/impl/eth2.(*impl).ProcessSlots(0xc086f66808?, {0x32e4ea0, 0xc227dd9320}, 0x9f07c0)
        github.com/ledgerwatch/erigon/cl/transition/impl/eth2/operations.go:987 +0x196
github.com/ledgerwatch/erigon/cl/transition/machine.TransitionState({0x32ab4c0, 0xc3b7bb95d0}, {0x32e4ea0, 0xc227dd9320}, 0xc0b09f0af0)
        github.com/ledgerwatch/erigon/cl/transition/machine/transition.go:11 +0x4d
github.com/ledgerwatch/erigon/cl/transition.TransitionState(...)
        github.com/ledgerwatch/erigon/cl/transition/compat.go:18
github.com/ledgerwatch/erigon/cl/phase1/forkchoice/fork_graph.(*forkGraphDisk).GetState(0xc000656588, {0xd, 0x54, 0x13, 0x45, 0x7a, 0xb9, 0x53, 0xc2, 0x48, ...}, ...)
        github.com/ledgerwatch/erigon/cl/phase1/forkchoice/fork_graph/fork_graph_disk.go:356 +0x433
github.com/ledgerwatch/erigon/cl/phase1/forkchoice.(*ForkChoiceStore).getCheckpointState(0xc00020a008, {0xc225b2aae0, 0x28, 0x28})
        github.com/ledgerwatch/erigon/cl/phase1/forkchoice/utils.go:106 +0x154
github.com/ledgerwatch/erigon/cl/phase1/forkchoice.(*ForkChoiceStore).GetHead(0xc00020a008)
        github.com/ledgerwatch/erigon/cl/phase1/forkchoice/get_head.go:41 +0x199
github.com/ledgerwatch/erigon/cl/phase1/stages.ConsensusClStages.func10({0x328c988, 0xc0551be8c0}, {0x32a17a8, 0xc13bcc10c0}, 0xc076bbeee0, {0x40, 0x4f83e, 0x4f83f, 0x9f07e0, 0x9f07e0, ...})
        github.com/ledgerwatch/erigon/cl/phase1/stages/clstages.go:563 +0x65
github.com/ledgerwatch/erigon/cl/clstages.(*StageGraph[...]).StartWithStage.func1()
        github.com/ledgerwatch/erigon/cl/clstages/clstages.go:38 +0x6e
created by github.com/ledgerwatch/erigon/cl/clstages.(*StageGraph[...]).StartWithStage in goroutine 7548
        github.com/ledgerwatch/erigon/cl/clstages/clstages.go:34 +0x3a5

Steps to reproduce the behaviour

NA

Backtrace

N/A

[mdragaschnig]

Using the official docker image for 2.60.10

[11-14|14:41:09.174] Build info    git_branch=release/2.60 git_tag=v2.60.9-4-g3afee08bff git_commit=3afee08bfff043a731485e0a92dd8650823d8884

for Ethereum Mainnet with internal CL, I ran into the same error earlier today:

[11-18|00:48:14.592] head updated                             hash=0x162f06a9544d0fb03c4ca8ae2c3bc70322d28d9eb4a5a6d4da772ebfc3193380 number=21211115
[11-18|00:48:23.930] P2P                                      app=caplin peers=67
panic: runtime error: integer divide by zero

goroutine 1070442202 [running]:
github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange.processRewardsAndPenaltiesPostAltair({0x323fb80, 0xc191d6f380}, {0xc31ac5a000, 0x106da7, 0x0?}, {0xc135e8f090, 0x3, 0x3})
   github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange/process_rewards_and_penalties.go:46 +0x697
github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange.ProcessRewardsAndPenalties({0x323fb80, 0xc191d6f380}, {0xc31ac5a000, 0x106da7, 0x19661b}, {0xc135e8f090, 0x3, 0x3})
   github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange/process_rewards_and_penalties.go:230 +0xf9
github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange.ProcessEpoch({0x323fb80, 0xc191d6f380})
   github.com/ledgerwatch/erigon/cl/transition/impl/eth2/statechange/process_epoch.go:47 +0x17c
github.com/ledgerwatch/erigon/cl/transition/impl/eth2.(*impl).ProcessSlots(0xc001900008?, {0x323fb80, 0xc191d6f380}, 0x9f07c0)
   github.com/ledgerwatch/erigon/cl/transition/impl/eth2/operations.go:987 +0x196
github.com/ledgerwatch/erigon/cl/transition/machine.TransitionState({0x3205c00, 0xc1059fbd10}, {0x323fb80, 0xc191d6f380}, 0xc19564fea0)
   github.com/ledgerwatch/erigon/cl/transition/machine/transition.go:11 +0x4d
github.com/ledgerwatch/erigon/cl/transition.TransitionState(...)
   github.com/ledgerwatch/erigon/cl/transition/compat.go:18
github.com/ledgerwatch/erigon/cl/phase1/forkchoice/fork_graph.(*forkGraphDisk).GetState(0xc000aff348, {0xd, 0x54, 0x13, 0x45, 0x7a, 0xb9, 0x53, 0xc2, 0x48, ...}, ...)
   github.com/ledgerwatch/erigon/cl/phase1/forkchoice/fork_graph/fork_graph_disk.go:356 +0x433
github.com/ledgerwatch/erigon/cl/phase1/forkchoice.(*ForkChoiceStore).getCheckpointState(0xc000a8c488, {0xc0ab6d24b0, 0x28, 0x28})
   github.com/ledgerwatch/erigon/cl/phase1/forkchoice/utils.go:106 +0x154
github.com/ledgerwatch/erigon/cl/phase1/forkchoice.(*ForkChoiceStore).GetHead(0xc000a8c488)
   github.com/ledgerwatch/erigon/cl/phase1/forkchoice/get_head.go:41 +0x199
github.com/ledgerwatch/erigon/cl/phase1/stages.ConsensusClStages.func10({0x31e6368, 0xc0c4a0ce60}, {0x31fbdc8, 0xc1ea846800}, 0xc1476f0d20, {0x3f, 0x4f83e, 0x4f83f, 0x9f07e0, 0x9f07e0, ...})
   github.com/ledgerwatch/erigon/cl/phase1/stages/clstages.go:563 +0x65
github.com/ledgerwatch/erigon/cl/clstages.(*StageGraph[...]).StartWithStage.func1()
   github.com/ledgerwatch/erigon/cl/clstages/clstages.go:38 +0x6e
created by github.com/ledgerwatch/erigon/cl/clstages.(*StageGraph[...]).StartWithStage in goroutine 414
   github.com/ledgerwatch/erigon/cl/clstages/clstages.go:34 +0x3a5

Restarting the client seems to fix it; it's syncing again.