error: panic: invalid height: 2; expected: 3

[jcstein]

I thought last night that this error was specific to lazy aggregation, but it is happening with lazy aggregation off too. it produces 2 blocks, then errors: panic: invalid height: 2; expected: 3

repro details: #802 (comment)

https://app.warp.dev/block/UfQ2rKEZRsgOd4XAwNisbQ full logs

[jcstein]

Weird, not happening now. Will reopen if it happens again

[jcstein]

Still occuring, this time at block 5: https://app.warp.dev/block/lxqGnI1puzDtEnEiPUpEIX

[jcstein]

Run 1

Replicated with v0.8.0-rc2 of celestia-node in https://github.com/jcstein/local-celestia-devnet

Rollup logs: https://app.warp.dev/block/fbwEXCK1f4eswx3ZbLkEkD
Video of crash: https://www.loom.com/share/4ac5744feb1f437397c8ed9eff69f14c

Run 2

Crash at block 4: https://app.warp.dev/block/jpXpuEy2GNLOkvP0Q81Xd7

[jcstein]

Run on Blockspace Race using a light node

i’m not sure what’s up with this. it doesn’t work on upgraded container for local-celestia-devnet, but works on the BSR network (logs). i’m positive i’ve cleared chain history and re-initing the chain when on local network

[jcstein]

Odd, still occuring on v0.8.0 of node on local-celestia-devnet: https://app.warp.dev/block/NaTXL9AoKvKeDolNfPCkZI
Logs of local devnet: https://app.warp.dev/block/LlA6ILiSv41zgbqauezZ7D

[jcstein]

However, it works when posting to Blockspace Race testnet on v0.8.0
Rollup logs: https://app.warp.dev/block/wnwC6o71JbNWqKFbW9iMdK

Balance change on DA node:

Before:

curl -X GET http://localhost:26659/balance
{"denom":"utia","amount":"4434000"}

After:

curl -X GET http://localhost:26659/balance
{"denom":"utia","amount":"4416000"}

[jcstein]

This is not fixed as there is no version with the fix. cc @nashqueue