lotus-miner panic when sealing

[strahe]

Checklist

• This is not a security-related bug/issue. If it is, please follow please follow the security policy.
• I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
• I am running the Latest release, the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• I did not make any code changes to lotus.

Lotus component

• lotus daemon - chain sync
• lotus fvm/fevm - Lotus FVM and FEVM interactions
• lotus miner/worker - sealing
• lotus miner - proving(WindowPoSt/WinningPoSt)
• lotus JSON-RPC API
• lotus message management (mpool)
• Other

Lotus Version

lotus-miner version 1.26.0-rc2+calibnet+git.5c0694c8d

Repro Steps

No response

Describe the Bug

When DealInfo is nil, getting EndEpoch will cause the lotus-miner to panic.

Logging Information

{"level":"debug","ts":"2024-03-11T18:50:00.616+0800","logger":"advmgr","caller":"sealer/sched_worker.go:458","msg":"assign worker sector 28500 (ready)"}
{"level":"debug","ts":"2024-03-11T18:50:00.616+0800","logger":"advmgr","caller":"sealer/sched_assigner_common.go:53","msg":"SCHED 5 queued; 4 open windows"}
{"level":"debug","ts":"2024-03-11T18:50:00.616+0800","logger":"advmgr","caller":"sealer/sched_resources.go:157","msg":"sched: not scheduling on worker 0a16f950-be6d-4ed7-bdb4-591b357ae5ce for startReady; at task limit tt=8:seal/v0/addpiece, curcount=map[]"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x23b9ced]

goroutine 1456317 [running]:
github.com/filecoin-project/lotus/storage/pipeline/piece.(*PieceDealInfo).isBuiltinMarketDeal(...)
        /root/lotus-sp/storage/pipeline/piece/piece_info.go:48
github.com/filecoin-project/lotus/storage/pipeline/piece.(*PieceDealInfo).EndEpoch(...)
        /root/lotus-sp/storage/pipeline/piece/piece_info.go:145
github.com/filecoin-project/lotus/storage/pipeline.(*SafeSectorPiece).EndEpoch(...)
        /root/lotus-sp/storage/pipeline/types.go:312
github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).handleWaitDeals(_, {{_, _}, _}, {{0x53f59d7, 0x9}, 0x6f50, 0x8, 0x65eee033, {0xc0bba77740, ...}, ...})
        /root/lotus-sp/storage/pipeline/input.go:37 +0x12d
github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).Plan.func1({{_, _}, _}, {{0x53f59d7, 0x9}, 0x6f50, 0x8, 0x65eee033, {0xc0bba77740, 0x3, ...}, ...})
        /root/lotus-sp/storage/pipeline/fsm.go:43 +0x76
reflect.Value.call({0x4db0840?, 0xc012e34f30?, 0x13?}, {0x53c3f43, 0x4}, {0xc0356daf98, 0x2, 0x2?})

[magik6k]

Did you use boost or something custom to accept / make the deal?

[strahe]

Did you use boost or something custom to accept / make the deal?

yes, boost and offline deal.

In the previous code, before calling EndEpoch, it checked whether DealInfo was nil (line37-38), but now I don’t know why it was deleted.

[magik6k]

Can you try #11708?

[strahe]

lotus-miner seems to be working fine, but I still need further testing.

[benjaminh83]

One of my calib miners, t017840 , is also actively sealing deals for FEVM testing and is running version 1.26.0-rc2+calibnet+git.4a1a7bd29+api2.3.0 seems to have similar issues. Cannot start:

Mar 12 01:18:35 ben-1 lotus-miner[2368061]: panic: runtime error: invalid memory address or nil pointer dereference
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2164e4d]
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: goroutine 1006 [running]:
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: github.com/filecoin-project/lotus/storage/pipeline/piece.(*PieceDealInfo).isBuiltinMarketDeal(...)
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/v1.26.0-rc2/storage/pipeline/piece/piece_info.go:48
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: github.com/filecoin-project/lotus/storage/pipeline/piece.(*PieceDealInfo).EndEpoch(...)
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/v1.26.0-rc2/storage/pipeline/piece/piece_info.go:145
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: github.com/filecoin-project/lotus/storage/pipeline.(*SafeSectorPiece).EndEpoch(...)
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/v1.26.0-rc2/storage/pipeline/types.go:312
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).handleWaitDeals(_, {{_, _}, _}, {{0xc014d20740, 0x9}, 0x147, 0x8, 0x65edf0ac, {0xc014d3e000, ...}, ...})
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/v1.26.0-rc2/storage/pipeline/input.go:37 +0x12d
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).Plan.func1({{_, _}, _}, {{0xc014d20740, 0x9}, 0x147, 0x8, 0x65edf0ac, {0xc014d3e000, 0x5, ...}, ...})
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/v1.26.0-rc2/storage/pipeline/fsm.go:43 +0x76
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: reflect.Value.call({0x4ed6d80?, 0xc0131b56f0?, 0x18?}, {0x54af426, 0x4}, {0xc013257f98, 0x2, 0x7d9acb?})
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /usr/local/go/src/reflect/value.go:586 +0xb0b
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: reflect.Value.Call({0x4ed6d80?, 0xc0131b56f0?, 0x17?}, {0xc013257f98?, 0x0?, 0x1?})
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /usr/local/go/src/reflect/value.go:370 +0xbc
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: github.com/filecoin-project/go-statemachine.(*StateMachine).run.func3()
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.3/machine.go:113 +0x309
Mar 12 01:18:35 ben-1 lotus-miner[2368061]: created by github.com/filecoin-project/go-statemachine.(*StateMachine).run
Mar 12 01:18:35 ben-1 lotus-miner[2368061]:         /home/benjamin/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.3/machine.go:109 +0x69a
Mar 12 01:18:35 ben-1 systemd[1]: lotus-miner-2.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Mar 12 01:18:35 ben-1 systemd[1]: lotus-miner-2.service: Failed with result 'exit-code'.

[strahe]

Looks like it's working.