computing seal proof failed: coordinate(s) do not lie on the curve

[hsienfu]

Describe the problem

sealing.SectorComputeProofFailed: coordinate(s) do not lie on the curve

Hardware:

CPU: AMD Ryzen Threadripper 3970X 32-Core Processor
MEM: 256G 2666MHZ
GPU: GeForce GTX 1080 Ti
SWAP: 256G NVMe
OS: Ubuntu 18.04 server
Disk: 14TiB  logical volume

Make lotus:

env RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean all

Sectors list

The output of ./lotus-storage-miner sectors list.

0: CommitFailed sSet: NO pSet: NO tktH: 113094 seedH: 114717 deals: [0]

Sectors status

The output of ./lotus-storage-miner sectors status --log 0 for the failed sector(s).

SectorID:       0
Status: CommitFailed
CommD:          6261666b3463687a6161353766377872767975666a676135666b61653667736d6b6a32376e37343434696b3372626e7a71336477687672357075793761
CommR:          6261666b3465687a617363333234746e6b7a3535376f7262646333357a73726636367a3576676e706a6e773362666a63786e6236366369667176667771
Ticket:         a11749f0e4e3ba7ad8caf72c44a6f3094f1166db903ea0946d928eabbfec7d54
TicketH:                113094
Seed:           65fff47891e7c01673b0c5779bbcbc62218db04e4ca5a5aa4f87360847eef6d1
SeedH:          114717
Proof:
Deals:          [0]
Retries:                0
--------
Event Log:
0.      2020-06-17 03:01:05 +0800 CST:  [event;sealing.SectorStart]     {"User":{"ID":0,"SectorType":1,"Pieces":[{"Piece":{"Size":34359738368,"PieceCID":{"/":"bafk4chzaa57f7xrvyufjga5fkae6gsmkj27n7444ik3rbnzq3dwhvr5puy7a"}},"DealInfo":null}]}}
1.      2020-06-17 03:01:05 +0800 CST:  [event;sealing.SectorPacked]    {"User":{"FillerPieces":null}}
2.      2020-06-17 07:35:40 +0800 CST:  [event;sealing.SectorPreCommit1]        {"User":{"PreCommit1Out":"eyJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjEiLCJsYWJlbHMiOnsiU3RhY2tlZERyZzMyR2lCVjEiOnsibGFiZWxzIjpbeyJwYXRoIjoiL21udC9zZGIvLmxvdHVzc3RvcmFnZS9jYWNoZS9zLXQwMjQ0NzIwLTAiLCJpZCI6ImxheWVyLTEiLCJzaXplIjoxMDczNzQxODI0LCJsZXZlbHMiOjd9LHsicGF0aCI6Ii9tbnQvc2RiLy5sb3R1c3N0b3JhZ2UvY2FjaGUvcy10MDI0NDcyMC0wIiwiaWQiOiJsYXllci0yIiwic2l6ZSI6MTA3Mzc0MTgyNCwibGV2ZWxzIjo3fSx7InBhdGgiOiIvbW50L3NkYi8ubG90dXNzdG9yYWdlL2NhY2hlL3MtdDAyNDQ3MjAtMCIsImlkIjoibGF5ZXItMyIsInNpemUiOjEwNzM3NDE4MjQsImxldmVscyI6N30seyJwYXRoIjoiL21udC9zZGIvLmxvdHVzc3RvcmFnZS9jYWNoZS9zLXQwMjQ0NzIwLTAiLCJpZCI6ImxheWVyLTQiLCJzaXplIjoxMDczNzQxODI0LCJsZXZlbHMiOjd9LHsicGF0aCI6Ii9tbnQvc2RiLy5sb3R1c3N0b3JhZ2UvY2FjaGUvcy10MDI0NDcyMC0wIiwiaWQiOiJsYXllci01Iiwic2l6ZSI6MTA3Mzc0MTgyNCwibGV2ZWxzIjo3fSx7InBhdGgiOiIvbW50L3NkYi8ubG90dXNzdG9yYWdlL2NhY2hlL3MtdDAyNDQ3MjAtMCIsImlkIjoibGF5ZXItNiIsInNpemUiOjEwNzM3NDE4MjQsImxldmVscyI6N30seyJwYXRoIjoiL21udC9zZGIvLmxvdHVzc3RvcmFnZS9jYWNoZS9zLXQwMjQ0NzIwLTAiLCJpZCI6ImxheWVyLTciLCJzaXplIjoxMDczNzQxODI0LCJsZXZlbHMiOjd9LHsicGF0aCI6Ii9tbnQvc2RiLy5sb3R1c3N0b3JhZ2UvY2FjaGUvcy10MDI0NDcyMC0wIiwiaWQiOiJsYXllci04Iiwic2l6ZSI6MTA3Mzc0MTgyNCwibGV2ZWxzIjo3fSx7InBhdGgiOiIvbW50L3NkYi8ubG90dXNzdG9yYWdlL2NhY2hlL3MtdDAyNDQ3MjAtMCIsImlkIjoibGF5ZXItOSIsInNpemUiOjEwNzM3NDE4MjQsImxldmVscyI6N30seyJwYXRoIjoiL21udC9zZGIvLmxvdHVzc3RvcmFnZS9jYWNoZS9zLXQwMjQ0NzIwLTAiLCJpZCI6ImxheWVyLTEwIiwic2l6ZSI6MTA3Mzc0MTgyNCwibGV2ZWxzIjo3fSx7InBhdGgiOiIvbW50L3NkYi8ubG90dXNzdG9yYWdlL2NhY2hlL3MtdDAyNDQ3MjAtMCIsImlkIjoibGF5ZXItMTEiLCJzaXplIjoxMDczNzQxODI0LCJsZXZlbHMiOjd9XSwiX2giOm51bGx9fSwiY29uZmlnIjp7InBhdGgiOiIvbW50L3NkYi8ubG90dXNzdG9yYWdlL2NhY2hlL3MtdDAyNDQ3MjAtMCIsImlkIjoidHJlZS1kIiwic2l6ZSI6MjE0NzQ4MzY0NywibGV2ZWxzIjo3fSwiY29tbV9kIjpbNywxMjYsOTUsMjIyLDUzLDE5NywxMCwxNDcsMywxNjUsODAsOSwyMjcsNzMsMTM4LDc4LDE5MCwyMjMsMjQzLDE1Niw2NiwxODMsMTYsMTgzLDQ4LDIxNiwyMzYsMTIyLDE5OSwxNzUsMTY2LDYyXX0=","TicketValue":"oRdJ8OTjunrYyvcsRKbzCU8RZtuQPqCUbZKOq7/sfVQ=","TicketEpoch":113094}}
3.      2020-06-17 10:44:14 +0800 CST:  [event;sealing.SectorPreCommit2]        {"User":{"Sealed":{"/":"bafk4ehzasc324tnkz557orbdc35zsrf66z5vgnpjnw3bfjcxnb66cifqvfwq"},"Unsealed":{"/":"bafk4chzaa57f7xrvyufjga5fkae6gsmkj27n7444ik3rbnzq3dwhvr5puy7a"}}}
4.      2020-06-17 10:44:21 +0800 CST:  [event;sealing.SectorPreCommitted]      {"User":{"Message":{"/":"bafy2bzaceb3u4czfls5jsu3pi2rx73unqtzcimdyykoi22thuj2n2tpi7kt5s"}}}
5.      2020-06-17 10:51:28 +0800 CST:  [event;sealing.SectorSeedReady] {"User":{"SeedValue":"Zf/0eJHnwBZzsMV3m7y8YiGNsE5MpaWqT4c2CEfu9tE=","SeedEpoch":114717}}
6.      2020-06-17 11:53:48 +0800 CST:  [event;sealing.SectorComputeProofFailed]        {"User":{}}
        computing seal proof failed: coordinate(s) do not lie on the curve
7.      2020-06-17 11:54:48 +0800 CST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
8.      2020-06-17 12:57:22 +0800 CST:  [event;sealing.SectorCommitFailed]      {"User":{}}
        commit check error: invalid proof (compute error?)

Lotus storage miner logs

2020-06-17T10:44:16.063+0800    INFO    sectors storage-fsm@v0.0.0-20200522010518-83fd743db8bc/states.go:153    submitting precommit for sector: 0                                                                                                                                                                                                                          
2020-06-17T10:44:22.189+0800    INFO    sectors storage-fsm@v0.0.0-20200522010518-83fd743db8bc/states.go:164    Sector precommitted: 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
2020-06-17T10:46:36.207+0800    INFO    sectors storage-fsm@v0.0.0-20200522010518-83fd743db8bc/states.go:175    precommit message landed on chain: 0                                                                                                                                                                                                                        
2020-06-17T10:51:28.089+0800    INFO    sectors storage-fsm@v0.0.0-20200522010518-83fd743db8bc/states.go:213    scheduling seal proof computation...                                                                                                                                                                                                                        
2020-06-17T10:51:28.089+0800    INFO    sectors storage-fsm@v0.0.0-20200522010518-83fd743db8bc/states.go:215    KOMIT 0 a11749f0e4e3ba7ad8caf72c44a6f3094f1166db903ea0946d928eabbfec7d54(113094); 65fff47891e7c01673b0c5779bbcbc62218db04e4ca5a5aa4f87360847eef6d1(114717); [{34359738368 bafk4chzaa57f7xrvyufjga5fkae6gsmkj27n7444ik3rbnzq3dwhvr5puy7a}]; r:6261666b3465687
a617363333234746e6b7a3535376f7262646333357a73726636367a3576676e706a6e773362666a63786e6236366369667176667771; d:6261666b3463687a6161353766377872767975666a676135666b61653667736d6b6a32376e37343434696b3372626e7a71336477687672357075793761                                                                                                                                   
2020-06-17T10:51:28.090+0800    DEBUG   advmgr  sector-storage@v0.0.0-20200522011946-a59ca7536a95/localworker.go:66     acquired sector {244720 0} (e:6; a:0): {{0 0}  /mnt/sdb/.lotusstorage/sealed/s-t0244720-0 /mnt/sdb/.lotusstorage/cache/s-t0244720-0}                                                                                                                
2020-06-17T10:51:28.090+0800    DEBUG   advmgr  sector-storage@v0.0.0-20200522011946-a59ca7536a95/localworker.go:66     acquired sector {244720 0} (e:6; a:0): {{0 0}  /mnt/sdb/.lotusstorage/sealed/s-t0244720-0 /mnt/sdb/.lotusstorage/cache/s-t0244720-0} 

Version

The output of ./lotus --version.

lotus version 0.3.0'+git6bbea74a'

[magik6k]

This seems to be caused by compute/memory errors. Not much we can do other than auto-retry, which we now do.