Store or cache payloads after build process finishes

[Rjected]

Describe the feature

Some of the hive withdrawals/Withdrawals Fork on tests fail with:

>> (42643f5d) {"jsonrpc":"2.0","id":4,"method":"engine_getPayloadV1","params":["0x584f0665c885d563"]}
<< (42643f5d) {"jsonrpc":"2.0","result":{"parentHash":"0x8efd0af809ef64f159eed0717051a46a6a019783ba42657c16529ef18385dabb","feeRecipient":"0x0000000000000000000000000000000000000000","stateRoot":"0xa9ba00ae004a6c4faf51fcf60c3a2a14f6015191fdaefbad2706cf287205fd14","receiptsRoot":"0xc3b02cdaaa0473586550f8cde520fdf7c1d0c1892a624119e5865ce6ecb40bd6","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prevRandao":"0x78541859ee1b2249c2261355da80ac42391a0269a8d43a5ddf2c180965f441d4","blockNumber":"0x1","gasLimit":"0x2fefd8","gasUsed":"0xc9d2c","timestamp":"0x1235","extraData":"0x99726574682f76302e312e302d616c7068612e382f6c696e7578","baseFeePerGas":"0x342770c0","blockHash":"0x9584c5effe8a36e418b3521279cacd53a349751323e65fa0714eb49f616f052d","transactions":["0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018032a041897d60b337f18e991c979482c0a22983d2c72de6f688310e099e0f25829d24a0286670065266e8cca5639f75ed79bfbe0bb8f28ca37987c237eabf92e2159e5c","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018031a04870e0c9b5f8f1f632ab2ba5fe9b4cde4dc25ea164315bdffee1c9bb31e78540a06a9d0fbe853eac64993c820128f92c4b60731ff933c780516f9515e1d76ad29a","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018031a026445e971c7cbefec1a56502a9d0719547c06771cb62f495014e4186be8572d4a00e7fa5cc6d55255181a8be4186563114af8dc3f5e471698a7f21f121411d6249","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018032a0c59254c9f3cac87f26030f92502d5685a1a8a4b22a5df21d0372126b9e4bdf5ca06bffba6fc267d59b1ab6b32c20827cf582396e8d7ce4790dba07dea3278d0ef1","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018031a0a6fa74e8dad217249cfdab81295526753e79e585bf4614d7255a216c6c9416aca05c51ce5ebcc0414d8337d206d5005059691bb9ecfd216e98df0c63576f9ec25b","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018031a030ced8b064a2998cdcdef2beb057bd017c29c150d993d9f29b90672a345f7f88a064a31a7bc2a902729c92b34187ed22ed1dd55cad36c65486c465f8450c70096a","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018031a0e0758bc8fe46bab31a2a711f6e3b835d5d1f7b252ba78f8f8300db70105ddb50a0697b378657650a19a87a27ddea348c4a3651fc0cf68a43f1234b25824b2f8619","0xf865808506fc23ac00830124f8940101010101010101010101010101010101010101018031a07835eeb8a56fac15c2e159b3a7cace369da0837029d056112927749f37f26a2da0398fcf71446386ffab839861af87c04cb230fc7e901e7f4d52a0d3263bf0ec11","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c001a0d755b21af03a9d72e9786fc74f042ff1ffc5357a4a906701fc66a4031e319ca4a003a0d98542d4af1d7e4f51b38973d350de4aa313995aaf2b1d146fb4d7f42ce3","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c080a04f5677e4b1321282ba3da1a4c042f0cbd3fcc68b467ab4398e16c8d73852298fa064483beb6931ae103fcc6c6fced46814cc3bb09ca58bbc571c27cc93bcc6f93c","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c001a0fd0aa463f1f8930178681e1082f42cb58788c634dde2dd5d0014a2bf5b472d63a0032fa62adf0619f0c6cd39065b764bc58db1012c7674a2df95209ca27807857a","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c001a06e480f2974840e05c678624dbdb66a3eb082ad9350b535d935d090ccb5055989a034252393cb18c7d0c47236773a333c32e87b3b72f1f28e1c65fbfc05407a3200","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c080a08dcabc99b6dadf05d825ee8604f7de0143e76865fd560167f0d5697ab13cbbf8a06778ceae13f7e3e60cde2c2389f872c77716b6be5bf20342b4e1661df2a12887","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c080a004a21ac939174c2a197eab44f9108bc9410c8c5048b2c98713a611a6cd6ff11fa035c1489b4c94ac8ecadb810500c9ef2bed2cb9de18319fea1631f7dbd74053d3","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c080a0a009595e0217ee478eb17e7f7096cffedfaf104b39dfc19776c576784d2bbcbca068a633764da59fba93915c6afde86af4e4e59f8e1a1c624317f6e50e5c5545dc","0x02f86c0780843b9aca008506fc23ac00830124f89402020202020202020202020202020202020202020180c001a074641385d7dc50f6f992feb925e91dbc51da1460460eedc0349c853b71d5b161a00ac1686384c81a1842cf46390bdd262af964acde364c6a4986252c359c908deb"]},"id":4}
>> (42643f5d) {"jsonrpc":"2.0","id":5,"method":"engine_getPayloadV2","params":["0x584f0665c885d563"]}
<< (42643f5d) {"jsonrpc":"2.0","error":{"code":-38001,"message":"Unknown payload"},"id":5}
DEBUG (Withdrawals Fork on Block 2): Failed `Expect*` routine called from: file=/source/suites/withdrawals/tests.go, line=1208
FAIL (Withdrawals Fork on Block 2): Expected no error on EngineGetPayloadV2: error=Unknown payload

This is because we return and remove the payload when it resolves. The engine_getPayload spec says this:

Specification

1. Given the payloadId client software MUST return the most recent version of the payload that is available in the corresponding build process at the time of receiving the call.

2. The call MUST return -38001: Unknown payload error if the build process identified by the payloadId does not exist.

3. Client software MAY stop the corresponding build process after serving this call.

We do stop the corresponding build process, but we may need to store / cache payloads even after the build process is done.

Additional context

No response

[mattsse]

hmm, if I read this correctly then

MUST return the most recent version of the payload that is available in the corresponding build process

Client software MAY stop the corresponding build process after serving this call.

then returning

The call MUST return -38001: Unknown payload error if the build process identified by the payloadId does not exist.

would be allowed?

emphasis on build process

[Rjected]

emphasis on build process

ah, that's right, it seems like the tests violate the spec then?

[mattsse]

definitely needs clarification

[mattsse]

closing as outdated