[feature] Optimistic v2 submission flow #524

michaelneuder · 2023-09-08T19:36:10Z

📝 Summary

Spiritual successor to #466.

This follows up on the series: #479, #491, #494, #498, #513, #514, #518, which aim at reducing the diff and productionizing #466.

⛱ Motivation and Context

This PR implements the handleSubmitNewBlockV2 and the surrounding test infrastructure. The general flow is:

use the SubmitBlockRequestV2Optimistic SSZ type from Create ssz type: SubmitBlockRequestV2Optimistic #518 to decode the header only from the stream.
use checkSubmissionSlotDetails, checkBuilderEntry, checkSubmissionPayloadAttrs, and checkBidFloorValue to run prechecks against the submission. each of these functions was part of the refactor set of the original handleSubmitNewBlock function.
calls SaveBidAndUpdateTopBid with just the getHeaderResponse because the getPayloadResponse is not yet available.
calls optimisticV2SlowPath, which currently just does nothing. The logic from the slow path will come in a follow up PR.

In an effort to increase review-ability of this PR, I stopped here. the slow path logic is quite simple in that it just decodes the full block and simulates it before updating redis with the getPayloadResponse.

Also have nearly 100% coverage of the new lines!

📚 References

https://notes.ethereum.org/@mikeneuder/optimistic-v2

https://github.com/michaelneuder/optimistic-relay-documentation/blob/main/towards-epbs.md#optimistic-relay-v2-header-only-parsing

✅ I have run these commands

make lint
make test-race
go mod tidy
I have seen and agree to CONTRIBUTING.md

blombern · 2023-09-14T08:34:51Z

services/api/service.go

+		"value":                  bid.Value.String(),
+	})
+
+	// Verify the signature


Is there a reason why the signature verification was moved above some of the other checks? (compared to handleSubmitNewBlock v1)

honestly it was just a gut feeling that the signature check should probably come first.

good rule of thumb to do inexpensive checks first, if they don't lead to DoS issues and can quickly determine if the request is bad or not

then ramp up in resource demand to signature verification

yep agree, signature checks are expensive and should come after the cheaper checks

done in 3e5f962 !

ty

fix(redis): move WasBidSaved to after exec We have several conditions where we don't execute the pipelined commands. Only set the WasBidSaved flag after we execute the pipeline successfully. Do note, if the pipeline fails we may or may not have actually saved the bid.

Bumps [github.com/flashbots/go-utils](https://github.com/flashbots/go-utils) from 0.4.11 to 0.4.12. - [Commits](flashbots/go-utils@v0.4.11...v0.4.12) --- updated-dependencies: - dependency-name: github.com/flashbots/go-utils dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add feature flags for backwards compatibility * improve readability of flags * renaming flags * remove fork schedule checks and flag

Edited descriptions for Validators stats

* Try fix log point in builder api * Fix log reassignment in builder submission

Bumps [github.com/flashbots/go-utils](https://github.com/flashbots/go-utils) from 0.4.12 to 0.5.0. - [Commits](flashbots/go-utils@v0.4.12...v0.5.0) --- updated-dependencies: - dependency-name: github.com/flashbots/go-utils dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) from 1.12.2 to 1.13.1. - [Release notes](https://github.com/ethereum/go-ethereum/releases) - [Commits](ethereum/go-ethereum@v1.12.2...v1.13.1) --- updated-dependencies: - dependency-name: github.com/ethereum/go-ethereum dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

codecov-commenter · 2023-09-21T20:12:16Z

Codecov Report

Merging #524 (6bab6ac) into main (e0d1248) will increase coverage by 2.41%.
Report is 7 commits behind head on main.
The diff coverage is 76.13%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##             main     #524      +/-   ##
==========================================
+ Coverage   33.23%   35.64%   +2.41%     
==========================================
  Files          24       24              
  Lines        5079     5397     +318     
==========================================
+ Hits         1688     1924     +236     
- Misses       3173     3248      +75     
- Partials      218      225       +7

Flag	Coverage Δ
unittests	`35.64% <76.13%> (+2.41%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
common/test_utils.go	`26.25% <0.00%> (-11.25%)`	⬇️
common/types_spec.go	`0.00% <0.00%> (ø)`
datastore/redis.go	`59.87% <62.50%> (+0.16%)`	⬆️
services/api/service.go	`46.34% <89.81%> (+6.58%)`	⬆️

... and 2 files with indirect coverage changes

metachris · 2023-10-02T18:12:35Z

services/api/service.go

+	pipeliner             redis.Pipeliner
+}
+
+func (api *RelayAPI) optimisticV2SlowPath(r io.Reader, v2Opts v2SlowPathOpts) {


what is slowPath supposed to be doing?

metachris · 2023-10-19T15:10:54Z

A new API should be proposed and merged in the relay specs first: https://github.com/flashbots/relay-specs

Would it be good if v2 also supports non-optimistic submissions?

optimistic v2 submission flow

335bff2

blombern reviewed Sep 14, 2023

View reviewed changes

michaelneuder mentioned this pull request Sep 15, 2023

optimistic v2 slowpath #530

Open

4 tasks

michaelneuder and others added 9 commits September 21, 2023 09:24

move signature below other checks

3e5f962

Add feature flags for backwards compatibility (#521)

e8e7bc4

* Add feature flags for backwards compatibility * improve readability of flags * renaming flags * remove fork schedule checks and flag

Update website.html (#526)

2ca43aa

Edited descriptions for Validators stats

Fix nil pointer on log in builder submission (#529)

bf93e9b

* Try fix log point in builder api * Fix log reassignment in builder submission

fix lint

1318cd8

cleanup rebase

6bab6ac

metachris reviewed Oct 2, 2023

View reviewed changes

michaelneuder mentioned this pull request Oct 3, 2023

v2 builder init commit michaelneuder/builder#7

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature] Optimistic v2 submission flow #524

[feature] Optimistic v2 submission flow #524

michaelneuder commented Sep 8, 2023 •

edited

Loading

blombern Sep 14, 2023

michaelneuder Sep 15, 2023

ralexstokes Sep 15, 2023

metachris Sep 17, 2023

michaelneuder Sep 21, 2023

codecov-commenter commented Sep 21, 2023 •

edited

Loading

metachris Oct 2, 2023

metachris commented Oct 19, 2023 •

edited

Loading

[feature] Optimistic v2 submission flow #524

Are you sure you want to change the base?

[feature] Optimistic v2 submission flow #524

Conversation

michaelneuder commented Sep 8, 2023 • edited Loading

📝 Summary

⛱ Motivation and Context

📚 References

✅ I have run these commands

blombern Sep 14, 2023

Choose a reason for hiding this comment

michaelneuder Sep 15, 2023

Choose a reason for hiding this comment

ralexstokes Sep 15, 2023

Choose a reason for hiding this comment

metachris Sep 17, 2023

Choose a reason for hiding this comment

michaelneuder Sep 21, 2023

Choose a reason for hiding this comment

codecov-commenter commented Sep 21, 2023 • edited Loading

Codecov Report

metachris Oct 2, 2023

Choose a reason for hiding this comment

metachris commented Oct 19, 2023 • edited Loading

michaelneuder commented Sep 8, 2023 •

edited

Loading

codecov-commenter commented Sep 21, 2023 •

edited

Loading

metachris commented Oct 19, 2023 •

edited

Loading