Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ed25519 async batch verification for JoinSplit signatures #1952

Merged
merged 9 commits into from
Mar 30, 2021
Merged

Conversation

dconnolly
Copy link
Contributor

@dconnolly dconnolly commented Mar 26, 2021

Motivation

We've been verifying JoinSplitSigs one-by-one pre-ZIP-215. Now as we're post-ZIP-215,
we can take advantage of the batch math to validate this signatures.

Solution

This adds another tower::Service that allows async batch validation of Ed25519 signatures. This is very similar boilerplate to redjubjub::Verifier and similar tests.
I removed the single check and replaced its invocation with a call to this communal ed25519::Verifier instance.

I would have pumped all the joinsplits in our MAINNET_BLOCKS test vectors but these
signatures are over the sighash, which needs the NU code to compute, and once we're
doing all that set up, we're basically doing transaction validation, so.

The code in this pull request has:

  • Documentation Comments
  • Unit Tests and Property Tests

Review

Not urgent, a quick chunk of work we've recently been unblocked on doing.

Related Issues

Resolves #1944

Follow Up Work

This is the third async batch verifier tower::Service, and won't be the last (Orchard will bring halo2::Verifier and redpallas::Verifier eventually). There is a lot of boilerplate being copy-pasted around, which may be ok for easy deletion, but when it's duplicated amongst ~5 variants, it's a signal to revisit where we can DRY this up or make a more generic async batch Verifier, which can then be instantiated with different Items/ItemWrappers and some call method/function. Tracked in #1951.

These batch verifiers are returning their own errors from their underlying batchers, we should figure out the best way to turn those into TransactionErrors, either in zebra-consensus::transaction or in the async service layer.

@dconnolly dconnolly added A-consensus Area: Consensus rule updates NU Sprout Network Upgrade: Sprout specific tasks (before Overwinter) A-rust Area: Updates to Rust code NU-4 Canopy Network Upgrade: Canopy specific tasks C-enhancement Category: This is an improvement labels Mar 26, 2021
@dconnolly dconnolly added this to the 2021 Sprint 6 milestone Mar 26, 2021
@dconnolly dconnolly self-assigned this Mar 26, 2021
@mpguerra mpguerra removed this from the 2021 Sprint 6 milestone Mar 26, 2021
@dconnolly dconnolly force-pushed the ed25519-batch branch 2 times, most recently from cf193a5 to 626462c Compare March 26, 2021 19:19
We've been verifying JoinSplitSigs one-by-one pre-ZIP-215. Now as we're post-ZIP-215,
we can take advantage of the batch math to validate this signatures.

I would have pumped all the joinsplits in our MAINNET_BLOCKS test vectors but these
signatures are over the sighash, which needs the NU code to compute, and once we're
doing all that set up, we're basically doing transaction validation, so.

Resolves #1944
Copy link
Contributor

@oxarbitrage oxarbitrage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not an expert on async code but i think this looks good.

I found a few minors that we can fix before merging:

  • remove some not needed code and commented code.
  • add "ed25519" string to some tracing messages that are too generic and can be confused with other similar ones.

zebra-consensus/src/primitives/ed25519.rs Show resolved Hide resolved
zebra-consensus/src/transaction.rs Outdated Show resolved Hide resolved
zebra-chain/Cargo.toml Outdated Show resolved Hide resolved
zebra-consensus/src/primitives/ed25519.rs Outdated Show resolved Hide resolved
zebra-consensus/src/primitives/ed25519.rs Outdated Show resolved Hide resolved
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Copy link
Contributor

@teor2345 teor2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

This PR could merge as-is, but let's do some tweaks and open some follow-ups first.

@teor2345
Copy link
Contributor

I'm testing some full mainnet and testnet syncs locally with this branch and #1950.

I don't want to block this PR on those tests, it's not a high risk PR.

@teor2345
Copy link
Contributor

I'm testing some full mainnet and testnet syncs locally with this branch and #1950.

Seems to work fine after 2 full mainnet syncs and 2 full testnet syncs, and after syncing to tip.

@teor2345
Copy link
Contributor

Seeing 2 mainnet instances at around 180% CPU, spread across all cores.

(The CPU might even be a bit elevated due to all the "Invalid binding signature" logs.)

dconnolly and others added 5 commits March 30, 2021 11:25
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
@dconnolly dconnolly requested a review from teor2345 March 30, 2021 15:57
@dconnolly dconnolly added this to the 2021 Sprint 6 milestone Mar 30, 2021
Copy link
Contributor

@teor2345 teor2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, let's do more batching!

@teor2345
Copy link
Contributor

I'm going to admin-merge because @oxarbitrage's changes were addressed.

@teor2345 teor2345 merged commit 0ffab6d into main Mar 30, 2021
@teor2345 teor2345 deleted the ed25519-batch branch March 30, 2021 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-consensus Area: Consensus rule updates A-rust Area: Updates to Rust code C-enhancement Category: This is an improvement NU Sprout Network Upgrade: Sprout specific tasks (before Overwinter) NU-4 Canopy Network Upgrade: Canopy specific tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Async batch Ed25519 signature verification
4 participants