forked from ethereum/go-ethereum
-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimized validate_blob_transaction_wrapper()
function
#4
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Draft
asn-d6
force-pushed
the
lincomb-verify-blobs
branch
from
March 1, 2022 22:43
82cfcba
to
6e2f2d8
Compare
asn-d6
changed the title
Optimized
Optimized Mar 1, 2022
verify_blobs()
functionvalidate_blob_transaction_wrapper()
function
asn-d6
force-pushed
the
lincomb-verify-blobs
branch
from
March 2, 2022 16:15
6e2f2d8
to
b78632f
Compare
Awesome, Merged into #1 (github doesn't seem to detect the rebase/merge though):
|
asn-d6
added a commit
to asn-d6/EIPs
that referenced
this pull request
May 11, 2022
To validate a 4844 transaction in the mempool, the verifier checks that each provided KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment Before this patch, to do this validation, we reconstructed the commitment from the blob data (d_i above), and checked it against the provided commitment. This was expensive because computing a commitment from blob data (even using Lagrange basis) involves N scalar multiplications, where N is the number of field elements per blob. Initial benchmarking showed that this was about 40ms for N=4096 which was deemed too expensive. For more details see: https://hackmd.io/@protolambda/eip-4844-implementer-notes#Optimizations protolambda/go-ethereum#4 In this patch, we speed this up by providing a KZG proof for each commitment. The verifier can check that proof to ensure that the KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment, proof To do so, we evaluate the blob data polynomial at a random point `x` to get a value `y`. We then use the KZG proof to ensure that the commited polynomial (i.e. the commitment) also evaluates to `y` at `x`. If the check passes, it means that the KZG commitment matches the polynomial represented by the blob data. This is significantly faster since evaluating the blob data polynomial at a random point using the Barycentric formula can be done efficiently with only field operations (see https://hackmd.io/@vbuterin/barycentric_evaluation). Then, verifying a KZG proof takes two pairing operations (which take about 0.6ms each). This brings the total verification cost to about 2 ms per blob. With some additional optimizations (using linear combination tricks as the ones linked above) we can batch all the blobs together into a single efficient verification, and hence verify the entire transaction in 2.5 ms. The same techniques can be used to efficiently verify blocks on the consensus side.
asn-d6
added a commit
to asn-d6/EIPs
that referenced
this pull request
May 11, 2022
To validate a 4844 transaction in the mempool, the verifier checks that each provided KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment Before this patch, to do this validation, we reconstructed the commitment from the blob data (d_i above), and checked it against the provided commitment. This was expensive because computing a commitment from blob data (even using Lagrange basis) involves N scalar multiplications, where N is the number of field elements per blob. Initial benchmarking showed that this was about 40ms for N=4096 which was deemed too expensive. For more details see: https://hackmd.io/@protolambda/eip-4844-implementer-notes#Optimizations protolambda/go-ethereum#4 In this patch, we speed this up by providing a KZG proof for each commitment. The verifier can check that proof to ensure that the KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment, proof To do so, we evaluate the blob data polynomial at a random point `x` to get a value `y`. We then use the KZG proof to ensure that the commited polynomial (i.e. the commitment) also evaluates to `y` at `x`. If the check passes, it means that the KZG commitment matches the polynomial represented by the blob data. This is significantly faster since evaluating the blob data polynomial at a random point using the Barycentric formula can be done efficiently with only field operations (see https://hackmd.io/@vbuterin/barycentric_evaluation). Then, verifying a KZG proof takes two pairing operations (which take about 0.6ms each). This brings the total verification cost to about 2 ms per blob. With some additional optimizations (using linear combination tricks as the ones linked above) we can batch all the blobs together into a single efficient verification, and hence verify the entire transaction in 2.5 ms. The same techniques can be used to efficiently verify blocks on the consensus side.
asn-d6
added a commit
to asn-d6/EIPs
that referenced
this pull request
May 23, 2022
To validate a 4844 transaction in the mempool, the verifier checks that each provided KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment Before this patch, to do this validation, we reconstructed the commitment from the blob data (d_i above), and checked it against the provided commitment. This was expensive because computing a commitment from blob data (even using Lagrange basis) involves N scalar multiplications, where N is the number of field elements per blob. Initial benchmarking showed that this was about 40ms for N=4096 which was deemed too expensive. For more details see: https://hackmd.io/@protolambda/eip-4844-implementer-notes#Optimizations protolambda/go-ethereum#4 In this patch, we speed this up by providing a KZG proof for each commitment. The verifier can check that proof to ensure that the KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment, proof To do so, we evaluate the blob data polynomial at a random point `x` to get a value `y`. We then use the KZG proof to ensure that the commited polynomial (i.e. the commitment) also evaluates to `y` at `x`. If the check passes, it means that the KZG commitment matches the polynomial represented by the blob data. This is significantly faster since evaluating the blob data polynomial at a random point using the Barycentric formula can be done efficiently with only field operations (see https://hackmd.io/@vbuterin/barycentric_evaluation). Then, verifying a KZG proof takes two pairing operations (which take about 0.6ms each). This brings the total verification cost to about 2 ms per blob. With some additional optimizations (using linear combination tricks as the ones linked above) we can batch all the blobs together into a single efficient verification, and hence verify the entire transaction in 2.5 ms. The same techniques can be used to efficiently verify blocks on the consensus side.
asn-d6
added a commit
to asn-d6/EIPs
that referenced
this pull request
Jun 29, 2022
To validate a 4844 transaction in the mempool, the verifier checks that each provided KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment Before this patch, to do this validation, we reconstructed the commitment from the blob data (d_i above), and checked it against the provided commitment. This was expensive because computing a commitment from blob data (even using Lagrange basis) involves N scalar multiplications, where N is the number of field elements per blob. Initial benchmarking showed that this was about 40ms for N=4096 which was deemed too expensive. For more details see: https://hackmd.io/@protolambda/eip-4844-implementer-notes#Optimizations protolambda/go-ethereum#4 In this patch, we speed this up by providing a KZG proof for each commitment. The verifier can check that proof to ensure that the KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment, proof To do so, we evaluate the blob data polynomial at a random point `x` to get a value `y`. We then use the KZG proof to ensure that the commited polynomial (i.e. the commitment) also evaluates to `y` at `x`. If the check passes, it means that the KZG commitment matches the polynomial represented by the blob data. This is significantly faster since evaluating the blob data polynomial at a random point using the Barycentric formula can be done efficiently with only field operations (see https://hackmd.io/@vbuterin/barycentric_evaluation). Then, verifying a KZG proof takes two pairing operations (which take about 0.6ms each). This brings the total verification cost to about 2 ms per blob. With some additional optimizations (using linear combination tricks as the ones linked above) we can batch all the blobs together into a single efficient verification, and hence verify the entire transaction in 2.5 ms. The same techniques can be used to efficiently verify blocks on the consensus side.
kodiakhq bot
pushed a commit
to ethereum/EIPs
that referenced
this pull request
Jun 29, 2022
…ofs) (#5088) * Fix missing variables/funcs in validate_blob_transaction_wrapper() There is no `tx.message.blob_commitments` anymore, or `kzg_to_commitment()` * Introduce KZGProof as its own type instead of using KZGCommitment * Introduce high-level logic of new efficient transaction validation To validate a 4844 transaction in the mempool, the verifier checks that each provided KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment Before this patch, to do this validation, we reconstructed the commitment from the blob data (d_i above), and checked it against the provided commitment. This was expensive because computing a commitment from blob data (even using Lagrange basis) involves N scalar multiplications, where N is the number of field elements per blob. Initial benchmarking showed that this was about 40ms for N=4096 which was deemed too expensive. For more details see: https://hackmd.io/@protolambda/eip-4844-implementer-notes#Optimizations protolambda/go-ethereum#4 In this patch, we speed this up by providing a KZG proof for each commitment. The verifier can check that proof to ensure that the KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment, proof To do so, we evaluate the blob data polynomial at a random point `x` to get a value `y`. We then use the KZG proof to ensure that the commited polynomial (i.e. the commitment) also evaluates to `y` at `x`. If the check passes, it means that the KZG commitment matches the polynomial represented by the blob data. This is significantly faster since evaluating the blob data polynomial at a random point using the Barycentric formula can be done efficiently with only field operations (see https://hackmd.io/@vbuterin/barycentric_evaluation). Then, verifying a KZG proof takes two pairing operations (which take about 0.6ms each). This brings the total verification cost to about 2 ms per blob. With some additional optimizations (using linear combination tricks as the ones linked above) we can batch all the blobs together into a single efficient verification, and hence verify the entire transaction in 2.5 ms. The same techniques can be used to efficiently verify blocks on the consensus side. * Introduce polynomial helper functions for transaction validation * Implement high-level logic of aggregated proof verification * Add helper functions for aggregated proof verification Also abstract `lincomb()` out of the `blob_to_kzg()` function to be used in the verification. * Fixes after review on the consensus PR
protolambda
pushed a commit
that referenced
this pull request
Sep 16, 2022
Add BlobVersionedHashes to JSON-RPC Transaction result
protolambda
pushed a commit
that referenced
this pull request
Sep 16, 2022
The mint field was not being reported on deposit transactions over the JSON RPC interface because the field was misnamed.
protolambda
added a commit
that referenced
this pull request
Sep 16, 2022
protolambda
pushed a commit
that referenced
this pull request
Jan 4, 2023
The mint field was not being reported on deposit transactions over the JSON RPC interface because the field was misnamed.
protolambda
added a commit
that referenced
this pull request
Jan 4, 2023
nachomazzara
pushed a commit
to nachomazzara/EIPs
that referenced
this pull request
Jan 13, 2023
…ofs) (ethereum#5088) * Fix missing variables/funcs in validate_blob_transaction_wrapper() There is no `tx.message.blob_commitments` anymore, or `kzg_to_commitment()` * Introduce KZGProof as its own type instead of using KZGCommitment * Introduce high-level logic of new efficient transaction validation To validate a 4844 transaction in the mempool, the verifier checks that each provided KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment Before this patch, to do this validation, we reconstructed the commitment from the blob data (d_i above), and checked it against the provided commitment. This was expensive because computing a commitment from blob data (even using Lagrange basis) involves N scalar multiplications, where N is the number of field elements per blob. Initial benchmarking showed that this was about 40ms for N=4096 which was deemed too expensive. For more details see: https://hackmd.io/@protolambda/eip-4844-implementer-notes#Optimizations protolambda/go-ethereum#4 In this patch, we speed this up by providing a KZG proof for each commitment. The verifier can check that proof to ensure that the KZG commitment matches the polynomial represented by the corresponding blob data. | d_1 | d_2 | d_3 | ... | d_4096 | -> commitment, proof To do so, we evaluate the blob data polynomial at a random point `x` to get a value `y`. We then use the KZG proof to ensure that the commited polynomial (i.e. the commitment) also evaluates to `y` at `x`. If the check passes, it means that the KZG commitment matches the polynomial represented by the blob data. This is significantly faster since evaluating the blob data polynomial at a random point using the Barycentric formula can be done efficiently with only field operations (see https://hackmd.io/@vbuterin/barycentric_evaluation). Then, verifying a KZG proof takes two pairing operations (which take about 0.6ms each). This brings the total verification cost to about 2 ms per blob. With some additional optimizations (using linear combination tricks as the ones linked above) we can batch all the blobs together into a single efficient verification, and hence verify the entire transaction in 2.5 ms. The same techniques can be used to efficiently verify blocks on the consensus side. * Introduce polynomial helper functions for transaction validation * Implement high-level logic of aggregated proof verification * Add helper functions for aggregated proof verification Also abstract `lincomb()` out of the `blob_to_kzg()` function to be used in the verification. * Fixes after review on the consensus PR
protolambda
added a commit
that referenced
this pull request
Mar 3, 2023
This commit squashes the op-geth fork history into a more maintainable diff for rebasing upon upstream geth. reference-optimistic-geth changes (origins of op-geth in early Bedrock development stage): - Deposit TX Type - Enable deposit tx in EVM/tx pool - Change deposit nonce to not be the max nonce - Extend PayloadAttributesV1 with a Transactions field - Force deposits at the start of each L2 block - Fix height check - noTxPool flag, reproduce block in verifier mode without tx pool interference - Fix RPC json marshalling (ref op-geth PR 4) - Deposit txs block height check in block body validation (ref op-geth PR 5) - core: do not try to reinject deposit txs into tx-pool (ref-op-geth PR 6) - deposit source hash field instead of L2 block height and tx index combination - Include invalid deposits, rewind state, but always persist mint (ethereum#10) - Provide gas to Call/Create in deposit transactions (ethereum#12) - Add docker builds (ref-op-geth PR 16, 17) - Don't panic on deposit transaction signature values or chain ID (ref-op-geth PR 18) - core: Add version to DepositTx (ref-op-geth PR 19) - Enable Geth build/lint/test in CircleCI (ref-op-geth PR 23) - core: Include guaranteed gas in the gas pool (ref-op-geth PR 21) - core: handle base fee, l1 availability fee, tx fee (ref-op-geth PR 27) - fix: deposit tx hash - fix l1 fee cache, rpc, tracing and tx pool - core: remove deposit-tx sub-type (a.k.a. deposit version byte) - eth/catalyst: allow engine user to reorg own chain - miner: restore ability to reorg deep as block builder - params: print Optimism consensus type in banner - core/types: remove unused protected() method, see upstream PR 23376 - core: do not mutate original balance value in tx pool l1 cost adjustment - core: subtract deposit gas from pool, so other txs do not use the same gas. And fail tx processing if deposits reach gas limit - core/types: deposits do not tip, avoid basefee subtraction - Unmeter the L1 Attributes Transaction - miner: handle force tx errors as critical, clean up diff - ci: Switch branch - eth,miner: return STATUS_INVALID when failing to process forced transactions in request (ref-op-geth PR 40) - verifier: forward tx to sequencer based on flag - txpool: add flag to disable tx gossip (ref-op-geth PR 42) - Add op-geth version in addition to geth version (ref-op-geth PR 43) - ci: CircleCI improvements (ref-op-geth PR 44) - Rename to op-geth - Build latest tag on optimism branch op-geth changes: - Expose cache config in simulated backend (#2) - Add EIP-1559 parameters - eth/catalyst: update payload id computation (#1) - make eip1559 configurable (#4) - post-merge network should not log warnings about missing transition information (#5) - Make the simulator more configurable (ethereum#6) - fix OPB-6 - IsDepositTx check instead of artificial nonce value check (ethereum#7) - Simulated backend - enable proof of stake consensus type and fix performance issue (ethereum#8) - accounts: simulated backend consensus engine option and immediate tx indexing - consensus/beacon: recognize all blocks as reached TTD with 0 TTD in chain config - Add --rollup.historicalhttp CLI flag and fix backend iface - Flags and interfaces for historical RPC requests (ethereum#12) - Redirect historical RPC requests (ethereum#13) - Use the pre-existing ethereum.NotFound error (ethereum#18) - Add historical endpoint to TraceBlockByNumber and TraceBlockByHash (ethereum#19) - Add historical endpoint to TraceTransaction (ethereum#20) - Add historical endpoint to TraceCall (ethereum#21) - optimism: fee params from info txi, update l1 cost func GPO params read (ethereum#15) - add hardcoded addresses for fee payouts (ethereum#23) - dynamic gas limit via engine API (ethereum#22) Co-authored-by: Matthew Slipper <me@matthewslipper.com> Co-authored-by: Joshua Gutow <jgutow@oplabs.co> Co-authored-by: protolambda <proto@protolambda.com> Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com> Co-authored-by: Maurelian <maurelian@protonmail.ch>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same logic should be used for
verify_blobs()
on the CL.