feat: add gas costs to TxEffect #5337

just-mitch · 2024-03-20T15:58:43Z

Built to the description provided in the yellowpaper changes.

TLDR: Adds da_gas_used throughout the kernel circuits, and includes it in the TxEffect that gets published, as well as the TxReceipt. See the token contract e2e test for end flow.

Immediate future work is to update/accumulate the gas during public kernel execution. See #5507

Fix #4101
Fix #5306

just-mitch · 2024-03-20T15:59:01Z

docs(yellow-paper): private DA metering #5388
feat: add gas costs to TxEffect #5337 👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @just-mitch and the rest of your teammates on Graphite

github-actions · 2024-03-23T19:46:32Z

Changes to circuit sizes

Generated at commit: 4f0376f61985bab167d9231486ddb818b7f1548a, compared to commit: 0249737e8b925406e9278b80fc7adc0f6ab5468d

🧾 Summary (100% most significant diffs)

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
public_kernel_setup	+552 ❌	+1.88%	+559 ❌	+0.31%
public_kernel_teardown	+552 ❌	+1.88%	+559 ❌	+0.31%
private_kernel_tail_simulated	+1 ❌	+0.22%	+2 ❌	+0.22%
public_kernel_app_logic	+552 ❌	+0.98%	+560 ❌	+0.18%
rollup_base	+65 ❌	+0.04%	+2,978 ❌	+0.17%
private_kernel_init_simulated	+1 ❌	+0.11%	+2 ❌	+0.11%
private_kernel_inner_simulated	+1 ❌	+0.11%	+2 ❌	+0.11%
public_kernel_teardown_simulated	+1 ❌	+0.11%	+2 ❌	+0.11%
public_kernel_app_logic_simulated	+1 ❌	+0.11%	+2 ❌	+0.11%
public_kernel_setup_simulated	+1 ❌	+0.11%	+2 ❌	+0.11%
public_kernel_tail_simulated	+1 ❌	+0.11%	+2 ❌	+0.11%
public_kernel_tail	+198 ❌	+0.16%	+277 ❌	+0.08%
private_kernel_tail	+296 ❌	+0.08%	+300 ❌	+0.04%
private_kernel_inner	+2 ❌	+0.00%	+5 ❌	+0.00%
private_kernel_init	+1 ❌	+0.00%	+2 ❌	+0.00%

Full diff report 👇

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
public_kernel_setup	29,896 (+552)	+1.88%	183,657 (+559)	+0.31%
public_kernel_teardown	29,981 (+552)	+1.88%	183,743 (+559)	+0.31%
private_kernel_tail_simulated	454 (+1)	+0.22%	911 (+2)	+0.22%
public_kernel_app_logic	56,627 (+552)	+0.98%	315,314 (+560)	+0.18%
rollup_base	171,546 (+65)	+0.04%	1,733,721 (+2,978)	+0.17%
private_kernel_init_simulated	905 (+1)	+0.11%	1,813 (+2)	+0.11%
private_kernel_inner_simulated	905 (+1)	+0.11%	1,813 (+2)	+0.11%
public_kernel_teardown_simulated	908 (+1)	+0.11%	1,819 (+2)	+0.11%
public_kernel_app_logic_simulated	908 (+1)	+0.11%	1,819 (+2)	+0.11%
public_kernel_setup_simulated	908 (+1)	+0.11%	1,819 (+2)	+0.11%
public_kernel_tail_simulated	908 (+1)	+0.11%	1,819 (+2)	+0.11%
public_kernel_tail	125,169 (+198)	+0.16%	359,247 (+277)	+0.08%
private_kernel_tail	373,773 (+296)	+0.08%	839,883 (+300)	+0.04%
private_kernel_inner	95,869 (+2)	+0.00%	520,377 (+5)	+0.00%
private_kernel_init	48,603 (+1)	+0.00%	349,515 (+2)	+0.00%

socket-security · 2024-03-26T17:16:57Z

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report↗︎

just-mitch · 2024-03-28T15:08:34Z

yarn-project/prover-client/src/orchestrator/block-building-helpers.ts

@@ -515,6 +517,15 @@ export async function executeBaseRollupCircuit(
 ): Promise<[BaseOrMergeRollupPublicInputs, Proof]> {
  logger?.(`Running base rollup for ${tx.hash}`);
  const rollupOutput = await simulator.baseRollupCircuit(inputs);
+  const txEffect = toTxEffect(tx);
+  const txEffectHash = txEffect.hash();
+  if (!rollupOutput.txsEffectsHash.toBuffer().equals(txEffectHash)) {


Just flagging: I believe at present this would kill the whole proving ticket, which is probably not the behavior we want when one TX is invalid.

This addition at least enables a "fail fast" in that the ticket gets killed as soon as it knows that a single transaction would have invalidated the block, instead of waiting until the after the root rollup runs.

AztecBot · 2024-03-28T15:16:08Z

Benchmark results

Metrics with a significant change:

circuit_simulation_time_in_ms (base-parity): 3,579 (-17%)
circuit_input_size_in_bytes (base-rollup): 165,768 (+42%)
circuit_input_size_in_bytes (public-kernel-app-logic): 35,260 (-26%)
circuit_input_size_in_bytes (public-kernel-tail): 40,936 (-23%)
circuit_input_size_in_bytes (public-kernel-teardown): 35,260 (-26%)
circuit_input_size_in_bytes (public-kernel-setup): 35,260 (-26%)
circuit_output_size_in_bytes (private-kernel-ordering): 14,304 (-64%)
circuit_output_size_in_bytes (public-kernel-app-logic): 28,225 (-31%)
circuit_output_size_in_bytes (public-kernel-tail): 28,225 (+113%)
circuit_output_size_in_bytes (public-kernel-teardown): 28,225 (-31%)
circuit_output_size_in_bytes (public-kernel-setup): 28,225 (-31%)
tx_size_in_bytes (0): 14,976 (-63%)
l2_block_public_tx_process_time_in_ms (8): 4,024 (-16%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit f7e6cc86 and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	676	676	676
l1_rollup_calldata_gas	6,412	6,412	6,388 (-1%)
l1_rollup_execution_gas	585,745	585,745	585,721
l2_block_processing_time_in_ms	1,321 (-1%)	4,709 (-1%)	9,205 (-1%)
note_successful_decrypting_time_in_ms	235 (-10%)	566 (-4%)	1,007
note_trial_decrypting_time_in_ms	37.6 (-40%)	74.5 (+7%)	24.6 (-3%)
l2_block_building_time_in_ms	11,451 (-4%)	41,989 (-4%)	83,017 (-2%)
l2_block_rollup_simulation_time_in_ms	7,400 (+4%)	26,078 (+5%)	50,844 (+7%)
l2_block_public_tx_process_time_in_ms	⚠️ 4,024 (-16%)	15,860 (-15%)	32,076 (-13%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	13,906 (-2%)	26,982 (-1%)
note_history_successful_decrypting_time_in_ms	1,346 (-1%)	2,466 (+1%)
note_history_trial_decrypting_time_in_ms	82.1 (-38%)	111 (-3%)
node_database_size_in_bytes	18,636,880	35,078,224
pxe_database_size_in_bytes	29,859	59,414

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	182 (+1%)	44,379	28,222 (+8%)
private-kernel-ordering	171 (+6%)	52,888 (+4%)	⚠️ 14,304 (-64%)
base-parity	⚠️ 3,579 (-17%)	128	311
root-parity	1,269 (+8%)	1,244	311
base-rollup	15,080 (+4%)	⚠️ 165,768 (+42%)	861
root-rollup	49.4 (-2%)	4,359	725
private-kernel-inner	226 (+3%)	73,803 (+3%)	28,222 (+8%)
public-kernel-app-logic	98.0 (-18%)	⚠️ 35,260 (-26%)	⚠️ 28,225 (-31%)
public-kernel-tail	142 (-13%)	⚠️ 40,936 (-23%)	⚠️ 28,225 (+113%)
merge-rollup	8.61 (-25%)	2,568	861
public-kernel-teardown	93.1 (-20%)	⚠️ 35,260 (-26%)	⚠️ 28,225 (-31%)
public-kernel-setup	91.9 (-20%)	⚠️ 35,260 (-26%)	⚠️ 28,225 (-31%)

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves	32 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.0 (-2%)	16.0 (-2%)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.6	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.583 (-1%)	0.494 (-2%)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	46.2 (-4%)	72.2 (-1%)	230 (-1%)	448	870	1,720 (-1%)	N/A
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	96.0	159	543	1,055	2,079	4,127	N/A
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.473 (-4%)	0.444 (-1%)	0.420	0.418	0.414	0.412 (-1%)	N/A
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	54.7	106 (-1%)	335 (-2%)	665	1,308 (-1%)	2,603 (-1%)	N/A
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	105	207	691	1,363	2,707	5,395	N/A
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.481	0.480 (-1%)	0.457 (-1%)	0.456	0.455	0.453	N/A
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	61.4
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	109
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.535

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	⚠️ 14,976 (-63%)	495,328 (-1%)

Transaction size based on fee payment method

Metric	none fee payment method	native fee payment method	fpc_public fee payment method	fpc_private fee payment method
tx_with_fee_size_in_bytes	781	913 (+1%)	1,169 (+1%)	1,385 (+1%)

Transaction processing duration by data writes.

Metric	0 new note hashes	1 new note hashes	2 new note hashes
tx_pxe_processing_time_ms	1,818 (+5%)	1,125 (+2%)	5,551 (+2%)

Metric	0 public data writes	1 public data writes	2 public data writes	3 public data writes	4 public data writes	5 public data writes	8 public data writes
tx_sequencer_processing_time_ms	9.54	498 (-14%)	386 (-12%)	918 (-14%)	557 (-6%)	1,497 (-14%)	553 (-6%)

socket-security · 2024-04-03T12:15:45Z

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@types/estree@1.0.2	None	`0`	25.7 kB	types
npm/@types/express-serve-static-core@4.17.37	None	`+4`	3.99 MB	types
npm/@types/express@4.17.19	None	`+4`	3.95 MB	types
npm/@types/mime@1.3.3	None	`0`	4.09 kB	types
npm/@types/serve-static@1.15.3	None	`+2`	3.94 MB	types
npm/@webassemblyjs/ast@1.11.6	None	`+5`	341 kB	xtuc
npm/@webassemblyjs/wasm-parser@1.11.6	None	`+7`	329 kB	xtuc
npm/accepts@1.3.8	None	`+3`	268 kB	dougwilson
npm/follow-redirects@1.15.3	network	`0`	28.6 kB	rubenverborgh
npm/obuf@1.1.2	None	`0`	19.1 kB	indutny
npm/rc@1.2.8	environment, filesystem	`+2`	31.5 kB	dominictarr

🚮 Removed packages: npm/safe-buffer@5.2.1, npm/safer-buffer@2.1.2, npm/semver@7.5.4, npm/serialize-javascript@6.0.1, npm/shell-quote@1.8.1, npm/signal-exit@3.0.7, npm/space-separated-tokens@1.1.5, npm/string-width@5.1.2, npm/strip-ansi@6.0.1, npm/style-to-object@0.3.0, npm/svgo@2.8.0, npm/tapable@2.2.1, npm/terser-webpack-plugin@5.3.9, npm/terser@5.21.0, npm/tslib@2.6.2, npm/typescript@4.9.5, npm/unified@9.2.2, npm/unist-builder@2.0.3, npm/unist-util-visit@2.0.3, npm/unpipe@1.0.0, npm/util-deprecate@1.0.2, npm/vfile-location@3.2.0, npm/webpack-sources@3.2.3, npm/xtend@4.0.2, npm/yaml@1.10.2

View full report↗︎

socket-security · 2024-04-03T12:15:48Z

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@types/estree@1.0.2	None	`0`	25.7 kB	types
npm/@types/express-serve-static-core@4.17.37	None	`+4`	3.99 MB	types
npm/@types/express@4.17.19	None	`+4`	3.95 MB	types
npm/@types/mime@1.3.3	None	`0`	4.09 kB	types
npm/@types/serve-static@1.15.3	None	`+2`	3.94 MB	types
npm/@webassemblyjs/ast@1.11.6	None	`+5`	341 kB	xtuc
npm/@webassemblyjs/wasm-parser@1.11.6	None	`+7`	329 kB	xtuc
npm/accepts@1.3.8	None	`+3`	268 kB	dougwilson
npm/follow-redirects@1.15.3	network	`0`	28.6 kB	rubenverborgh
npm/obuf@1.1.2	None	`0`	19.1 kB	indutny
npm/rc@1.2.8	environment, filesystem	`+2`	31.5 kB	dominictarr

🚮 Removed packages: npm/safe-buffer@5.2.1, npm/safer-buffer@2.1.2, npm/semver@7.5.4, npm/serialize-javascript@6.0.1, npm/shell-quote@1.8.1, npm/signal-exit@3.0.7, npm/space-separated-tokens@1.1.5, npm/string-width@5.1.2, npm/strip-ansi@6.0.1, npm/style-to-object@0.3.0, npm/svgo@2.8.0, npm/tapable@2.2.1, npm/terser-webpack-plugin@5.3.9, npm/terser@5.21.0, npm/tslib@2.6.2, npm/typescript@4.9.5, npm/unified@9.2.2, npm/unist-builder@2.0.3, npm/unist-util-visit@2.0.3, npm/unpipe@1.0.0, npm/util-deprecate@1.0.2, npm/vfile-location@3.2.0, npm/webpack-sources@3.2.3, npm/xtend@4.0.2, npm/yaml@1.10.2

View full report↗︎

just-mitch mentioned this pull request Mar 20, 2024

feat!: Mark transactions as reverted on L1 #5226

Merged

This was referenced Mar 20, 2024

chore: rename reverted to revertCode #5301

Merged

chore: reduce size of revert code from Field to u8 #5309

Merged

just-mitch force-pushed the shrink-revert-code-to-u8 branch from c8511f9 to 9e1d351 Compare March 20, 2024 16:01

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch from 5c95101 to 606fdd5 Compare March 20, 2024 16:01

just-mitch force-pushed the shrink-revert-code-to-u8 branch 2 times, most recently from f4530d4 to e91c5a1 Compare March 20, 2024 18:51

Base automatically changed from shrink-revert-code-to-u8 to master March 20, 2024 18:56

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch 3 times, most recently from f33cdd4 to f1663fc Compare March 22, 2024 11:42

just-mitch changed the base branch from master to docs-da-metering-design March 22, 2024 11:42

just-mitch mentioned this pull request Mar 22, 2024

docs(yellow-paper): private DA metering #5388

Merged

just-mitch marked this pull request as ready for review March 22, 2024 11:42

just-mitch marked this pull request as draft March 22, 2024 11:42

just-mitch force-pushed the docs-da-metering-design branch 2 times, most recently from 4302fd3 to c820cf8 Compare March 22, 2024 15:15

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch 2 times, most recently from e9a42b3 to 8b29014 Compare March 22, 2024 22:26

just-mitch changed the base branch from docs-da-metering-design to master March 22, 2024 22:27

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch from 8b29014 to cdd0d2b Compare March 23, 2024 18:35

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch 3 times, most recently from e093115 to 5caf400 Compare March 26, 2024 17:13

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch 2 times, most recently from 4f20cc7 to a9d6d84 Compare March 28, 2024 14:54

just-mitch commented Mar 28, 2024

View reviewed changes

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch 4 times, most recently from 61fb9d3 to d71f161 Compare March 28, 2024 18:22

just-mitch marked this pull request as ready for review March 28, 2024 20:27

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch from d71f161 to 95438ff Compare April 3, 2024 12:14

plumb da_gas_used through kernel and rollup. Basic computation.

7f428b3

just-mitch force-pushed the 03-19-feat_add_gas_costs_to_txeffect branch from 95438ff to 7f428b3 Compare April 3, 2024 15:13

just-mitch marked this pull request as draft April 3, 2024 16:03

just-mitch closed this Apr 26, 2024

ludamad deleted the 03-19-feat_add_gas_costs_to_txeffect branch August 22, 2024 15:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add gas costs to TxEffect #5337

feat: add gas costs to TxEffect #5337

just-mitch commented Mar 20, 2024 •

edited

Loading

just-mitch commented Mar 20, 2024 •

edited

Loading

github-actions bot commented Mar 23, 2024 •

edited

Loading

socket-security bot commented Mar 26, 2024 •

edited

Loading

just-mitch Mar 28, 2024 •

edited

Loading

AztecBot commented Mar 28, 2024 •

edited

Loading

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

socket-security bot commented Apr 3, 2024

socket-security bot commented Apr 3, 2024

feat: add gas costs to TxEffect #5337

feat: add gas costs to TxEffect #5337

Conversation

just-mitch commented Mar 20, 2024 • edited Loading

just-mitch commented Mar 20, 2024 • edited Loading

github-actions bot commented Mar 23, 2024 • edited Loading

Changes to circuit sizes

🧾 Summary (100% most significant diffs)

socket-security bot commented Mar 26, 2024 • edited Loading

just-mitch Mar 28, 2024 • edited Loading

Choose a reason for hiding this comment

AztecBot commented Mar 28, 2024 • edited Loading

Benchmark results

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

socket-security bot commented Apr 3, 2024

socket-security bot commented Apr 3, 2024

just-mitch commented Mar 20, 2024 •

edited

Loading

just-mitch commented Mar 20, 2024 •

edited

Loading

github-actions bot commented Mar 23, 2024 •

edited

Loading

socket-security bot commented Mar 26, 2024 •

edited

Loading

just-mitch Mar 28, 2024 •

edited

Loading

AztecBot commented Mar 28, 2024 •

edited

Loading