docs: gas and accounting

[just-mitch]

Please read contributing guidelines and remove this line.

[just-mitch]

• docs: gas and accounting #5855 👈
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @just-mitch and the rest of your teammates on Graphite

[AztecBot]

Docs Preview

Hey there! 👋 You can check your preview at https://662b99adcd3a97c044c8cca5--aztec-docs-dev.netlify.app

[spalladino]

Awesome work! Thanks for taking the time to spec all this out.

[spalladino]

Question: are these messages published in plain to the DA layer, or their hashes?

[just-mitch]

Short answer: on TxEffect we have public l2ToL1Msgs: Fr[], which are truncated sha256.

Explanation:

My understanding is that you can publish whatever you want, so long as it fits in a field. We have the following on public and private contexts:

    pub fn message_portal(&mut self, recipient: EthAddress, content: Field) {
        // docs:end:context_message_portal
        let message = L2ToL1Message { recipient, content };
        self.new_l2_to_l1_msgs.push(message);
    }

And it is used like context.message_portal(context.this_portal_address(), note.get_commitment());

We later we compute a hash as while running the public kernels within propagate_new_l2_to_l1_messages:

    let inputs = [
        contract_address.to_field(), rollup_version_id, message.recipient.to_field(), chain_id, message.content
    ];
   // compute sha256, then truncate

Then we accumulate the hashes of the 2 (potentially zero/empty) L2->L1 messages per transaction in the base rollup (let out_hash = compute_kernel_out_hash(self.kernel_data.public_inputs.end);), and finally insert the accumulated hashes across all TXs in the root (let new_l1_to_l2_message_tree_snapshot = append_only_tree::insert_subtree_to_snapshot_tree(...))

Then on L1 we make available the root of this tree available, and apps are free to submit membership proofs to consume messages. (Outbox.sol)

[spalladino]

Got it, thanks for the explainer! I'd say let's then rename this from "new l2 -> l1 messages" to "new l2 -> l1 messages hashes".

[spalladino]

Why is the teardown function differentiated here, if it's also an enqueued public function call?

[just-mitch]

I am expecting that the teardown function hash will occupy its own space within the kernel public inputs:

• PrivateKernelCircuitPublicInputs.public_teardown_function_hash
• PrivateKernelCircuitPublicInputs.PrivateAccumulatedData.public_call_stack

Then, when a Tx is submitted, it will have:

• enqueuedPublicFunctionCalls: PublicCallRequest[]
• publicTeardownFunctionCall: PublicCallRequest

[spalladino]

Hmm I'd push back on this, I think it's easier if we merge it together with other public call requests. We can discuss sync!

[just-mitch]

Just for completeness (and other people reading), this is resolved in later comments.

[spalladino]

Should we place this into a PublicConstantData, since it's a piece of data that doesn't change once it reaches public-land?

[just-mitch]

That might be a good place to group fee_payer and public_teardown_function_hash!

[spalladino]

I'm thinking we may also need to place the block_fees in the PublicConstantData, so it gets copied over from one kernel to another, and used to validate the global variables injected into the public app circuit.

[just-mitch]

I'm imagining that when we get to a place where the feePer values change per block, the new values will be calculated as part of the root rollup, and published as the block header of the current block (call it block 1). Then block 2 will use block 1's feePer values, which it references with constants.historical_header.global_variables.gas_fees. Does that not work?

[spalladino]

It works, but it feels odd that the gas fees for a given block are stored in the previous block.

[spalladino]

Also, it gives us less time to react to changes in L1 gas pricing, in case that becomes an important factor in our gas prices.

[just-mitch]

Wouldn't we want every transaction in a block to use the same feePer values?

If we were to compute at the time of block building, wouldn't we need some new circuit that gets run before any of the individual transactions?

[spalladino]

I'd say yes, since I'm not sure we can infer it from the fee-paying storage update

[just-mitch]

My concern is what it leaks: I now know that this TX paid with this FPC, or was sponsored by this Dapp.

The nice thing is that you get clarity on how much people are using this feature (FPCs, dapp-sponsoring), and maybe its okay since we obviously don't know who did the thing, or what it was.

Its semi-moot since we're leaking to the sequencer though, so maybe its better to include to avoid some advantage the sequencer could obtain from this.

At the same time, I don't know why anyone needs to know who paid, as long as the protocol enforces that it was paid: would it be better to just save the user the additional cost in terms of DA overhead if we don't need to publish it?

[spalladino]

Its semi-moot since we're leaking to the sequencer though, so maybe its better to include to avoid some advantage the sequencer could obtain from this.

This is a good point for a more general discussion: if there's a tx that's "private" to a sequencer, is it ok if the sequencer ends up with knowledge not shared with the rest of the world?

[just-mitch]

Exactly. It's a question of how much alpha there is in in that "private" data, how egalitarian we care to be, and how cheap we want transactions to be.

Maybe an ideal case is one where we don't force the sequencer to publish it, but have some mechanism for them to be incentivized to publish it (along with other useful "private" metadata). Obviously that's pie in the sky talk, but I'd be curious to know how it could theoretically be done: an incentivized side channel for valuable (and ideally proven) "exhaust data".

In the immediate term, do you still say "yes" to fee_payer on TxEffects? I think I'm against it, but I'm not sure.

[spalladino]

Let's default to the least effort if we're in doubt, so let's not add it.

[spalladino]

even if the transaction does not have a teardown phase, the gas will still be consumed.

What if we just validate in the kernel tail that teardown gas must be zero if there's no teardown function enqueued?

[just-mitch]

Only arguments I can produce are:

• consistency: you always lose teardown limits
• future proof: what if we want private teardowns? If we put up guardrails to stop you when you erroneously provide teardown limits, but we add a private teardown in the future, it will simply no longer error if you erroneously provide teardown limits.

But if there's no way we add private teardown, then the argument is flimsy and I agree with you.

[PhilWindle]

I don't really agree with this approach of effectively using all of the teardown gas up front. To take your example, if my app logic uses 920 DA gas and teardown uses 50 DA gas, my transaction will revert, despite not breaching either the teardown or global gas limit.

Whilst the user gets charged for the complete teardown gas limit, it should still be correctly metered for the purpose of reverting.

[PhilWindle]

Additionally, this metered value of teardown gas is what should go into the block level gas used published to L1.

[just-mitch]

In that case, the amount of gas that is available to teardown is the minimum between the teardown limit specified, and the global limit specified less non/revertible gas used.

A problem with this is that it removes any guarantee we could supply the FPC/dapp running teardown: as part of private setup it can no longer look at the amount the user has allocated for teardown and be confident that it will actually receive that amount, since app logic can bleed into it without its knowledge/control.

this metered value of teardown gas is what should go into the block level gas used published to L1.

I agree with this. I will make that point more clear.

[spalladino]

But a sequencer can though can't they? They can maniuplate the block's gas_used to be whatever they want (below the limit) without having to do the work or incur the corresponding DA cost. Or have I misunderstood?

True, a sequencer could include a spurious tx with an arbitrarily high teardown gas, and fill the block with that. Whether it's costly for them or not depends on whether they keep the associated fees or not, and whether the fees are dictated by the protocol or not.

As for the actual compute cost (ie doing the work), actually running those functions or not is not a big effort, so I doubt it makes much of a difference. Eg gas limits in ethereum are tight not to protect the sequencer from doing too much work, but for ensuring that every node in the network can follow along.

But I see your point in that it's odd having a gas limit that behaves as a limit, and another gas limit that's spent automatically regardless of the work done, and having the gas_used not reflective of the gas actually used.

I think we may be able to make things clearer if we make teardown_gas_limit not part of the total gas_limit. We could rename teardown_gas_limit to teardown_gas_allocation, have gas_used report on the gas of app+setup only, and have the total_gas equal to fixed_base_cost + gas_used + teardown_allocation. Do you think this would help?

[just-mitch]

if a tx reverts do the gas_used figures get set to the limits of the tx?

Yes

But a sequencer can though can't they?

Yes, but to no real benefit- they're charged a fee that goes back to themselves, and they're reducing the net new fees they could take in. And assuming that we adopt a max percentage change to prices from block per block, the impact on the network from a malicious sequencer that wants to increase everyone's prices should be constrained in time and size.

Maybe we don't need to publish the gas_used on the block at all.

For sure! I think the only reason to publish gas_used is to allow other people to compute the transaction fees. But maybe we ought to just publish the transaction_fee!

another gas limit that's spent automatically regardless of the work done

Though teardown_gas_limit is still a limit in that it cannot be exceeded, it's just the case that any surplus is not refunded.

I think best is to publish transaction_fee instead of gas_used + inclusion_fees, and rename teardown_gas_limit to teardown_gas_allocation to make it clear that you don't get a refund, but keep it part of gas_limit so you don't need to do math to get to the real limit.

[just-mitch]

@spalladino , @PhilWindle
What do we think here? Just publish transaction_fee and rename teardown_gas_limit to teardown_gas_allocation?

[spalladino]

+1 to the rename.

As for what to publish: publishing the resulting tx_fee is more efficient in terms of DA cost (1 field vs 1 field + 3 u32s). But publishing the gas_used and inclusion_fee may give us better insights of what public txs are doing, distribution of gas usage, etc, which we'd otherwise not get unless we instrument sequencers and have them report that info somewhere out of their own goodwill.

Hmm I think that, when in doubt, we should favor the user, and here that means cheaper txs. So I think I'd go with publishing the tx_fee. But I'm not 100% sure.

[just-mitch]

We can also reverse engineer the approximate breakdown by looking at the TxEffects. Given that, I agree that we should favor the user.

[spalladino]

I love the seriousness of the explanation with an asset named "bananas"

[spalladino]

Heads up a public authwit requires a public function execution, it cannot be set on private setup. We need to think of a different flow here.

[just-mitch]

Ah yes, the public authwit should be a separate transaction ahead of time.

[spalladino]

Maybe we could explain here a setup without the need for a public authwit? Hybrid private/public bananas?

[spalladino]

Got this one late: I'm not sure we can check this here, since the fee per gas for the block is unknown in private-land.

[just-mitch]

Yep you're right. I had been thinking that it could use the historical header, but I don't think that is the behavior we want. Looks like it is up to the mempool/sequencer to just check (and obviously base rollup).

[PhilWindle]

Thanks for this Mitch. A great writeup. Made some comments.

[PhilWindle]

I think nullifier checks should be an L2 gas cost. A per tx nullifier gas value.

[just-mitch]

Agreed! I think that this will be baked into the L2 gas cost of the NULLIFIEREXISTS AVM op code. Similar to:

• NOTEHASHEXISTS
• EMITNOTEHASH
• EMITNULLIFIER

[PhilWindle]

But isn't that only for nullifiers emitted during public calls?

[just-mitch]

Great point!
I'd suggest that in the PrivateKernelTail(ToPublic), in addition to computing DA gas, we tack on L2 gas for such things as new nullifiers. It should be a fraction of the metered gas used for the EMIT/EXISTS opcodes on the AVM.

I think we should raise this point in the docs now as a "warning", and provide the above as a possible future solution.

[just-mitch]

I added a note to this point above.

[PhilWindle]

If it's a fixed amount of work that's performed by the sequener in every block then I assume it's being rolled into the inclusion fee?

[PhilWindle]

So this is saying, the gas_used after this function call is equal to gas_limit - (gas_used_at_start_of_call + gas_used_during_call)?

[just-mitch]

Not quite. call_gas_left is equal to gas_limit - (gas_used_at_start_of_call + gas_used_during_call)

So this is saying the (non_revertible) gas_used here is equal to

gas_limit - call_gas_left - revertible_gas_used
= (gas_used_at_start_of_call + gas_used_during_call) - revertible_gas_used

[PhilWindle]

I don't really agree with this approach of effectively using all of the teardown gas up front. To take your example, if my app logic uses 920 DA gas and teardown uses 50 DA gas, my transaction will revert, despite not breaching either the teardown or global gas limit.

Whilst the user gets charged for the complete teardown gas limit, it should still be correctly metered for the purpose of reverting.

[PhilWindle]

We will want to publish the total amount of gas used in the block I suspect.

[just-mitch]

I'm not sure. If you're saying that because we will want to meter against a block target, and subsequently update these feePer values, we could instead compute the sum during the rollup circuits and publish the block with the new feePer values (which the following block would use for its fees).

I'm not sure why else we would publish them.

[PhilWindle]

Additionally, this metered value of teardown gas is what should go into the block level gas used published to L1.

[PhilWindle]

I commented more further down, but I don't think this is right.

[just-mitch]

Do you still think so?

[PhilWindle]

I'm still not sure about this. Doesn't it create a dangerous incentive for sequencers to influence transactions to fail (by e.g. including their own transactions earlier in the block) so they can bank all of the fees and incur little expense?

[just-mitch]

Agreed that it creates a negative incentive. I'd say we update it in the future to be more advantageous to the user, as doing so would require little to no interface change from this first implementation.

I'll add a note to this point.

[just-mitch]

I added a note to the public app logic section of the kernel tracking page.

[AztecBot]

Benchmark results

Metrics with a significant change:

• note_history_trial_decrypting_time_in_ms (5): 83.4 (+148%)
• note_history_trial_decrypting_time_in_ms (10): 174 (+157%)
• note_successful_decrypting_time_in_ms (8): 162 (-34%)
• note_successful_decrypting_time_in_ms (32): 528 (-18%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	772	772	772
l1_rollup_calldata_gas	6,820	6,832	6,832
l1_rollup_execution_gas	587,301	587,313	587,313
l2_block_processing_time_in_ms	1,480 (-6%)	5,334 (-6%)	10,631 (+3%)
note_successful_decrypting_time_in_ms	⚠️ 162 (-34%)	⚠️ 528 (-18%)	1,099 (-2%)
note_trial_decrypting_time_in_ms	54.1 (+44%)	27.9 (-27%)	120 (+2%)
l2_block_building_time_in_ms	18,672 (-2%)	73,056 (-2%)	145,111 (-1%)
l2_block_rollup_simulation_time_in_ms	18,467 (-2%)	72,304 (-1%)	143,656 (-1%)
l2_block_public_tx_process_time_in_ms	8,477 (-3%)	33,195 (-1%)	66,211 (-2%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	16,542 (+1%)	31,034 (+3%)
note_history_successful_decrypting_time_in_ms	1,418 (+4%)	2,696 (+5%)
note_history_trial_decrypting_time_in_ms	⚠️ 83.4 (+148%)	⚠️ 174 (+157%)
node_database_size_in_bytes	20,725,840	37,384,272
pxe_database_size_in_bytes	29,868	59,425

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	158 (+1%)	47,763	31,978
private-kernel-inner	176	80,794	31,978
private-kernel-ordering	137 (+1%)	57,732	49,313
base-parity	8.83 (-5%)	128	265
root-parity	65.2 (-1%)	1,244	265
base-rollup	11,902	110,926	957
root-rollup	49.4 (-1%)	4,551	821
public-kernel-app-logic	1,540	62,017	26,202
public-kernel-tail	4,159 (+1%)	168,566	3,812
merge-rollup	3,181 (-1%)	2,760	957
public-kernel-teardown	119	62,017	34,711
public-kernel-setup	131 (+1%)	62,017	34,711

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves	32 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.9 (-1%)	17.9	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.7	31.8	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.636 (-1%)	0.551	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	51.4 (-1%)	80.0 (-1%)	261 (-1%)	498 (-1%)	986	1,955	N/A
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	543	1,055	2,079	4,127	N/A
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.525	0.495 (-1%)	0.476 (-1%)	0.466 (-1%)	0.468	0.467	N/A
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	60.5 (-1%)	119 (-2%)	374 (-1%)	736 (-1%)	1,463 (-1%)	2,909	N/A
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	105	208	692	1,363	2,707	5,395	N/A
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.533 (-1%)	0.537 (-1%)	0.511	0.508 (-1%)	0.506	0.507	N/A
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	66.5 (-1%)
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	108
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.586 (-1%)

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	51,411	536,920

Transaction size based on fee payment method

Metric	native fee payment method	fpc_public fee payment method	fpc_private fee payment method
tx_with_fee_size_in_bytes	897	1,145	1,345

Transaction processing duration by data writes.

Metric	0 new note hashes	1 new note hashes	2 new note hashes
tx_pxe_processing_time_ms	1,219 (-2%)	859	3,470 (-1%)

Metric	1 public data writes	2 public data writes	3 public data writes	4 public data writes	5 public data writes	8 public data writes
tx_sequencer_processing_time_ms	834 (-1%)	765 (+1%)	1,363 (+1%)	901 (+1%)	2,099 (+1%)	1,081