Define desirable runtime performance for Aurora contracts

[MaksymZavershynskyi]

We want EVM contract to be cheaper. However, without knowing how much cheaper we want it to be, it is hard to make decisions on which technical direction to pursue in medium and long term.

It would help if Aurora team gave us several examples of their use cases with current gas usage, minimum acceptable gas usage for these contracts, and degree of importance.

[MaksymZavershynskyi]

@joshuajbouw @ailisp said the following:

• Our current gas limit of 200Tgas per contract call maps to 750k of EVM gas;
• Aurora needs to be able to have transactions with 8M EVM gas.

It means that we need to make Aurora contracts 10x cheaper. Can you confirm? CC @artob

Will 3M EVM gas be sufficient for now or will it still be blocking Aurora?

[birchmd]

@nearmax It is not quite as simple as this because there is not a clean, linear mapping between EVM gas and NEAR gas.

There are some 4M EVM gas transactions which we can execute today (they cost under 150 Tgas on NEAR). These are deploy transactions, so it points to storage costs being much lower on NEAR relative to Ethereum.

However, there are also some 600k EVM gas transactions which we cannot execute today (they cost more than 200 Tgas on NEAR), for example some from uniswap contract.

And even more extreme example is the ecpair precompile, which costs under 200k EVM gas, but nearly 2000 Tgas on NEAR! (We might need to do what we did with ecrecover for that one)

In short, I do not think there is a meaningful measure of how much EVM gas is sufficient for Aurora because it does not translate into a NEAR gas limit. A blanket 10x improvement on the NEAR side will certainly be welcomed, but we may need to still target specific use cases with additional performance improvements, depending on the needs of partners. We are in the process of gathering specific examples from partners which do not work today so we can look into what specific performance improvements may help.

[bowenwang1996]

@birchmd that makes sense. I wonder whether the outstanding examples you mentioned are because they mostly consist of wasm operations. Would be nice to see a breakdown of gas usage in the two examples you mentioned (uniswap and ecpair). cc @matklad

[birchmd]

@birchmd that makes sense. I wonder whether the outstanding examples you mentioned are because they mostly consist of wasm operations. Would be nice to see a breakdown of gas usage in the two examples you mentioned (uniswap and ecpair). cc @matklad

The breakdown of uniswap is linked in the above comment; it is 86% wasm execution.

The breakdown of a transaction calling ecpair is as follows

------------------------------
Total gas:      1916794660830105
Host gas:          7688486937927 [0.004% total]
Action gas:                    0 [0% total]
Wasm execution: 1909106173892178 [99.6% total]
------ Host functions --------
base -> 34155086319 [0% host]
contract_compile_base -> 35445963 [0% host]
contract_compile_bytes -> 206893727250 [2% host]
read_memory_base -> 195739740000 [2% host]
read_memory_byte -> 6017510139 [0% host]
write_memory_base -> 98132820135 [1% host]
write_memory_byte -> 20120503764 [0% host]
read_register_base -> 90617946696 [1% host]
read_register_byte -> 734385462 [0% host]
write_register_base -> 137545079328 [1% host]
write_register_byte -> 53788329036 [0% host]
utf8_decoding_base -> 3111779061 [0% host]
utf8_decoding_byte -> 18369570177 [0% host]
keccak256_base -> 17638473825 [0% host]
keccak256_byte -> 2490648180 [0% host]
ecrecover_base -> 3365369625000 [43% host]
log_base -> 3543313050 [0% host]
log_byte -> 831523833 [0% host]
storage_write_base -> 770360832000 [10% host]
storage_write_key_byte -> 13250778996 [0% host]
storage_write_value_byte -> 12035193132 [0% host]
storage_write_evicted_byte -> 10277538240 [0% host]
storage_read_base -> 2366987521500 [30% host]
storage_read_key_byte -> 21078674973 [0% host]
storage_read_value_byte -> 75681235440 [0% host]
storage_remove_base -> 160419091500 [2% host]
storage_remove_key_byte -> 2522545344 [0% host]
storage_remove_ret_value_byte -> 738019584 [0% host]
------ Actions --------
------------------------------

You can see it is 99.6% wasm execution.

[joshuajbouw]

It looks like we have certainly hit the deploy target, especially considering the patch to the benchmark is incredibly promising that we have no issues there. Simply we have a problem with the gas cost with the runtime of a contract itself. That is a bit trickier as it there is no 1 to 1 translation from NEAR to ETH which is expected. For example, we could have one transaction that could be considerably lower in the equivalent in NEAR Tgas than ETH gas, and vice versa.

The performance improvements that had been achieved so far are still substantial of course.

@birchmd covered basically all those points already though above.

We do know right now that we have more than enough gas for a deployment. As per runtime, that is considerably trickier. While we should have enough gas to unblock partners for execution. @birchmd has access to the partner's contract and is able to get more granular details about their particular methods gas costs.

[joshuajbouw]

Generally as discussed, it looks like we are well beyond our deployment target which is awesome. However when it comes to execution, as described above, this gets tricky.

• Current 8x target is acceptable to unblock partners. Unfortunately given the nature of the unknown when it comes to an execution, this may need to be looked at. For now however, 8x is acceptable.
• Deployment target is well beyond hit.

[birchmd]

@birchmd has access to the partner's contract and is able to get more granular details about their particular methods gas costs.

The only partner code I was given access to didn't actually have a gas limit problem; they were encountering a different error and not interpreting it correctly. The only measurable examples of we have exceeding the NEAR gas limit with Aurora transactions are the uniswap and ecpair ones listed above. I'll post here if we find more examples through working with partners.

[bowenwang1996]

For now however, 8x is acceptable.

@joshuajbouw what do you mean by "8x"?

[joshuajbouw]

For now however, 8x is acceptable.

@joshuajbouw what do you mean by "8x"?

8x our current benchmark, which is what is possible right now.

[joshuajbouw]

After talking with @bowenwang1996 concretely it was asked for 5 contracts to be benchmarked. This is going to be worked on as soon as we put out some fires on our end.

@birchmd is going to put these together.

[birchmd]

Please see our new documentation on benchmarks: https://github.com/aurora-is-near/aurora-engine/blob/develop/doc/benchmarks.md

I believe this document is sufficient to close this issue. Please reach out if more clarity is needed.

[matklad]

Will take a closer look this week, but one concern I have is that the current setup of using nearcore as a library doesn't allow nearcore folks to easily check if changes to nearcore actually make the stuff cheaper. Ie, if I have an awesome patched version of nearcore with an optimiation which I think will benefit aurora a lot, I'll need some time to point aurora to this new nearcore, potentially fixing API incompatibility issues. Not sure what's the best solution here. Clearly, the core problem is that aurora has to use nearcore as a library to write a benchmark...

[birchmd]

Thanks @matklad .

One solution to this problem could be to spin up a local near node for benchmarking instead of using it as a library. In this case you only need to recompile neard with new changes. However, this approach complicates the setup of the benchmarks on our end, so maybe isn't ideal.

Another solution is for Aurora to strive to stay up to date with the nearcore libraries (resolving any API incompatibility along the way) so that only a change to the Cargo.toml is needed if someone from nearcore is experimenting with a new change. I personally prefer this option because it means our benchmark flow stays self-contained (it does not require a local node to be running in the background), and I think keeping Aurora up date with nearcore should not be too big of a burden on us.

Let me know if you have other preferences or suggestions.

[ailisp]

@birchmd I read and follow https://github.com/aurora-is-near/aurora-engine/blob/develop/doc/benchmarks.md. All tests mentioned are pass or fail as documented and tests make sense and document is really well written!

As for not use nearcore as a library, I think the gas benchmark can be obtained via treat nearcore as a rpc endpoint, but for the time benchmark - it's not possible. So Michael's solution sounds reasonable 🤔