Debug V1 algorightm and use more realistic values in gas price analysis #2129
Conversation
MitchTurner
added
the
no changelog
MitchTurner
changed the title
There was a problem hiding this comment.
Some naming suggestions and a bunch of questions from me.
Unrelated to this particular PR I find it interesting that we're using integer arithmetic throughout the entire algorithm. I would have expected this type of algorithm to be computed using floats internally, and then discretize the resulting DA price change before returning it. This would give us more fine grained control over the dynamics if we want to further tune the P and D values (and a potential I value if we want to include that down the line).
| actual_profit, | ||
| projected_profit, | ||
| pessimistic_costs, | ||
| } = best; | ||
|
|
||
| let plot_width = 640 * 2; | ||
| // let min_actual_profit = pretty(actual_profit.iter().min().unwrap()); |
| gen_noisy_signal(input, COMPONENTS) | ||
| } | ||
|
|
||
| fn arb_cost_per_byte(size: usize, update_period: usize) -> Vec<u64> { |
There was a problem hiding this comment.
What is arb short for here? "arbitrary"?
| /// the previous profit | ||
| last_last_profit: i64, |
There was a problem hiding this comment.
I find the naming a bit confusing here. How about something like:
| /// the previous profit | |
| last_last_profit: i64, | |
| /// The average profit over the second last `avg_window` blocks | |
| second_last_profit: i64, |
| pub last_profit: i64, | ||
| /// The profit before last | ||
| pub last_last_profit: i64, |
There was a problem hiding this comment.
Also I think we can improve the naming here:
| pub last_profit: i64, | |
| /// The profit before last | |
| pub last_last_profit: i64, | |
| pub last_profit: i64, | |
| /// The second last profit | |
| pub second_last_profit: i64, |
Co-authored-by: Mårten Blankfors <nilssonmartenx@gmail.com>
| @@ -180,7 +190,7 @@ fn fullness_and_bytes_per_block(size: usize, capacity: u64) -> Vec<(u64, u64)> { | |||
| let bytes_scale: Vec<_> = std::iter::repeat(()) | |||
| .take(size) | |||
| .map(|_| rng.gen_range(0.5..1.0)) | |||
| .map(|x| x * 4.0) | |||
| .map(|x| x * 0.01) | |||
There was a problem hiding this comment.
| .map(|x| x * 0.01) | |
| .map(|x| x * BYTES_SCALE_INC) |
it would be nice to avoid use of magic numbers here
| .map(noisy_eth_price) | ||
| .map(|x| x * 100. + 110.) | ||
| .map(|x| x * 5. + 5.) |
You're right. We might have been able to get away with floats here and it would be a lot easier. Something I didn't realize when I started the algorithm, but I do now, Is that this algorithm will be completely off-chain. Meaning the PoA node can use whatever values she chooses and it won't be validated on-chain. In the future, we will have gas price as part of the state transition function. This will allow it to be part of fraud proving and later, when/if we move to PoS, be verified by other nodes during consensus. Float arithmetic is not deterministic, and that is why I went with integer math because I assumed this need to be run deterministically as part of the state transition function. So, technically, right now it is YAGNI. So we could move to floats. In the future, it needs to be fully deterministic, which floats are not. That's why it's integers. |
## Version v0.36.0 ### Added - [2135](#2135): Added metrics logging for number of blocks served over the p2p req/res protocol. - [2151](#2151): Added limitations on gas used during dry_run in API. - [2188](#2188): Added the new variant `V2` for the `ConsensusParameters` which contains the new `block_transaction_size_limit` parameter. - [2163](#2163): Added runnable task for fetching block committer data. - [2204](#2204): Added `dnsaddr` resolution for TLD without suffixes. ### Changed #### Breaking - [2199](#2199): Applying several breaking changes to the WASM interface from backlog: - Get the module to execute WASM byte code from the storage first, an fallback to the built-in version in the case of the `FUEL_ALWAYS_USE_WASM`. - Added `host_v1` with a new `peek_next_txs_size` method, that accepts `tx_number_limit` and `size_limit`. - Added new variant of the return type to pass the validation result. It removes block serialization and deserialization and should improve performance. - Added a V1 execution result type that uses `JSONError` instead of postcard serialized error. It adds flexibility of how variants of the error can be managed. More information about it in FuelLabs/fuel-vm#797. The change also moves `TooManyOutputs` error to the top. It shows that `JSONError` works as expected. - [2145](#2145): feat: Introduce time port in PoA service. - [2155](#2155): Added trait declaration for block committer data - [2142](#2142): Added benchmarks for varied forms of db lookups to assist in optimizations. - [2158](#2158): Log the public address of the signing key, if it is specified - [2188](#2188): Upgraded the `fuel-vm` to `0.57.0`. More information in the [release](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.57.0). ## What's Changed * chore(p2p_service): add metrics for number of blocks requested over p2p req/res protocol by @rymnc in #2135 * Weekly `cargo update` by @github-actions in #2149 * Debug V1 algorightm and use more realistic values in gas price analysis by @MitchTurner in #2129 * feat(gas_price_service): include trait declaration for block committer data by @rymnc in #2155 * Convert gas price analysis tool to CLI by @MitchTurner in #2156 * chore: add benchmarks for varied forms of lookups by @rymnc in #2142 * Add label nochangelog on weekly cargo update by @AurelienFT in #2152 * Log consensus-key signer address if specified by @acerone85 in #2158 * chore(rocks_db): move ShallowTempDir to benches crate by @rymnc in #2168 * chore(benches): conditional dropping of databases in benchmarks by @rymnc in #2170 * feat: Introduce time port in PoA service by @netrome in #2145 * Get DA costs from predefined data by @MitchTurner in #2157 * chore(shallow_temp_dir): panic if not panicking by @rymnc in #2172 * chore: Add initial CODEOWNERS file by @netrome in #2179 * Weekly `cargo update` by @github-actions in #2177 * fix(db_lookup_times): rework core logic of benchmark by @rymnc in #2159 * Add verification on transaction dry_run that they don't spend more than block gas limit by @AurelienFT in #2151 * bug: fix algorithm overflow issues by @MitchTurner in #2173 * feat(gas_price_service): create runnable task for expensive background polling for da metadata by @rymnc in #2163 * Weekly `cargo update` by @github-actions in #2197 * Fix bug with gas price factor in V1 algorithm by @MitchTurner in #2201 * Applying several breaking changes to the WASM interface from backlog by @xgreenx in #2199 * chore(p2p): dnsaddr recursive resolution by @rymnc in #2204 ## New Contributors * @acerone85 made their first contribution in #2158 **Full Changelog**: v0.35.0...v0.36.0
Linked Issues
#2148
Description
Tuning and improving the V1 gas price algorithm is still and ongoing process, but this has drastically improved the accuracy of the algorithm when trying to track the costs of the DA block posting.
Some terms I'll use here:
"exec gas price": The portion of the L2 gas price that goes toward executing transations on L2
"da gas price": The portion of the L2 gas price that goes toward paying for committing block to DA
"cost": The amount in base asset to commit a block to the DA
"reward": The amount in base asset received by block producer from the "da gas price"
"profit": The difference between "reward" and "cost". Negative "profit" means it has costed more in total to commit blocks than has been rewarded in total to the block producer
"accuracy": The sum of all profit errors over the sample (either positive or negative). The optimization is trying to minimize the error for the highest "accuracy"
As you can see, however, there is room to improve:
During a relatively stable period where the posting costs barely change, the profit is still oscillating. We can dampen this by increasing the
Dvalue, but this doesn't result in a better accuracy:The max error is much higher in this case. i.e. the profit gets much higher/lower. Maybe this is okay?
We can continue to improve the algorithm and do more tuning. A followup issue to this is to use real values instead of simulated DA gas prices #2138. Perhaps that will have less noise :) Another possibility is that we choose more inaccuracy in favor of a smoother algorithm.
This PR includes some of the following changes:
Dcomponent (ignore bytes for now)Checklist
Before requesting review
After merging, notify other teams
[Add or remove entries as needed]