Change parameter estimations to work in presence of caches #7681
Labels
A-params-estimator
Area: runtime params estimator
T-contract-runtime
Team: issues relevant to the contract runtime team
Estimations today disable nearcore internal caches on top of RocksDB. Specifically, trie caches and compiled code caches.
Disabling these caches overestimates the amount of IO we need, as we often access the same data multiple times in normal processing. So far this was "no problem" because it appears estimations were mostly reading data from RockDB's internal in-memory caches. But now with #7484 and #7502 removing that estimator bug, some estimation results have sky-rocketed. (see #7519)
I suggest we enable the caches from now in estimations. However, that only makes sense if we modify existing estimations.
For example the estimations for storage related host functions otherwise read all values from the cache after the first iteration. The iterations in that case are inside the estimator smart contract, so we have to fix it on that level.
Another example is code compilation. Maybe we can keep those caches disabled. Or we could compile different code in each iteration. Or even just clear the cache between iterations.
The goal here is that we can change the estimator setup to make use of caches where it makes sense. Ideally, all estimations should then be roughly what they were a few weeks ago, or there should be a clear correlation between the changed results and the fixed IO estimation bug.
The text was updated successfully, but these errors were encountered: