meter overrun (was: just a transcript we'd like to make easier to debug) #4219
Comments
|
The underlying bug turned out to be that we were right at the edge of exhausting the contract vat's initial meter. @erights 's changes increased the meter consumption slightly, but just enough to cross the limit and exhaust the vat's meter before zoe had a chance to refill it. The initial batch of deliveries to a new ZCF vat contains a large
When the new ZCF vat starts, the first delivery is an The sequence of operations is:
However, @erights 's additions increase the computron usage by just enough to cross over the 50M threshold. The initial
So the immediate fix is to change The medium fix is that we need some sort of CI to tell us how close we are to exhausting the initial meter. We should have learned months ago that the ZCF
The longer-term fix is some deep thinking about the purpose of metering, especially during contract startup, and how we figure out what a "reasonable" amount it. We don't want the initial meter capacity to be large enough to allow denial-of-service during contract startup, however it's not at all surprising that startup takes more CPU time than later messages, especially the initial large |
|
Labeled as |
warner
changed the title
|
Currently we do not enforce a meter on any vats: static vats are not metered, and Zoe no longer provides a Meter object on the dynamic vat it creates for each contract. That's a quick-and-dirty way to avoid the meter overrun observed in this ticket. Specifically, Zoe does not assign a MeterID to any of the dynamic vats it creates for contracts, so the vat-loader creates That means swingset currently only provides one level to control vat metering. If I'm hearing folks say that this isn't good: we should still be able to catch runaway vats, even if we're not yet trying to charge for usage (or limit cumulative usage). Currently, if you want a per-delivery limit but don't want to be pestered with refills, you can use But that returns us to the question raised above: what should that per-delivery limit be? And what should happen if it is exceeded? I'm not convinced that applying a per-delivery limit would help us for MN-1, where we don't have a lot of new contracts. In this timeframe, all contracts are a critical part of the economy, so I think we'd rather have the entire kernel halt than terminate a single vat (since vat termination deletes all of its state, which is a lot harder to recover from). One interesting option would be to halt swingset, but leave the cosmos side running so it could manage a governance vote to decide on a fix (#4516), but we'd need to implement that (and be confident that it wouldn't enable more problems or attacks) before it made sense to turn on vat-threatening delivery limits. I've opened #5295 to track the task of automated measurements of contract CPU usage and bundle size, which would inform our limit decisions. I've opened #5296 to track the question of what we should do if an MN-1 contract vat exceeds the per-delivery limit. I'm going to close this one now: we've identified the problem and have two more-specific tickets to manage the new questions. |
The text was updated successfully, but these errors were encountered: