Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better represent heap cost in run policy #7373

Open
mhofman opened this issue Apr 10, 2023 · 0 comments
Open

Better represent heap cost in run policy #7373

mhofman opened this issue Apr 10, 2023 · 0 comments
Labels
cosmic-swingset package: cosmic-swingset enhancement New feature or request SwingSet package: SwingSet xsnap the XS execution tool

Comments

@mhofman
Copy link
Member

mhofman commented Apr 10, 2023

What is the Problem Being Solved?

The run policy is a way to limit the work done in a block to an approximate amount of wall clock time. Since we cannot deterministically use time elapsed for a chain node, we instead use proxy measures for the work performed. This currently roughly takes the shape of a computron count per completed delivery, which is a proxy for the program's computational complexity.

This however does not represent the impact that the program's heap usage has. In particular:

  • GC has to traverse the whole heap every time it triggers, including when forced like for bringOutYourDead
  • Heap snapshots have to serialize, hash and compresse the whole heap

While we attempt to hide the effects of GC (see #1872 and #3830), we know we are currently sensitive to organic gc (see #6784) at the syscall level. We also know we will never be insensitive to minor engine difference during consensus execution, as that would often result in computrons/metering differences.

Description of the Design

For each delivery, report:

  • the occurrence of gc, possibly with some heap size information
  • the number of slot and chunk allocations

Inform the run policy with:

Add bean cost metrics for the above new policy inputs.

Note that we are already reporting computron usage for the bringOutYourDead delivery.

Security Considerations

We need to ensure that the gc behavior is deterministic during consensus execution. In practice we should be, especially once forced reload from snapshot (#6943) is implemented. To avoid downstream divergences due to the run policy making different decisions, the information provided to the run policy should be included in consensus data like proposed in #6770.

During replay, the run policy is not consulted, so gc determinism is relaxed, and we should still be able to perform minor XS version upgrades that change allocation behavior.

Scaling Considerations

None I can think of

Test Plan

TBD

@mhofman mhofman added enhancement New feature or request SwingSet package: SwingSet cosmic-swingset package: cosmic-swingset xsnap the XS execution tool labels Apr 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cosmic-swingset package: cosmic-swingset enhancement New feature or request SwingSet package: SwingSet xsnap the XS execution tool
Projects
None yet
Development

No branches or pull requests

1 participant