upgrading liveslots: controller.setLiveslotsBundleID()

[warner]

What is the Problem Being Solved?

As with #4375 to upgrade the kernel, it would be nice to be able to upgrade the version of liveslots that we use in all vats.

Upgrading liveslots, without also changing the vat bundle, could be used to add new features to liveslots, or to fix a bug that the vat has not yet provoked. It can only work if the vat is already prepared for upgrade (by recording its important state in Durable collections, and providing references to them in the #4325 "baggage", and by having a starting point which knows how to reconnect everything).

The idea is:

• we store the liveslots bundle into the bundle store, just like vat bundles (in implement bundlecaps, bundle device #4372) and the kernel bundle (upgrading the kernel itself: controller.setKernelBundleID() #4375)
• each vat's kvStore includes a key that says what bundleID we use to get liveslots
• each vat-worker supervisor is modified to accept this bundle from the kernel, over the kernel-to-worker control pipe, instead of having an import { makeLiveSlots } from '.../liveslots.js' which drags it into the supervisor bundle
  • (come to think of it, it might be interesting to manage the supervisor bundle this way too)
• the kvStore has an additional key that tells the kernel what liveslotsBundleID to use for new vats
• we add a controller API to change this bundleID to something new

So once a vat is created, it keeps using the same liveslots, even if some new version is installed in the application. Various vats can use various versions of liveslots, depending upon whatever was current when they were first created.

Later, when you want to upgrade the liveslots in a given vat, you do a vat-upgrade but leave the vat source code untouched (or, you upgrade liveslots at the same time as you upgrade the vat bundle). This requires the kind of upgrade where we delete the heap snapshot and delete the transcript, and start the vat over from a new (/old) bundle and let it recover from durable data.

I suspect that most existing vats won't really benefit from upgrading liveslots, however being able to upgrade the vat code and have it take advantage of new liveslots features would be really valuable. So I'm thinking that vatAdminFacet~.upgrade(newVatBundleID, { liveslotsVersion: 2 }) would be great. (Or, maybe vats automatically get the latest liveslots when their vat bundles are upgraded).

Description of the Design

Security Considerations

We might consider allowing new vats to be created with an arbitrary bundle to use as liveslots. Liveslots enforces ocap security within a vat, so the code within the vat must not be able to tamper with the liveslots that supports it (else it could violate ocap rules).

However, the caller who asks for a vat to be created is already effectively in control of the whole vat. If they don't care for ocap safety within their new vat, that's their perogative, and it doesn't violate the security of the rest of the system. So it probably isn't technically wrong to allow upgrade() or createVat() to let the caller control what is used as liveslots. But anyone reading the vat source code (or any contract code later loaded into that vat) would need to know whether they can expect ocap rules or not, so it might be safer overall to restrict the options to well-known versions of liveslots.

Test Plan

unit tests

[warner]

Don't give upgraders the option to use old liveslots, unless necessary for vat-code compatibility.

Think about versions on the capdata, especially w.r.t. auxdata.

[warner]

For MN-1, I want to have most of this in place, but I'm ok with not having a controller API to set change the version used for new vats. That interacts with how we upgrade the kernel/chain as a whole (do we want a reinitializeSwingset() function that the host app calls, at some consensus-defined time, during reboot, just before launching the kernel?), and I don't know what that should look like yet, and I think we can afford to figure it out later (after MN-1). But having the worker evaluate the liveslots bundle as a separate step feels like it would line up the pieces better.

[warner]

I measured some bundle sizes to get a sense of how much extra space we'll consume by separating liveslots from the supervisor bundle.

In these tables, the size value is the length of the string that contains the source code: .source for a getExport -format bundle, and .endoZipBase64 for a endoZipBase64 bundle.

On trunk (as of a few days ago):

vat/device	format	size
adminDevice	endoZipBase64	226008
adminVat	endoZipBase64	260260
comms	endoZipBase64	373868
vattp	endoZipBase64	189440
timer	endoZipBase64	253472
--	--	--
lockdown	getExport	291834
supervisor	getExport	509064

with the split:

vat/device	format	size
lockdown	getExport	291834
supervisor	getExport	213986
liveslots	endoZipBase64	679744

It's surprising that the split liveslots is larger than the unsplit supervisor, because the current supervisor bundle should include all of:

• A: the supervisor code
• B: the liveslots code
• C: their shared dependencies
• D: dependencies that are exclusive to the supervisor
• E: dependencies that are exclusive to liveslots

whereas in the split bundles, supervisor should contain A+C+D, while liveslots should contain B+C+E, and there should be no way for B+C+E to be larger than A+B+C+D+E.

I think this is explained by two factors. The base64-ness of the endoZipBase64 format expands the source code by a factor of ln2(256)/ln2(64) = 8/6 = 33%. In addition, Endo's archiver performs a more literal walk of what needs to be included, and probably does less tree-shaking than what rollup does for the getExport format.

[warner]

This will be folded into the #6596 "worker-v1" effort, because we need stability from both XS, and all of the JS code that runs on top of it (and gets recorded in the heap snapshot): SES, supervisor bundle, liveslots, user code. The kvStore ${vatID}.options will include a managerType that names worker-v1, and the requirement will be for any version of @agoric/swingset-worker-xsnap-v1 to accomodate those heap snapshots.