Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

implement bundlecaps, bundle device #4372

Closed
warner opened this issue Jan 25, 2022 · 0 comments · Fixed by #4485
Closed

implement bundlecaps, bundle device #4372

warner opened this issue Jan 25, 2022 · 0 comments · Fixed by #4485
Assignees
@warner
Labels
enhancement New feature or request SwingSet package: SwingSet
Milestone
Mainnet 1 RC0

Comments

@warner
Copy link
Member

warner commented Jan 25, 2022

What is the Problem Being Solved?

As detailed in #3269 (comment) , the goal of moving large contract bundles out of vat messages requires a "kernel-native" way to refer to code bundles. This also helps with vat upgrade, and even kernel/liveslots upgrade.

This ticket is specifically about implementing the bundle device and bundlecaps, not about how Agoric code will use them.

Description of the Design

As copied from #3269:

  • a "bundle" is an object, with moduleFormat: and some format-specific properties, where endoZipBase64 is the only one accepted
  • a "bundleID" is a string, b1- concatenated with the lowercase hex-encoded SHA512 hash of the "compartment map" file
  • a valid bundle matches that hash, contains components for everything named in the compartment map, those components match the hash named in the compartment map, and contains no spurious components
  • a "bundleCap" is a swingset device node (or more commonly an imported reference to such a device node) with a single D(bundleCap).getBundle() -> bundle method
  • controller.installBundle(allegedBundleID, bundle) -> bundleID or throws is how you install a bundle at runtime, from the outside of the kernel
    • this throws an error if the bundle is invalid or doesn't match the alleged bundleID
    • (the first version of our implementation won't perform validation)
  • a new "bundle device" will create and host bundleCap device nodes
  • D(devices.bundle).getBundleCap(bundleID) -> bundleCap turns the hash into a bundleCap
    • this will throw an error if nobody did controller.installBundle first
  • E(vatAdmin).createVat(bundleCap) -> { root, controlFacet } turns bundleCaps into dynamic vats
  • importBundle(D(bundleCap).getBundle()) -> Promise<namespace> evaluates the bundle and gives you the resulting namespace, e.g. for when ZCF loads a contract bundle

Security Considerations

Test Plan
@warner warner added enhancement New feature or request SwingSet package: SwingSet labels Jan 25, 2022
@warner warner self-assigned this Jan 25, 2022
This was referenced Jan 25, 2022
Open
Closed
Closed
@warner warner added the MN-1 label Jan 25, 2022
@warner warner mentioned this issue Feb 4, 2022
Open
warner added a commit that referenced this issue Feb 7, 2022
@warner
feat(swingset): devices.bundle, install-bundle, bundlecaps, createVat…
ed2b189 
…(bundlecap)

Add kernel support for code "bundles", specifically objects with `{
moduleFormat: "EndoZipBase64" }` whose `.EndoZipBase64` property is a large
string (base64-encoded zipfile with a compartment map and module components).
Each bundle has a "bundleID" which is the versioning prefix `b1-` followed by
the lowercase hex encoding of the SHA512 hash of the compartment map bytes.

Bundles are represented within userspace as "bundlecaps", which are device
nodes owned by a new "bundle device" (`devices.bundle`). These can be passed
in messages from one vat to another, just like Remotables. Bundlecaps are
used to create vats in lieu of passing the actual (large) code bundles around
through messages. Bundlecaps can also be asked for their code bundle in case
you need to `importBundle` one directly into userspace (e.g. when ZCF
evaluates a contract bundle).

The `config.bundles` table is now handled by installing the bundles at
`initializeSwingset` time, and populating a name->ID table for later.

The new APIs are:

* `computedBundleID = controller.validateAndInstallBundle(bundle,
allegedBundleID)` will validate the bundle against the claimed ID and add it
to the kernel tables (NOTE: validation is minimal so far, must be improved
before release)
* `kernel.installBundle(bundleID, bundle)` will install a bundle under the
given ID without validation
* `devices.bundle` provides access to bundles
  * `D(devices.bundle).getBundleCap(bundleID)` yields a bundlecap or
    `undefined` if no bundle was installed with that ID
  * `D(devices.bundle).getNamedBundleCap(name)` yields a bundlecap or
    `undefined` if config.bundles lacked a bundle with that name
* bundlescaps are device nodes
  * `D(bundlecap).getBundleID()` yields the bundleID
  * `D(bundlecap).getBundle()` yields a code bundle, for `importBundle()`
* `E(vatAdminService).createVat(bundleOrBundleCap)` creates a dynamic vat
  * eventually we'll remove the option to use a bundle, making this strictly
    `E(vatAdminService).createVat(bundlecap)`
  * `E(vatAdminService).createVatByName(name)` still works, but eventually it
    will be removed in favor of userspace doing `getNamedBundleCap` first

refs #4372
This was referenced Feb 7, 2022
Merged
Open
warner added a commit that referenced this issue Feb 7, 2022
@warner
fix(swingset): remove support for createVat(bundle)
cf45824 
This removes support for passing a full bundle to
`E(vatAdminService).createVat()`, leaving a bundlecap as the only remaining
option. Userspace is obligated to obtain a bundlecap as early as possible.

This can't land until Zoe/ertp/governance are changed to use bundlecaps, as
well as changing their unit tests to match.

Eventually I also want to remove `E(vatAdminService).createVatByName()`, and
have the code that needs it (zoe creating ZCF vats?) use
`getNamedBundleCap()` first.

refs #4372
@warner warner mentioned this issue Feb 7, 2022
Open
@Tartuffo Tartuffo removed the MN-1 label Feb 7, 2022
@mergify mergify bot closed this as completed in #4485 Feb 9, 2022
mergify bot pushed a commit that referenced this issue Feb 9, 2022
@warner
feat(swingset): devices.bundle, install-bundle, bundlecaps, createVat…
1c39ebd 
…(bundlecap)

Add kernel support for code "bundles", specifically objects with `{
moduleFormat: "EndoZipBase64" }` whose `.EndoZipBase64` property is a large
string (base64-encoded zipfile with a compartment map and module components).
Each bundle has a "bundleID" which is the versioning prefix `b1-` followed by
the lowercase hex encoding of the SHA512 hash of the compartment map bytes.

Bundles are represented within userspace as "bundlecaps", which are device
nodes owned by a new "bundle device" (`devices.bundle`). These can be passed
in messages from one vat to another, just like Remotables. Bundlecaps are
used to create vats in lieu of passing the actual (large) code bundles around
through messages. Bundlecaps can also be asked for their code bundle in case
you need to `importBundle` one directly into userspace (e.g. when ZCF
evaluates a contract bundle).

The `config.bundles` table is now handled by installing the bundles at
`initializeSwingset` time, and populating a name->ID table for later.

The new APIs are:

* `computedBundleID = controller.validateAndInstallBundle(bundle,
allegedBundleID)` will validate the bundle against the claimed ID and add it
to the kernel tables (NOTE: validation is minimal so far, must be improved
before release)
* `kernel.installBundle(bundleID, bundle)` will install a bundle under the
given ID without validation
* `devices.bundle` provides access to bundles
  * `D(devices.bundle).getBundleCap(bundleID)` yields a bundlecap or
    `undefined` if no bundle was installed with that ID
  * `D(devices.bundle).getNamedBundleCap(name)` yields a bundlecap or
    `undefined` if config.bundles lacked a bundle with that name
* bundlescaps are device nodes
  * `D(bundlecap).getBundleID()` yields the bundleID
  * `D(bundlecap).getBundle()` yields a code bundle, for `importBundle()`
* `E(vatAdminService).createVat(bundleOrBundleCap)` creates a dynamic vat
  * eventually we'll remove the option to use a bundle, making this strictly
    `E(vatAdminService).createVat(bundlecap)`
  * `E(vatAdminService).createVatByName(name)` still works, but eventually it
    will be removed in favor of userspace doing `getNamedBundleCap` first

refs #4372
closes #3269
closes #4373
warner added a commit that referenced this issue Feb 9, 2022
@warner
docs(swingset): update dynamic-vats.md to mention bundlecaps
543aecb 
refs #4372
@warner warner mentioned this issue Feb 9, 2022
Merged
warner added a commit that referenced this issue Feb 9, 2022
@warner
docs(swingset): update dynamic-vats.md docs
a74dfb1 
Mention bundlecaps, update options, update metering, update vat termination
description.

refs #4372
warner added a commit that referenced this issue Feb 9, 2022
@warner
fix(zoe): teach fakeVatAdmin to createVatByName(zcf)
6f88e2b 
Zoe needs a way to create a new ZCF vat to host each contract instance.
Inside swingset, zoe uses `vatAdminService`, and this is used from unit
tests (zoe and other packages that use zoe) when they are willing to take the
time to run a full swingset environment.

Unit tests that use zoe, but not swingset, use `tools/fakeVatAdmin.js` as a
replacement, which implements `createVat(bundle)` but not
`createVatByName(name)`. This is used to evaluate the ZCF bundle in a new
Compartment (using `importBundle` and `evalContractBundle`).

The primary export of `@agoric/zoe` is `makeZoeKit()`, which is called with
`vatAdminService` and an optional `zcfBundleName`. This function runs
`setupCreateZCFVat(vatAdminService)` to attenuate the vat-making power into
one that can only create ZCF vats. Previously, `setupCreateZCFVat` closed
over a copy of the ZCF bundle, and passed it to
`vatAdminService~.createVat(zcfBundle)`. This hides the existence of the ZCF
bundle from other packages that just want to use Zoe.

However, to remove these large code bundles from userspace messages (#4372),
we need the ZCF bundle to be installed "off to the side", by the code that
prepares the swingset environment (usually as `config.bundles.zcf=`).

This commit changes `fakeVatAdmin.js` to implement `createVatByName('zcf')`,
and to move the copy of the zcfBundle out of `makeZoeKit()` and into
`fakeVatAdmin.js` . In addition, it changes `createZCFVat.js` to provide a
default bundle name of 'zcf'. Any unit test that uses `fakeVatAdmin.js` can
continue to do so without changes, and the fake service will magically know
how to create Zoe's ZCF vat without any additional configuration.

Unit tests that use swingset, however, need to be updated to install the ZCF
bundle into the kernel, with something like:

```
import zcfBundle from `@agoric/zoe/bundles/bundle-contractFacet.js`;
...
config.bundles.zcf = { bundle: zcfBundle };
```

Note: if we used `{ sourceSpec: '@agoric/zoe/contractFacet.js' }`, then we
would not depend upon a recent `cd packages/zoe && yarn build`, and each
kernel instance would bundle its own copy. This would be slower, but perhaps
less prone to stale-bundle surprises, and might be a nicer export
commitment.

refs #4487
This was referenced Feb 13, 2022
Merged
Closed
turadg pushed a commit that referenced this issue Feb 17, 2022
@warner
fix(swingset): use consistent lowercase `bundlecap' in API/docs
872f956 
* `D(devices.bundle).getBundlecap()`, not `getBundleCap`
* `D(devices.bundle).getNamedBundlecap()`, not `getNamedBundleCap`

I want code to use `bundlecap` in variable names, rather than `bundleCap`,
and this reinforces the pattern.

refs #4372
@Tartuffo Tartuffo added this to the Mainnet 1 milestone Mar 23, 2022
@warner warner mentioned this issue Apr 5, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@warner warner

Labels
enhancement New feature or request SwingSet package: SwingSet
Projects
None yet
Milestone
Mainnet 1 RC0
Development

Successfully merging a pull request may close this issue.

feat(swingset): devices.bundle, install-bundle, bundlecaps, createVat(bundlecap) Agoric/agoric-sdk
2 participants
@warner @Tartuffo