-
Notifications
You must be signed in to change notification settings - Fork 72
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(ses): Add XS variant of shim (#2471)
Closes: #2251 ## Description To take advantage of native XS compartments while retaining backward compatibility and parity with the SES shim, we introduce an `xs` specific Compartment adapter that requires two levels of opt-in. 1. The SES shim must be bundled with the `xs` package export/import condition. 2. The constructor of a Compartment must specify the `__native__` option to sacrifice the ability to use precompiled module sources (as generated by `@endo/module-source` _without_ the `xs` package export/import condition) and instead use adapted native `ModuleSource` (as generated by `@endo/module-source` _with_ the `xs` package export/import condition). This allows `@endo/import-bundle`, for example, to use the JSON serialization of a precompiled module source that is captured in a bundle and also opt-in for `__native__` treatment if the archive/bundle contains original sources. This change introduces an XS-specific shim that is an adapter for `Compartment` and `lockdown`, and also papers over parity gaps like the XS `Object.freeze` second boolean argument. The adapter creates parallel native and shim (virtual) compartment trees, where any individual Compartment can elect to use the native or shim variant for a child Compartment. We have not yet found a workable design that obviates the need for the `__native__` opt-in. Such a design would need to create an adapter from precompiled module sources to XS’s virtual module source protocol. To do that would require native module to emit notifications for the mutation of exported live bindings and also require the native Compartment evaluate method to accept an argument like the shim’s `__moduleGlobalLexicals__`. ### Security Considerations Uncountably numerous. Among them, with the `__native__` option, censorship does not occur, so dynamic import and direct eval are possible. ### Scaling Considerations The native ModuleSource makes it practical to defer module parsing to runtime, and should improve the performance of execution as well. ### Documentation Considerations The NEWS.md qualifies these changes as under "incubation". When the shape of these changes settles, the NEWS will need to reiterate the final user facing API in README.md and NEWS.md. With the `__native__` option, censorship does not occur, so dynamic import and direct eval are possible. ### Testing Considerations This change contains a token of `xst` testing that is exercised in CI with `test:xs`. This demonstrates the use of `@endo/compartment-mapper/bundle.js` to thread the `xs` package export/import condition and generate a script that can `xst` can run directly. This gives us some modest confidence that lockdown works and demonstrates the `__native__` feature but does not provide sufficient confidence of parity for the gamut of Compartment usage, both legacy and XS, for all of the accepted module descriptors and other Compartment features. This is an exercise that will begin with a subsequent change that introduces `hardened262`, a comprehensive parity checking framework for the full cross-product of [ SES on Node.js, SES on XS, and XS stand-alone ] ⨉ [ Lockdown, not Lockdown ] ⨉ [ Compartment, no Compartment ] ⨉ [ Sloppy, Strict, Module ]. ### Compatibility Considerations This change preserves all existing usage and introduces an unstable alternate version of SES for XS that requires two layers of opt-in. The use of the `xs` condition introduces an adapter for `Compartment` that may not have full parity with the underlying implementation, and requires additional testing. The `__native__` option elects to break some usage (precompiled moduels) in favor of others (dynamic import, direct eval, top-level-await). ### Upgrade Considerations In order to realize these changes on the Agoric chain will likely require a more recent version of XS and switching the bundle format for the lockdown/bootstrap script for xsnap swingset workers.
- Loading branch information
Showing
30 changed files
with
1,025 additions
and
32 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| export { defaultParserForLanguage } from './src/import-archive-all-parsers.js'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
29 changes: 29 additions & 0 deletions
29
packages/compartment-mapper/src/import-archive-all-parsers.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| /* Provides a set of default language behaviors (parsers) suitable for | ||
| * evaluating archives (zip files with a `compartment-map.json` and a file for | ||
| * each module) with pre-compiled _or_ original ESM and CommonJS. | ||
| * | ||
| * This module does not entrain a dependency on Babel on XS, but does on other | ||
| * platforms like Node.js. | ||
| */ | ||
| /** @import {ParserForLanguage} from './types.js' */ | ||
|
|
||
| import parserPreCjs from './parse-pre-cjs.js'; | ||
| import parserJson from './parse-json.js'; | ||
| import parserText from './parse-text.js'; | ||
| import parserBytes from './parse-bytes.js'; | ||
| import parserPreMjs from './parse-pre-mjs.js'; | ||
| import parserMjs from './parse-mjs.js'; | ||
| import parserCjs from './parse-cjs.js'; | ||
|
|
||
| /** @satisfies {Readonly<ParserForLanguage>} */ | ||
| export const defaultParserForLanguage = Object.freeze( | ||
| /** @type {const} */ ({ | ||
| 'pre-cjs-json': parserPreCjs, | ||
| 'pre-mjs-json': parserPreMjs, | ||
| cjs: parserCjs, | ||
| mjs: parserMjs, | ||
| json: parserJson, | ||
| text: parserText, | ||
| bytes: parserBytes, | ||
| }), | ||
| ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| tmp |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| /* eslint-env node */ | ||
| /* glimport/no-extraneous-dependenciesobal process */ | ||
| import '../index.js'; | ||
| import { promises as fs } from 'fs'; | ||
| // Lerna does not like dependency cycles. | ||
| // With an explicit devDependency from module-source to compartment-mapper, | ||
| // the build script stalls before running every package's build script. | ||
| // yarn lerna run build | ||
| // Omitting the dependency from package.json solves the problem and works | ||
| // by dint of shared workspace node_modules. | ||
| // eslint-disable-next-line import/no-extraneous-dependencies | ||
| import { makeBundle } from '@endo/compartment-mapper/bundle.js'; | ||
| // eslint-disable-next-line import/no-extraneous-dependencies | ||
| import { ModuleSource } from '@endo/module-source'; | ||
| import { fileURLToPath } from 'url'; | ||
|
|
||
| const read = async location => { | ||
| const path = fileURLToPath(location); | ||
| return fs.readFile(path); | ||
| }; | ||
| const write = async (location, content) => { | ||
| const path = fileURLToPath(location); | ||
| await fs.writeFile(path, content); | ||
| }; | ||
|
|
||
| const main = async () => { | ||
| await fs.mkdir(fileURLToPath(new URL('../tmp', import.meta.url)), { | ||
| recursive: true, | ||
| }); | ||
|
|
||
| const meaningText = await fs.readFile( | ||
| fileURLToPath(new URL('../test/_meaning.js', import.meta.url)), | ||
| 'utf8', | ||
| ); | ||
| const meaningModuleSource = new ModuleSource(meaningText); | ||
|
|
||
| await fs.writeFile( | ||
| fileURLToPath(new URL('../tmp/_meaning.pre-mjs.json', import.meta.url)), | ||
| JSON.stringify(meaningModuleSource), | ||
| ); | ||
|
|
||
| const xsPrelude = await makeBundle( | ||
| read, | ||
| new URL('../test/_xs.js', import.meta.url).href, | ||
| { | ||
| tags: new Set(['xs']), | ||
| }, | ||
| ); | ||
|
|
||
| await write(new URL('../tmp/test-xs.js', import.meta.url).href, xsPrelude); | ||
| }; | ||
|
|
||
| main().catch(err => { | ||
| console.error('Error running main:', err); | ||
| process.exitCode = 1; | ||
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| /** | ||
| * @module In the spirit of ../src/commons.js, this module captures native | ||
| * functions specific to the XS engine during initialization, so vetted shims | ||
| * are free to modify any intrinsic without risking the integrity of SES. | ||
| */ | ||
|
|
||
| /// <reference types="ses"/> | ||
|
|
||
| import { | ||
| getOwnPropertyDescriptor, | ||
| globalThis, | ||
| uncurryThis, | ||
| } from '../src/commons.js'; | ||
|
|
||
| /** @type {typeof Compartment} */ | ||
| export const NativeStartCompartment = /** @type {any} */ (globalThis) | ||
| .Compartment; | ||
| export const nativeCompartmentPrototype = NativeStartCompartment.prototype; | ||
| export const nativeImport = uncurryThis(nativeCompartmentPrototype.import); | ||
| export const nativeImportNow = uncurryThis( | ||
| nativeCompartmentPrototype.importNow, | ||
| ); | ||
| /** @type {(compartment: any, source: string) => unknown} */ | ||
| export const nativeEvaluate = uncurryThis(nativeCompartmentPrototype.evaluate); | ||
| /** @type {(compartment: typeof Compartment) => typeof globalThis} */ | ||
| export const nativeGetGlobalThis = uncurryThis( | ||
| // @ts-expect-error we know it is there on XS | ||
| getOwnPropertyDescriptor(nativeCompartmentPrototype, 'globalThis').get, | ||
| ); |
Oops, something went wrong.