SES: Adapt Native XS Compartment to SES Compartment #2251
Comments
29 tasks
kriskowal
changed the title
ses such that the SES Compartment wraps the XS native Compartment providing parity with the current ses Compartment interface using the XS native implementation.
kriskowal
added a commit
that referenced
this issue
Jul 15, 2024
Refs: #400 #2251 ## Description Pursuant to arriving at parity with XS, this change internally reörients the SES module loader around the concept of a “module descriptor”. This creates some clarity and removes some existing duplication of work for “alias” module descriptors, and prepares the material for a richer module descriptor schema to match those implemented by XS. ### Security Considerations No changes. ### Scaling Considerations No changes. ### Documentation Considerations No changes. ### Testing Considerations No changes. ### Compatibility Considerations No changes. ### Upgrade Considerations No changes.
kriskowal
added a commit
that referenced
this issue
Jul 16, 2024
Refs: #400 #2251 ## Description This change introduces support for XS-compatible module descriptors with the discriminating properties `source` and `namespace`, including support for loading a module from the parent compartment. ### Security Considerations None. ### Scaling Considerations None. ### Documentation Considerations This change is additive. ### Testing Considerations Covered. ### Compatibility Considerations Some usage patterns are now deprecated but continue to be supported without complaint. ### Upgrade Considerations None.
This was referenced Sep 26, 2024
kriskowal
added a commit
that referenced
this issue
Nov 14, 2024
Closes: #2250 Refs: #2251 ## Description Toward using native XS compartments and module sources, this change adds an XS-specific variant of `@endo/module-source` that patches the native `ModuleSource` in place instead of entraining Babel. This will allow an XS-specific `@endo/import-bundle` variant to load both pre-compiled sources (`pre-json-mjs`) and original ESM sources (`mjs`). This change is opt-in, provided the `xs` runtime condition. These are conditions introduced by Node.js with the `node -C development entry.js` precedent and supported by `@endo/compartment-mapper` for both `endoScript` and `endoZipBase64` bundle formats. ### Security Considerations This change increases exposure to a native ModuleSource as implemented in a memory-unsafe language. ### Scaling Considerations We expect a significant improvement in performance at both bundle time and run time upon completion of #2251 since native ModuleSource is considerably faster than a virtualization. ### Documentation Considerations Captured in NEWS.md and README.md. ### Testing Considerations This change introduces a minimal `text:xs` that runs with `xst` and is exercised by Continuous Integration. This minimal test can be expanded as we make more progress with the XS-variant of SES, which will allow the virtual Compartment to opt-in to use native ModuleSource. ### Compatibility Considerations The XS-specific shim is not a one-to-one replacement for the Babel implementation and does not support some of the optional arguments. Code that expects these hooks to be called will be dismayed. Despite these limitations, no existing program should break: the `xs` condition should not be present in any existing application. ### Upgrade Considerations This change will not come to bear on the Agoric chain until after a new version of XS becomes available with a new XS-specific lockdown preamble.
kriskowal
added a commit
that referenced
this issue
Dec 6, 2024
Closes: #2251 ## Description To take advantage of native XS compartments while retaining backward compatibility and parity with the SES shim, we introduce an `xs` specific Compartment adapter that requires two levels of opt-in. 1. The SES shim must be bundled with the `xs` package export/import condition. 2. The constructor of a Compartment must specify the `__native__` option to sacrifice the ability to use precompiled module sources (as generated by `@endo/module-source` _without_ the `xs` package export/import condition) and instead use adapted native `ModuleSource` (as generated by `@endo/module-source` _with_ the `xs` package export/import condition). This allows `@endo/import-bundle`, for example, to use the JSON serialization of a precompiled module source that is captured in a bundle and also opt-in for `__native__` treatment if the archive/bundle contains original sources. This change introduces an XS-specific shim that is an adapter for `Compartment` and `lockdown`, and also papers over parity gaps like the XS `Object.freeze` second boolean argument. The adapter creates parallel native and shim (virtual) compartment trees, where any individual Compartment can elect to use the native or shim variant for a child Compartment. We have not yet found a workable design that obviates the need for the `__native__` opt-in. Such a design would need to create an adapter from precompiled module sources to XS’s virtual module source protocol. To do that would require native module to emit notifications for the mutation of exported live bindings and also require the native Compartment evaluate method to accept an argument like the shim’s `__moduleGlobalLexicals__`. ### Security Considerations Uncountably numerous. Among them, with the `__native__` option, censorship does not occur, so dynamic import and direct eval are possible. ### Scaling Considerations The native ModuleSource makes it practical to defer module parsing to runtime, and should improve the performance of execution as well. ### Documentation Considerations The NEWS.md qualifies these changes as under "incubation". When the shape of these changes settles, the NEWS will need to reiterate the final user facing API in README.md and NEWS.md. With the `__native__` option, censorship does not occur, so dynamic import and direct eval are possible. ### Testing Considerations This change contains a token of `xst` testing that is exercised in CI with `test:xs`. This demonstrates the use of `@endo/compartment-mapper/bundle.js` to thread the `xs` package export/import condition and generate a script that can `xst` can run directly. This gives us some modest confidence that lockdown works and demonstrates the `__native__` feature but does not provide sufficient confidence of parity for the gamut of Compartment usage, both legacy and XS, for all of the accepted module descriptors and other Compartment features. This is an exercise that will begin with a subsequent change that introduces `hardened262`, a comprehensive parity checking framework for the full cross-product of [ SES on Node.js, SES on XS, and XS stand-alone ] ⨉ [ Lockdown, not Lockdown ] ⨉ [ Compartment, no Compartment ] ⨉ [ Sloppy, Strict, Module ]. ### Compatibility Considerations This change preserves all existing usage and introduces an unstable alternate version of SES for XS that requires two layers of opt-in. The use of the `xs` condition introduces an adapter for `Compartment` that may not have full parity with the underlying implementation, and requires additional testing. The `__native__` option elects to break some usage (precompiled moduels) in favor of others (dynamic import, direct eval, top-level-await). ### Upgrade Considerations In order to realize these changes on the Agoric chain will likely require a more recent version of XS and switching the bundle format for the lockdown/bootstrap script for xsnap swingset workers.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Refs: #400
Add a Compartment adapter to
sessuch that the SESCompartmentwraps the XS nativeCompartmentproviding parity with the currentsesCompartmentinterface using the XS native implementation.- SES: Introduce XS-specific Compartment interface-parity-adapter around native Compartment such that existing usage requires no changes.
- Refactor SES Compartment to implement the XS Native Compartment API more directly and adjust Compartment Mapper to use these new forms.
- Refactor Compartment Mapper such that it drives
ModuleSourceandModuleconstructors directly and subsumes some dependency analysis responsibilities fromCompartment.The text was updated successfully, but these errors were encountered: