Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob #38905

Closed
wants to merge 6 commits into from
Closed

bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob #38905

wants to merge 6 commits into from

Commits on Aug 2, 2022

  1. net: create diagnostics channels lazily

    @joyeecheung
    joyeecheung committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    d6446e7 View commit details
    Browse the repository at this point in the history

  2. bootstrap: implement --snapshot-blob and --build-snapshot 

    This patch introduces `--build-snapshot` and `--snapshot-blob` options
for creating and using user land snapshots.

For the initial iteration, user land CJS modules and ESM are not yet
supported in the snapshot, so only one single file can be snapshotted
(users can bundle their applications into a single script with their
bundler of choice to build a snapshot though).

A subset of builtins should already work, and support for more builtins
are being added. This PR includes tests checking that the TypeScript
compiler and the marked markdown renderer (and the builtins they use)
can be snapshotted and deserialized.

To generate a snapshot using `snapshot.js` as entry point and write the
snapshot blob to `snapshot.blob`:

```
$ echo "globalThis.foo = 'I am from the snapshot'" > snapshot.js
$ node --snapshot-blob snapshot.blob --build-snapshot snapshot.js
```

To restore application state from `snapshot.blob`, with `index.js` as
the entry point script for the deserialized application:

```
$ echo "console.log(globalThis.foo)" > index.js
$ node --snapshot-blob snapshot.blob index.js
I am from the snapshot
```

Users can also use the `v8.startupSnapshot` API to specify an entry
point at snapshot building time, thus avoiding the need of an additional
entry script at deserialization time:

```
$ echo "require('v8').startupSnapshot.setDeserializeMainFunction(() => console.log('I am from the snapshot'))" > snapshot.js
$ node --snapshot-blob snapshot.blob --build-snapshot snapshot.js
$ node --snapshot-blob snapshot.blob
I am from the snapshot
```

Note that this patch only adds functionality to the `node` executable
for building run-time user-land snapshots, the generated snapshot is
stored into a separate file on disk. Building a single binary with both
Node.js and an embedded snapshot has already been possible with the
`--node-snapshot-main` option to the `configure` script if the user
compiles Node.js from source. It would be a different task to enable the
`node` executable to produce a single binary that contains both Node.js
and an embedded snapshot without building Node.js from source, which
should be layered on top of the SEA (Single Executable Apps) initiative.

Known limitations/bugs that are being fixed in the upstream:

- V8 hits a DCHECK when deserializing certain mutated globals, e.g.
  `Error.stackTraceLimit` (it should work fine in the release build,
  however): https://chromium-review.googlesource.com/c/v8/v8/+/3319481
- Layout of V8's read-only heap can be inconsistent after
  deserialization, resulting in memory corruption:
  https://bugs.chromium.org/p/v8/issues/detail?id=12921

PR-URL: nodejs#38905
Refs: nodejs#35711
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    @joyeecheung
    joyeecheung committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    6b3831f View commit details
    Browse the repository at this point in the history

  3. bootstrap: clean up warning setup during serialization 

    PR-URL: nodejs#38905
Refs: nodejs#35711
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    @joyeecheung
    joyeecheung committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    2ab92c7 View commit details
    Browse the repository at this point in the history

  4. test: add UMD module test with marked 

    PR-URL: nodejs#38905
Refs: nodejs#35711
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    @joyeecheung
    joyeecheung committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    a783fdc View commit details
    Browse the repository at this point in the history

  5. test: test snapshotting TypeScript compiler 

    PR-URL: nodejs#38905
Refs: nodejs#35711
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    @joyeecheung
    joyeecheung committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    7bf1941 View commit details
    Browse the repository at this point in the history

  6. fixup! bootstrap: implement --snapshot-blob and --build-snapshot

    @joyeecheung
    joyeecheung committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    23d964b View commit details
    Browse the repository at this point in the history