Skip to content

Commit

Permalink
src: support snapshot in single executable applications
Browse files Browse the repository at this point in the history
This patch adds snapshot support to single executable applications.
To build a snapshot from the main script when preparing the
blob that will be injected into the single executable application,
add `"useSnapshot": true` to the configuration passed to
`--experimental-sea-config`. For example:

```
{
    "main": "snapshot.js",
    "output": "sea-prep.blob",
    "useSnapshot": true
}
```

The main script used to build the snapshot must invoke
`v8.startupSnapshot.setDeserializeMainFunction()` to configure the
entry point. The generated startup snapshot would be part of the
preparation blob and get injected into the final executable.

When the single executable application is launched, instead of running
the `main` script from scratch, Node.js would instead deserialize the
snapshot to get to the state initialized during build-time directly.

PR-URL: nodejs#46824
Refs: nodejs/single-executable#57
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
  • Loading branch information
joyeecheung authored and pluris committed Aug 7, 2023
1 parent 0a9e788 commit 4313385
Show file tree
Hide file tree
Showing 9 changed files with 322 additions and 42 deletions.
41 changes: 40 additions & 1 deletion doc/api/single-executable-applications.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
added:
- v19.7.0
- v18.16.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/46824
description: Added support for "useSnapshot".
-->

> Stability: 1 - Experimental: This feature is being designed and will change.
Expand Down Expand Up @@ -169,14 +173,46 @@ The configuration currently reads the following top-level fields:
{
"main": "/path/to/bundled/script.js",
"output": "/path/to/write/the/generated/blob.blob",
"disableExperimentalSEAWarning": true // Default: false
"disableExperimentalSEAWarning": true, // Default: false
"useSnapshot": false // Default: false
}
```
If the paths are not absolute, Node.js will use the path relative to the
current working directory. The version of the Node.js binary used to produce
the blob must be the same as the one to which the blob will be injected.
### Startup snapshot support
The `useSnapshot` field can be used to enable startup snapshot support. In this
case the `main` script would not be when the final executable is launched.
Instead, it would be run when the single executable application preparation
blob is generated on the building machine. The generated preparation blob would
then include a snapshot capturing the states initialized by the `main` script.
The final executable with the preparation blob injected would deserialize
the snapshot at run time.
When `useSnapshot` is true, the main script must invoke the
[`v8.startupSnapshot.setDeserializeMainFunction()`][] API to configure code
that needs to be run when the final executable is launched by the users.
The typical pattern for an application to use snapshot in a single executable
application is:
1. At build time, on the building machine, the main script is run to
initialize the heap to a state that's ready to take user input. The script
should also configure a main function with
[`v8.startupSnapshot.setDeserializeMainFunction()`][]. This function will be
compiled and serialized into the snapshot, but not invoked at build time.
2. At run time, the main function will be run on top of the deserialized heap
on the user machine to process user input and generate output.

The general constraints of the startup snapshot scripts also apply to the main
script when it's used to build snapshot for the single executable application,
and the main script can use the [`v8.startupSnapshot` API][] to adapt to
these constraints. See
[documentation about startup snapshot support in Node.js][].
## Notes
### `require(id)` in the injected module is not file based
Expand Down Expand Up @@ -249,6 +285,9 @@ to help us document them.
[`process.execPath`]: process.md#processexecpath
[`require()`]: modules.md#requireid
[`require.main`]: modules.md#accessing-the-main-module
[`v8.startupSnapshot.setDeserializeMainFunction()`]: v8.md#v8startupsnapshotsetdeserializemainfunctioncallback-data
[`v8.startupSnapshot` API]: v8.md#startup-snapshot-api
[documentation about startup snapshot support in Node.js]: cli.md#--build-snapshot
[fuse]: https://www.electronjs.org/docs/latest/tutorial/fuses
[postject]: https://github.com/nodejs/postject
[signtool]: https://learn.microsoft.com/en-us/windows/win32/seccrypto/signtool
Expand Down
9 changes: 9 additions & 0 deletions lib/internal/main/mksnapshot.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ const {
anonymousMainPath,
} = internalBinding('mksnapshot');

const { isExperimentalSeaWarningNeeded } = internalBinding('sea');

const { emitExperimentalWarning } = require('internal/util');

const {
getOptionValue,
} = require('internal/options');
Expand Down Expand Up @@ -126,6 +130,7 @@ function requireForUserSnapshot(id) {
return require(normalizedId);
}


function main() {
prepareMainThreadExecution(true, false);
initializeCallbacks();
Expand Down Expand Up @@ -167,6 +172,10 @@ function main() {

const serializeMainArgs = [process, requireForUserSnapshot, minimalRunCjs];

if (isExperimentalSeaWarningNeeded()) {
emitExperimentalWarning('Single executable application');
}

if (getOptionValue('--inspect-brk')) {
internalBinding('inspector').callAndPauseOnStart(
runEmbedderEntryPoint, undefined, ...serializeMainArgs);
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/process/pre_execution.js
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ function patchProcessObject(expandArgv1) {
__proto__: null,
enumerable: true,
// Only set it to true during snapshot building.
configurable: getOptionValue('--build-snapshot'),
configurable: isBuildingSnapshot(),
value: process.argv[0],
});

Expand Down
83 changes: 60 additions & 23 deletions src/node.cc
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,17 @@ MaybeLocal<Value> StartExecution(Environment* env, StartExecutionCallback cb) {

CHECK(!env->isolate_data()->is_building_snapshot());

#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
if (sea::IsSingleExecutable()) {
sea::SeaResource sea = sea::FindSingleExecutableResource();
// The SEA preparation blob building process should already enforce this,
// this check is just here to guard against the unlikely case where
// the SEA preparation blob has been manually modified by someone.
CHECK_IMPLIES(sea.use_snapshot(),
!env->snapshot_deserialize_main().IsEmpty());
}
#endif

// TODO(joyeecheung): move these conditions into JS land and let the
// deserialize main function take precedence. For workers, we need to
// move the pre-execution part into a different file that can be
Expand Down Expand Up @@ -1198,49 +1209,66 @@ ExitCode GenerateAndWriteSnapshotData(const SnapshotData** snapshot_data_ptr,
return exit_code;
}

ExitCode LoadSnapshotDataAndRun(const SnapshotData** snapshot_data_ptr,
const InitializationResultImpl* result) {
ExitCode exit_code = result->exit_code_enum();
bool LoadSnapshotData(const SnapshotData** snapshot_data_ptr) {
// nullptr indicates there's no snapshot data.
DCHECK_NULL(*snapshot_data_ptr);

bool is_sea = false;
#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
if (sea::IsSingleExecutable()) {
is_sea = true;
sea::SeaResource sea = sea::FindSingleExecutableResource();
if (sea.use_snapshot()) {
std::unique_ptr<SnapshotData> read_data =
std::make_unique<SnapshotData>();
std::string_view snapshot = sea.main_code_or_snapshot;
if (SnapshotData::FromBlob(read_data.get(), snapshot)) {
*snapshot_data_ptr = read_data.release();
return true;
} else {
fprintf(stderr, "Invalid snapshot data in single executable binary\n");
return false;
}
}
}
#endif

// --snapshot-blob indicates that we are reading a customized snapshot.
if (!per_process::cli_options->snapshot_blob.empty()) {
// Ignore it when we are loading from SEA.
if (!is_sea && !per_process::cli_options->snapshot_blob.empty()) {
std::string filename = per_process::cli_options->snapshot_blob;
FILE* fp = fopen(filename.c_str(), "rb");
if (fp == nullptr) {
fprintf(stderr, "Cannot open %s", filename.c_str());
exit_code = ExitCode::kStartupSnapshotFailure;
return exit_code;
return false;
}
std::unique_ptr<SnapshotData> read_data = std::make_unique<SnapshotData>();
bool ok = SnapshotData::FromFile(read_data.get(), fp);
fclose(fp);
if (!ok) {
// If we fail to read the customized snapshot,
// simply exit with kStartupSnapshotFailure.
exit_code = ExitCode::kStartupSnapshotFailure;
return exit_code;
return false;
}
*snapshot_data_ptr = read_data.release();
} else if (per_process::cli_options->node_snapshot) {
// If --snapshot-blob is not specified, we are reading the embedded
// snapshot, but we will skip it if --no-node-snapshot is specified.
return true;
}

if (per_process::cli_options->node_snapshot) {
// If --snapshot-blob is not specified or if the SEA contains no snapshot,
// we are reading the embedded snapshot, but we will skip it if
// --no-node-snapshot is specified.
const node::SnapshotData* read_data =
SnapshotBuilder::GetEmbeddedSnapshotData();
if (read_data != nullptr && read_data->Check()) {
if (read_data != nullptr) {
if (!read_data->Check()) {
return false;
}
// If we fail to read the embedded snapshot, treat it as if Node.js
// was built without one.
*snapshot_data_ptr = read_data;
}
}

NodeMainInstance main_instance(*snapshot_data_ptr,
uv_default_loop(),
per_process::v8_platform.Platform(),
result->args(),
result->exec_args());
exit_code = main_instance.Run();
return exit_code;
return true;
}

static ExitCode StartInternal(int argc, char** argv) {
Expand Down Expand Up @@ -1275,7 +1303,8 @@ static ExitCode StartInternal(int argc, char** argv) {

std::string sea_config = per_process::cli_options->experimental_sea_config;
if (!sea_config.empty()) {
return sea::BuildSingleExecutableBlob(sea_config);
return sea::BuildSingleExecutableBlob(
sea_config, result->args(), result->exec_args());
}

// --build-snapshot indicates that we are in snapshot building mode.
Expand All @@ -1290,7 +1319,15 @@ static ExitCode StartInternal(int argc, char** argv) {
}

// Without --build-snapshot, we are in snapshot loading mode.
return LoadSnapshotDataAndRun(&snapshot_data, result.get());
if (!LoadSnapshotData(&snapshot_data)) {
return ExitCode::kStartupSnapshotFailure;
}
NodeMainInstance main_instance(snapshot_data,
uv_default_loop(),
per_process::v8_platform.Platform(),
result->args(),
result->exec_args());
return main_instance.Run();
}

int Start(int argc, char** argv) {
Expand Down
10 changes: 7 additions & 3 deletions src/node_main_instance.cc
Original file line number Diff line number Diff line change
Expand Up @@ -92,12 +92,16 @@ void NodeMainInstance::Run(ExitCode* exit_code, Environment* env) {
bool runs_sea_code = false;
#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
if (sea::IsSingleExecutable()) {
runs_sea_code = true;
sea::SeaResource sea = sea::FindSingleExecutableResource();
std::string_view code = sea.code;
LoadEnvironment(env, code);
if (!sea.use_snapshot()) {
runs_sea_code = true;
std::string_view code = sea.main_code_or_snapshot;
LoadEnvironment(env, code);
}
}
#endif
// Either there is already a snapshot main function from SEA, or it's not
// a SEA at all.
if (!runs_sea_code) {
LoadEnvironment(env, StartExecutionCallback{});
}
Expand Down
Loading

0 comments on commit 4313385

Please sign in to comment.