Skip to content

Commit

Permalink
sea: use JSON configuration and blob content for SEA
Browse files Browse the repository at this point in the history
PR-URL: #47125
Refs: nodejs/single-executable#58
Reviewed-By: Darshan Sen <raisinten@gmail.com>
  • Loading branch information
joyeecheung authored Apr 9, 2023
1 parent cfb654c commit 491a5c9
Show file tree
Hide file tree
Showing 16 changed files with 639 additions and 45 deletions.
14 changes: 14 additions & 0 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,18 @@ added: v16.6.0

Use this flag to disable top-level await in REPL.

### `--experimental-sea-config`

<!-- YAML
added: REPLACEME
-->

> Stability: 1 - Experimental
Use this flag to generate a blob that can be injected into the Node.js
binary to produce a [single executable application][]. See the documentation
about [this configuration][`--experimental-sea-config`] for details.

### `--experimental-shadow-realm`

<!-- YAML
Expand Down Expand Up @@ -2556,6 +2568,7 @@ done
[`"type"`]: packages.md#type
[`--cpu-prof-dir`]: #--cpu-prof-dir
[`--diagnostic-dir`]: #--diagnostic-dirdirectory
[`--experimental-sea-config`]: single-executable-applications.md#generating-single-executable-preparation-blobs
[`--experimental-wasm-modules`]: #--experimental-wasm-modules
[`--heap-prof-dir`]: #--heap-prof-dir
[`--import`]: #--importmodule
Expand Down Expand Up @@ -2594,6 +2607,7 @@ done
[scavenge garbage collector]: https://v8.dev/blog/orinoco-parallel-scavenger
[security warning]: #warning-binding-inspector-to-a-public-ipport-combination-is-insecure
[semi-space]: https://www.memorymanagement.org/glossary/s.html#semi.space
[single executable application]: single-executable-applications.md
[test reporters]: test.md#test-reporters
[timezone IDs]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
[tracking issue for user-land snapshots]: https://github.com/nodejs/node/issues/44014
Expand Down
93 changes: 65 additions & 28 deletions doc/api/single-executable-applications.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,17 @@ This feature allows the distribution of a Node.js application conveniently to a
system that does not have Node.js installed.

Node.js supports the creation of [single executable applications][] by allowing
the injection of a JavaScript file into the `node` binary. During start up, the
program checks if anything has been injected. If the script is found, it
executes its contents. Otherwise Node.js operates as it normally does.
the injection of a blob prepared by Node.js, which can contain a bundled script,
into the `node` binary. During start up, the program checks if anything has been
injected. If the blob is found, it executes the script in the blob. Otherwise
Node.js operates as it normally does.

The single executable application feature only supports running a single
embedded [CommonJS][] file.
The single executable application feature currently only supports running a
single embedded script using the [CommonJS][] module system.

A bundled JavaScript file can be turned into a single executable application
with any tool which can inject resources into the `node` binary.
Users can create a single executable application from their bundled script
with the `node` binary itself and any tool which can inject resources into the
binary.

Here are the steps for creating a single executable application using one such
tool, [postject][]:
Expand All @@ -28,12 +30,24 @@ tool, [postject][]:
$ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js
```

2. Create a copy of the `node` executable and name it according to your needs:
2. Create a configuration file building a blob that can be injected into the
single executable application (see
[Generating single executable preparation blobs][] for details):
```console
$ echo '{ "main": "hello.js", "output": "sea-prep.blob" }' > sea-config.json
```

3. Generate the blob to be injected:
```console
$ node --experimental-sea-config sea-config.json
```

4. Create a copy of the `node` executable and name it according to your needs:
```console
$ cp $(command -v node) hello
```

3. Remove the signature of the binary:
5. Remove the signature of the binary:

* On macOS:

Expand All @@ -50,35 +64,35 @@ tool, [postject][]:
$ signtool remove /s hello
```

4. Inject the JavaScript file into the copied binary by running `postject` with
6. Inject the blob into the copied binary by running `postject` with
the following options:

* `hello` - The name of the copy of the `node` executable created in step 2.
* `NODE_JS_CODE` - The name of the resource / note / section in the binary
where the contents of the JavaScript file will be stored.
* `hello.js` - The name of the JavaScript file created in step 1.
* `--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2` - The
* `NODE_SEA_BLOB` - The name of the resource / note / section in the binary
where the contents of the blob will be stored.
* `sea-prep.blob` - The name of the blob created in step 1.
* `--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2` - The
[fuse][] used by the Node.js project to detect if a file has been injected.
* `--macho-segment-name NODE_JS` (only needed on macOS) - The name of the
segment in the binary where the contents of the JavaScript file will be
* `--macho-segment-name NODE_SEA` (only needed on macOS) - The name of the
segment in the binary where the contents of the blob will be
stored.

To summarize, here is the required command for each platform:

* On systems other than macOS:
```console
$ npx postject hello NODE_JS_CODE hello.js \
--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2
$ npx postject hello NODE_SEA_BLOB sea-prep.blob \
--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
```

* On macOS:
```console
$ npx postject hello NODE_JS_CODE hello.js \
--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
--macho-segment-name NODE_JS
$ npx postject hello NODE_SEA_BLOB sea-prep.blob \
--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
--macho-segment-name NODE_SEA
```

5. Sign the binary:
7. Sign the binary:

* On macOS:

Expand All @@ -95,12 +109,33 @@ tool, [postject][]:
$ signtool sign /fd SHA256 hello
```

6. Run the binary:
8. Run the binary:
```console
$ ./hello world
Hello, world!
```

## Generating single executable preparation blobs

Single executable preparation blobs that are injected into the application can
be generated using the `--experimental-sea-config` flag of the Node.js binary
that will be used to build the single executable. It takes a path to a
configuration file in JSON format. If the path passed to it isn't absolute,
Node.js will use the path relative to the current working directory.

The configuration currently reads the following top-level fields:

```json
{
"main": "/path/to/bundled/script.js",
"output": "/path/to/write/the/generated/blob.blob"
}
```

If the paths are not absolute, Node.js will use the path relative to the
current working directory. The version of the Node.js binary used to produce
the blob must be the same as the one to which the blob will be injected.

## Notes

### `require(id)` in the injected module is not file based
Expand Down Expand Up @@ -135,15 +170,16 @@ of [`process.execPath`][].
### Single executable application creation process

A tool aiming to create a single executable Node.js application must
inject the contents of a JavaScript file into:
inject the contents of the blob prepared with `--experimental-sea-config"`
into:

* a resource named `NODE_JS_CODE` if the `node` binary is a [PE][] file
* a section named `NODE_JS_CODE` in the `NODE_JS` segment if the `node` binary
* a resource named `NODE_SEA_BLOB` if the `node` binary is a [PE][] file
* a section named `NODE_SEA_BLOB` in the `NODE_SEA` segment if the `node` binary
is a [Mach-O][] file
* a note named `NODE_JS_CODE` if the `node` binary is an [ELF][] file
* a note named `NODE_SEA_BLOB` if the `node` binary is an [ELF][] file

Search the binary for the
`NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2:0` [fuse][] string and flip the
`NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2:0` [fuse][] string and flip the
last character to `1` to indicate that a resource has been injected.

### Platform support
Expand All @@ -165,6 +201,7 @@ to help us document them.

[CommonJS]: modules.md#modules-commonjs-modules
[ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
[Generating single executable preparation blobs]: #generating-single-executable-preparation-blobs
[Mach-O]: https://en.wikipedia.org/wiki/Mach-O
[PE]: https://en.wikipedia.org/wiki/Portable_Executable
[Windows SDK]: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ To disable single executable application support, build Node.js with the
## Implementation

When built with single executable application support, the Node.js process uses
[`postject-api.h`][] to check if the `NODE_JS_CODE` section exists in the
[`postject-api.h`][] to check if the `NODE_SEA_BLOB` section exists in the
binary. If it is found, it passes the buffer to
[`single_executable_application.js`][], which executes the contents of the
embedded script.
Expand Down
2 changes: 2 additions & 0 deletions node.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@
'src/js_stream.cc',
'src/json_utils.cc',
'src/js_udp_wrap.cc',
'src/json_parser.h',
'src/json_parser.cc',
'src/module_wrap.cc',
'src/node.cc',
'src/node_api.cc',
Expand Down
80 changes: 80 additions & 0 deletions src/json_parser.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
#include "json_parser.h"
#include "node_errors.h"
#include "node_v8_platform-inl.h"
#include "util-inl.h"

namespace node {
using v8::ArrayBuffer;
using v8::Context;
using v8::Isolate;
using v8::Local;
using v8::Object;
using v8::String;
using v8::Value;

static Isolate* NewIsolate(v8::ArrayBuffer::Allocator* allocator) {
Isolate* isolate = Isolate::Allocate();
CHECK_NOT_NULL(isolate);
per_process::v8_platform.Platform()->RegisterIsolate(isolate,
uv_default_loop());
Isolate::CreateParams params;
params.array_buffer_allocator = allocator;
Isolate::Initialize(isolate, params);
return isolate;
}

void JSONParser::FreeIsolate(Isolate* isolate) {
per_process::v8_platform.Platform()->UnregisterIsolate(isolate);
isolate->Dispose();
}

JSONParser::JSONParser()
: allocator_(ArrayBuffer::Allocator::NewDefaultAllocator()),
isolate_(NewIsolate(allocator_.get())),
handle_scope_(isolate_.get()),
context_(isolate_.get(), Context::New(isolate_.get())),
context_scope_(context_.Get(isolate_.get())) {}

bool JSONParser::Parse(const std::string& content) {
DCHECK(!parsed_);

Isolate* isolate = isolate_.get();
Local<Context> context = context_.Get(isolate);

// It's not a real script, so don't print the source line.
errors::PrinterTryCatch bootstrapCatch(
isolate, errors::PrinterTryCatch::kDontPrintSourceLine);
Local<Value> json_string_value;
Local<Value> result_value;
if (!ToV8Value(context, content).ToLocal(&json_string_value) ||
!json_string_value->IsString() ||
!v8::JSON::Parse(context, json_string_value.As<String>())
.ToLocal(&result_value) ||
!result_value->IsObject()) {
return false;
}
content_.Reset(isolate, result_value.As<Object>());
parsed_ = true;
return true;
}

std::optional<std::string> JSONParser::GetTopLevelField(
const std::string& field) {
Isolate* isolate = isolate_.get();
Local<Context> context = context_.Get(isolate);
Local<Object> content_object = content_.Get(isolate);
Local<Value> value;
// It's not a real script, so don't print the source line.
errors::PrinterTryCatch bootstrapCatch(
isolate, errors::PrinterTryCatch::kDontPrintSourceLine);
if (!content_object
->Get(context, OneByteString(isolate, field.c_str(), field.length()))
.ToLocal(&value) ||
!value->IsString()) {
return {};
}
Utf8Value utf8_value(isolate, value);
return utf8_value.ToString();
}

} // namespace node
39 changes: 39 additions & 0 deletions src/json_parser.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#ifndef SRC_JSON_PARSER_H_
#define SRC_JSON_PARSER_H_

#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS

#include <memory>
#include <optional>
#include <string>
#include "util.h"
#include "v8.h"

namespace node {
// This is intended to be used to get some top-level fields out of a JSON
// without having to spin up a full Node.js environment that unnecessarily
// complicates things.
class JSONParser {
public:
JSONParser();
~JSONParser() {}
bool Parse(const std::string& content);
std::optional<std::string> GetTopLevelField(const std::string& field);

private:
// We might want a lighter-weight JSON parser for this use case. But for now
// using V8 is good enough.
static void FreeIsolate(v8::Isolate* isolate);
std::unique_ptr<v8::ArrayBuffer::Allocator> allocator_;
DeleteFnPtr<v8::Isolate, FreeIsolate> isolate_;
v8::HandleScope handle_scope_;
v8::Global<v8::Context> context_;
v8::Context::Scope context_scope_;
v8::Global<v8::Object> content_;
bool parsed_ = false;
};
} // namespace node

#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS

#endif // SRC_JSON_PARSER_H_
5 changes: 5 additions & 0 deletions src/node.cc
Original file line number Diff line number Diff line change
Expand Up @@ -1239,6 +1239,11 @@ static ExitCode StartInternal(int argc, char** argv) {

uv_loop_configure(uv_default_loop(), UV_METRICS_IDLE_TIME);

std::string sea_config = per_process::cli_options->experimental_sea_config;
if (!sea_config.empty()) {
return sea::BuildSingleExecutableBlob(sea_config);
}

// --build-snapshot indicates that we are in snapshot building mode.
if (per_process::cli_options->per_isolate->build_snapshot) {
if (result->args().size() < 2) {
Expand Down
28 changes: 23 additions & 5 deletions src/node_errors.cc
Original file line number Diff line number Diff line change
Expand Up @@ -246,14 +246,19 @@ void PrintStackTrace(Isolate* isolate, Local<StackTrace> stack) {
std::string FormatCaughtException(Isolate* isolate,
Local<Context> context,
Local<Value> err,
Local<Message> message) {
Local<Message> message,
bool add_source_line = true) {
std::string result;
node::Utf8Value reason(isolate,
err->ToDetailString(context)
.FromMaybe(Local<String>()));
bool added_exception_line = false;
std::string source =
GetErrorSource(isolate, context, message, &added_exception_line);
std::string result = source + '\n' + reason.ToString() + '\n';
if (add_source_line) {
bool added_exception_line = false;
std::string source =
GetErrorSource(isolate, context, message, &added_exception_line);
result = source + '\n';
}
result += reason.ToString() + '\n';

Local<v8::StackTrace> stack = message->GetStackTrace();
if (!stack.IsEmpty()) result += FormatStackTrace(isolate, stack);
Expand Down Expand Up @@ -1209,6 +1214,19 @@ void TriggerUncaughtException(Isolate* isolate, const v8::TryCatch& try_catch) {
false /* from_promise */);
}

PrinterTryCatch::~PrinterTryCatch() {
if (!HasCaught()) {
return;
}
std::string str =
FormatCaughtException(isolate_,
isolate_->GetCurrentContext(),
Exception(),
Message(),
print_source_line_ == kPrintSourceLine);
PrintToStderrAndFlush(str);
}

} // namespace errors

} // namespace node
Expand Down
Loading

0 comments on commit 491a5c9

Please sign in to comment.