Skip to content

Commit

Permalink
sea: support embedding assets
Browse files Browse the repository at this point in the history
With this patch:

Users can now include assets by adding a key-path dictionary
to the configuration as the `assets` field. At build time, Node.js
would read the assets from the specified paths and bundle them into
the preparation blob. In the generated executable, users can retrieve
the assets using the `sea.getAsset()` and `sea.getAssetAsBlob()` API.

```json
{
  "main": "/path/to/bundled/script.js",
  "output": "/path/to/write/the/generated/blob.blob",
  "assets": {
    "a.jpg": "/path/to/a.jpg",
    "b.txt": "/path/to/b.txt"
  }
}
```

The single-executable application can access the assets as follows:

```cjs
const { getAsset } = require('node:sea');
// Returns a copy of the data in an ArrayBuffer
const image = getAsset('a.jpg');
// Returns a string decoded from the asset as UTF8.
const text = getAsset('b.txt', 'utf8');
// Returns a Blob containing the asset.
const blob = getAssetAsBlob('a.jpg');
```

Drive-by: update the  documentation to include a section dedicated
to the injected main script and refer to it as "injected main
script" instead of "injected module" because it's a script, not
a module.

PR-URL: #50960
Refs: nodejs/single-executable#68
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
  • Loading branch information
joyeecheung authored and targos committed Feb 15, 2024
1 parent 47186fb commit cea5295
Show file tree
Hide file tree
Showing 12 changed files with 578 additions and 12 deletions.
22 changes: 22 additions & 0 deletions doc/api/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -2364,6 +2364,17 @@ error indicates that the idle loop has failed to stop.
An attempt was made to use operations that can only be used when building
V8 startup snapshot even though Node.js isn't building one.

<a id="ERR_NOT_IN_SINGLE_EXECUTABLE_APPLICATION"></a>

### `ERR_NOT_IN_SINGLE_EXECUTABLE_APPLICATION`

<!-- YAML
added: REPLACEME
-->

The operation cannot be performed when it's not in a single-executable
application.

<a id="ERR_NOT_SUPPORTED_IN_SNAPSHOT"></a>

### `ERR_NOT_SUPPORTED_IN_SNAPSHOT`
Expand Down Expand Up @@ -2510,6 +2521,17 @@ The [`server.close()`][] method was called when a `net.Server` was not
running. This applies to all instances of `net.Server`, including HTTP, HTTPS,
and HTTP/2 `Server` instances.

<a id="ERR_SINGLE_EXECUTABLE_APPLICATION_ASSET_NOT_FOUND"></a>

### `ERR_SINGLE_EXECUTABLE_APPLICATION_ASSET_NOT_FOUND`

<!-- YAML
added: REPLACEME
-->

A key was passed to single executable application APIs to identify an asset,
but no match could be found.

<a id="ERR_SOCKET_ALREADY_BOUND"></a>

### `ERR_SOCKET_ALREADY_BOUND`
Expand Down
110 changes: 100 additions & 10 deletions doc/api/single-executable-applications.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,14 +178,52 @@ The configuration currently reads the following top-level fields:
"output": "/path/to/write/the/generated/blob.blob",
"disableExperimentalSEAWarning": true, // Default: false
"useSnapshot": false, // Default: false
"useCodeCache": true // Default: false
"useCodeCache": true, // Default: false
"assets": { // Optional
"a.dat": "/path/to/a.dat",
"b.txt": "/path/to/b.txt"
}
}
```
If the paths are not absolute, Node.js will use the path relative to the
current working directory. The version of the Node.js binary used to produce
the blob must be the same as the one to which the blob will be injected.
### Assets
Users can include assets by adding a key-path dictionary to the configuration
as the `assets` field. At build time, Node.js would read the assets from the
specified paths and bundle them into the preparation blob. In the generated
executable, users can retrieve the assets using the [`sea.getAsset()`][] and
[`sea.getAssetAsBlob()`][] APIs.
```json
{
"main": "/path/to/bundled/script.js",
"output": "/path/to/write/the/generated/blob.blob",
"assets": {
"a.jpg": "/path/to/a.jpg",
"b.txt": "/path/to/b.txt"
}
}
```
The single-executable application can access the assets as follows:
```cjs
const { getAsset } = require('node:sea');
// Returns a copy of the data in an ArrayBuffer.
const image = getAsset('a.jpg');
// Returns a string decoded from the asset as UTF8.
const text = getAsset('b.txt', 'utf8');
// Returns a Blob containing the asset.
const blob = getAssetAsBlob('a.jpg');
```
See documentation of the [`sea.getAsset()`][] and [`sea.getAssetAsBlob()`][]
APIs for more information.
### Startup snapshot support
The `useSnapshot` field can be used to enable startup snapshot support. In this
Expand Down Expand Up @@ -229,11 +267,58 @@ execute the script, which would improve the startup performance.
**Note:** `import()` does not work when `useCodeCache` is `true`.
## Notes
## In the injected main script
### `require(id)` in the injected module is not file based
### Single-executable application API
`require()` in the injected module is not the same as the [`require()`][]
The `node:sea` builtin allows interaction with the single-executable application
from the JavaScript main script embedded into the executable.
#### `sea.isSea()`
<!-- YAML
added: REPLACEME
-->
* Returns: {boolean} Whether this script is running inside a single-executable
application.
### `sea.getAsset(key[, encoding])`
<!-- YAML
added: REPLACEME
-->
This method can be used to retrieve the assets configured to be bundled into the
single-executable application at build time.
An error is thrown when no matching asset can be found.
* `key` {string} the key for the asset in the dictionary specified by the
`assets` field in the single-executable application configuration.
* `encoding` {string} If specified, the asset will be decoded as
a string. Any encoding supported by the `TextDecoder` is accepted.
If unspecified, an `ArrayBuffer` containing a copy of the asset would be
returned instead.
* Returns: {string|ArrayBuffer}
### `sea.getAssetAsBlob(key[, options])`
<!-- YAML
added: REPLACEME
-->
Similar to [`sea.getAsset()`][], but returns the result in a [`Blob`][].
An error is thrown when no matching asset can be found.
* `key` {string} the key for the asset in the dictionary specified by the
`assets` field in the single-executable application configuration.
* `options` {Object}
* `type` {string} An optional mime type for the blob.
* Returns: {Blob}
### `require(id)` in the injected main script is not file based
`require()` in the injected main script is not the same as the [`require()`][]
available to modules that are not injected. It also does not have any of the
properties that non-injected [`require()`][] has except [`require.main`][]. It
can only be used to load built-in modules. Attempting to load a module that can
Expand All @@ -250,15 +335,17 @@ const { createRequire } = require('node:module');
require = createRequire(__filename);
```
### `__filename` and `module.filename` in the injected module
### `__filename` and `module.filename` in the injected main script
The values of `__filename` and `module.filename` in the injected module are
equal to [`process.execPath`][].
The values of `__filename` and `module.filename` in the injected main script
are equal to [`process.execPath`][].
### `__dirname` in the injected module
### `__dirname` in the injected main script
The value of `__dirname` in the injected module is equal to the directory name
of [`process.execPath`][].
The value of `__dirname` in the injected main script is equal to the directory
name of [`process.execPath`][].
## Notes
### Single executable application creation process
Expand Down Expand Up @@ -298,9 +385,12 @@ to help us document them.
[Mach-O]: https://en.wikipedia.org/wiki/Mach-O
[PE]: https://en.wikipedia.org/wiki/Portable_Executable
[Windows SDK]: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
[`Blob`]: https://developer.mozilla.org/en-US/docs/Web/API/Blob
[`process.execPath`]: process.md#processexecpath
[`require()`]: modules.md#requireid
[`require.main`]: modules.md#accessing-the-main-module
[`sea.getAsset()`]: #seagetassetkey-encoding
[`sea.getAssetAsBlob()`]: #seagetassetasblobkey-options
[`v8.startupSnapshot.setDeserializeMainFunction()`]: v8.md#v8startupsnapshotsetdeserializemainfunctioncallback-data
[`v8.startupSnapshot` API]: v8.md#startup-snapshot-api
[documentation about startup snapshot support in Node.js]: cli.md#--build-snapshot
Expand Down
1 change: 1 addition & 0 deletions lib/internal/bootstrap/realm.js
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ const legacyWrapperList = new SafeSet([
// beginning with "internal/".
// Modules that can only be imported via the node: scheme.
const schemelessBlockList = new SafeSet([
'sea',
'test',
'test/reporters',
]);
Expand Down
4 changes: 4 additions & 0 deletions lib/internal/errors.js
Original file line number Diff line number Diff line change
Expand Up @@ -1632,6 +1632,8 @@ E('ERR_NETWORK_IMPORT_DISALLOWED',
"import of '%s' by %s is not supported: %s", Error);
E('ERR_NOT_BUILDING_SNAPSHOT',
'Operation cannot be invoked when not building startup snapshot', Error);
E('ERR_NOT_IN_SINGLE_EXECUTABLE_APPLICATION',
'Operation cannot be invoked when not in a single-executable application', Error);
E('ERR_NOT_SUPPORTED_IN_SNAPSHOT', '%s is not supported in startup snapshot', Error);
E('ERR_NO_CRYPTO',
'Node.js is not compiled with OpenSSL crypto support', Error);
Expand Down Expand Up @@ -1715,6 +1717,8 @@ E('ERR_SCRIPT_EXECUTION_INTERRUPTED',
E('ERR_SERVER_ALREADY_LISTEN',
'Listen method has been called more than once without closing.', Error);
E('ERR_SERVER_NOT_RUNNING', 'Server is not running.', Error);
E('ERR_SINGLE_EXECUTABLE_APPLICATION_ASSET_NOT_FOUND',
'Cannot find asset %s for the single executable application', Error);
E('ERR_SOCKET_ALREADY_BOUND', 'Socket is already bound', Error);
E('ERR_SOCKET_BAD_BUFFER_SIZE',
'Buffer size must be a positive integer', TypeError);
Expand Down
75 changes: 75 additions & 0 deletions lib/sea.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
'use strict';
const {
ArrayBufferPrototypeSlice,
} = primordials;

const { isSea, getAsset: getAssetInternal } = internalBinding('sea');
const { TextDecoder } = require('internal/encoding');
const { validateString } = require('internal/validators');
const {
ERR_NOT_IN_SINGLE_EXECUTABLE_APPLICATION,
ERR_SINGLE_EXECUTABLE_APPLICATION_ASSET_NOT_FOUND,
} = require('internal/errors').codes;
const { Blob } = require('internal/blob');

/**
* Look for the asset in the injected SEA blob using the key. If
* no matching asset is found an error is thrown. The returned
* ArrayBuffer should not be mutated or otherwise the process
* can crash due to access violation.
* @param {string} key
* @returns {ArrayBuffer}
*/
function getRawAsset(key) {
validateString(key, 'key');

if (!isSea()) {
throw new ERR_NOT_IN_SINGLE_EXECUTABLE_APPLICATION();
}

const asset = getAssetInternal(key);
if (asset === undefined) {
throw new ERR_SINGLE_EXECUTABLE_APPLICATION_ASSET_NOT_FOUND(key);
}
return asset;
}

/**
* Look for the asset in the injected SEA blob using the key. If the
* encoding is specified, return a string decoded from it by TextDecoder,
* otherwise return *a copy* of the original data in an ArrayBuffer. If
* no matching asset is found an error is thrown.
* @param {string} key
* @param {string|undefined} encoding
* @returns {string|ArrayBuffer}
*/
function getAsset(key, encoding) {
if (encoding !== undefined) {
validateString(encoding, 'encoding');
}
const asset = getRawAsset(key);
if (encoding === undefined) {
return ArrayBufferPrototypeSlice(asset);
}
const decoder = new TextDecoder(encoding);
return decoder.decode(asset);
}

/**
* Look for the asset in the injected SEA blob using the key. If
* no matching asset is found an error is thrown. The data is returned
* in a Blob. If no matching asset is found an error is thrown.
* @param {string} key
* @param {ConstructorParameters<Blob>[1]} [options]
* @returns {Blob}
*/
function getAssetAsBlob(key, options) {
const asset = getRawAsset(key);
return new Blob([asset], options);
}

module.exports = {
isSea,
getAsset,
getAssetAsBlob,
};
48 changes: 48 additions & 0 deletions src/json_parser.cc
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#include "util-inl.h"

namespace node {
using v8::Array;
using v8::Context;
using v8::Isolate;
using v8::Local;
Expand Down Expand Up @@ -101,4 +102,51 @@ std::optional<bool> JSONParser::GetTopLevelBoolField(std::string_view field) {
return value->BooleanValue(isolate);
}

std::optional<JSONParser::StringDict> JSONParser::GetTopLevelStringDict(
std::string_view field) {
Isolate* isolate = isolate_.get();
v8::HandleScope handle_scope(isolate);
Local<Context> context = context_.Get(isolate);
Local<Object> content_object = content_.Get(isolate);
Local<Value> value;
bool has_field;
// It's not a real script, so don't print the source line.
errors::PrinterTryCatch bootstrapCatch(
isolate, errors::PrinterTryCatch::kDontPrintSourceLine);
Local<Value> field_local;
if (!ToV8Value(context, field, isolate).ToLocal(&field_local)) {
return std::nullopt;
}
if (!content_object->Has(context, field_local).To(&has_field)) {
return std::nullopt;
}
if (!has_field) {
return StringDict();
}
if (!content_object->Get(context, field_local).ToLocal(&value) ||
!value->IsObject()) {
return std::nullopt;
}
Local<Object> dict = value.As<Object>();
Local<Array> keys;
if (!dict->GetOwnPropertyNames(context).ToLocal(&keys)) {
return std::nullopt;
}
std::unordered_map<std::string, std::string> result;
uint32_t length = keys->Length();
for (uint32_t i = 0; i < length; ++i) {
Local<Value> key;
Local<Value> value;
if (!keys->Get(context, i).ToLocal(&key) || !key->IsString())
return StringDict();
if (!dict->Get(context, key).ToLocal(&value) || !value->IsString())
return StringDict();

Utf8Value key_utf8(isolate, key);
Utf8Value value_utf8(isolate, value);
result.emplace(*key_utf8, *value_utf8);
}
return result;
}

} // namespace node
3 changes: 3 additions & 0 deletions src/json_parser.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include <memory>
#include <optional>
#include <string>
#include <unordered_map>
#include "util.h"
#include "v8.h"

Expand All @@ -15,11 +16,13 @@ namespace node {
// complicates things.
class JSONParser {
public:
using StringDict = std::unordered_map<std::string, std::string>;
JSONParser();
~JSONParser() = default;
bool Parse(const std::string& content);
std::optional<std::string> GetTopLevelStringField(std::string_view field);
std::optional<bool> GetTopLevelBoolField(std::string_view field);
std::optional<StringDict> GetTopLevelStringDict(std::string_view field);

private:
// We might want a lighter-weight JSON parser for this use case. But for now
Expand Down
Loading

0 comments on commit cea5295

Please sign in to comment.