Skip to content

Commit

Permalink
src,doc: experimental support for SEA
Browse files Browse the repository at this point in the history
- add strategy as discussed in next-10 mini-summit
  - https://github.com/nodejs/next-10/blob/main/meetings/summit-nov-2021.md#single-executable-applications
- add initial support single executable application support for linux

Signed-off-by: Michael Dawson <mdawson@devrus.com>
  • Loading branch information
mhdawson committed Mar 14, 2022
1 parent 72e83fc commit fe4688e
Show file tree
Hide file tree
Showing 5 changed files with 215 additions and 1 deletion.
110 changes: 110 additions & 0 deletions doc/contributing/maintaining-single-executable-application-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Maintaining Single Executable Applications support

Support for [single executable applications](https://github.com/nodejs/node/blob/master/doc/contributing/technical-priorities.md#single-executable-applications)
is on of the key technical priorities identified for the success of Node.js.

## High level strategy

From the [next-10 discussions](https://github.com/nodejs/next-10/blob/main/meetings/summit-nov-2021.md#single-executable-applications)
there are 2 approaches the projects believes are important to support:

* Compile with node into executable (boxnode approach)
* Bundled onto exising Node.js exe (pkg approach)

### Compile with node into executable

No specific code within the Node.js project is needed to support the
option of compiling a bundled application along with Node.js into a single
executable application.

### Bundled onto Existing Node.js

The project does not plan to provide the complete solution but instead the key
elements which are required in the Node.js executable in order to enabling
bundling with the shipping Node.js binaries. This includes:

* Looking for a segment within the executable that holds bundled code.
* Running the bundled code when such a segement is found.

It is left up to external tools/solutions to:

* Bundled code into a single script that can be executed with -e on
the command line.
* Generating a command line with appropriate options, including -e to
run the bundle script.
* adding a segment to the existing Node.js executable which contains
the command line and appropriate headers.
* re-generating or removing signatures on the resulting executable
* providing a virtual file system, and hooking it in if needed to
support native modules or reading file contents.

## Maintaining

### Compile with node into executable

There are not special consideration needed.

### Bundled onto Existing Node.js

The following header must be included in a segment in order to have it run
as a single executable application:

JSCODEVVVVVVVVFFFFFFFFF

where:

* VVVVVVVV string representing the version to be used to interpret the section,
for example `00000001`.
* FFFFFFFF string representing the flags to be used in the process of starting
the bundled application. Currently this must be `00000000` to indicate that
no flags are set.

The characters in both `VVVVVVVV` and `FFFFFFFF` are restricted to being
between the characters "0" and "F" such that they can each be converted to
a 32 bit integer.

The string following the header is treated as a set of command line options
that are used as a prefix to any additional command line options passed when
the executable was started. For example, for a simple single hello world
for version `00000001`could be:

```text
JSCODE0000000100000000-e \"console.log('Hello from single binary');\"
```

Support for bundling onto existing Node.js binaries is maintained
in `src/node_single_binary.*`.

Currently only POSIX compliant platform are suppported. The goal
is to expand this to include Windows and MacOS as well.

If a breaking change is required to the content after the header, the version
`VVVVVVVV` should be incremented. Support for a new format
may be introduced as a SemVer minor provided that older versions
are still supported. Removing support for a version is SemVer major.

The `FFFFFFFF` are a set of flags that are used to control the
process of starting the application. For example they might indicate
that some set of arguments should be suppressed on the command line.
Currently no flags are in use.

For test purposes \[LIFE}(<https://github.com/lief-project/LIEF>) can
be used to add a section in the required format. The following is a
simple example for doing so on Linux. It can be improved as it
currently replaces an existing section instead of adding a new
one:

```text
#!/usr/bin/env python
import lief
binary = lief.parse('node')
segment = lief.ELF.Segment()
segment.type = lief.ELF.SEGMENT_TYPES.LOAD
segment.flags = lief.ELF.SEGMENT_FLAGS.R
stringContent = "JSCODE0000000100000000-e \"console.log('Hello from single binary');\""
segment.content = bytearray(stringContent.encode())
segment = binary.replace(segment, binary[lief.ELF.SEGMENT_TYPES.NOTE])
binary.write("hello")
```
1 change: 1 addition & 0 deletions node.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,7 @@
'src/node_report_module.cc',
'src/node_report_utils.cc',
'src/node_serdes.cc',
'src/node_single_binary.cc',
'src/node_snapshotable.cc',
'src/node_sockaddr.cc',
'src/node_stat_watcher.cc',
Expand Down
11 changes: 10 additions & 1 deletion src/node.cc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
#include "node_revert.h"
#include "node_v8_platform-inl.h"
#include "node_version.h"
#include "node_single_binary.h"

#if HAVE_OPENSSL
#include "allocated_buffer-inl.h" // Inlined functions needed by node_crypto.h
Expand Down Expand Up @@ -1148,7 +1149,15 @@ void TearDownOncePerProcess() {
}

int Start(int argc, char** argv) {
InitializationResult result = InitializeOncePerProcess(argc, argv);
node::single_binary::NewArgs* newArgs =
node::single_binary::checkForSingleBinary(argc, argv);

InitializationResult result;
if (!newArgs->singleBinary) {
result = InitializeOncePerProcess(argc, argv);
} else {
result = InitializeOncePerProcess(newArgs->argc, newArgs->argv);
}
if (result.early_return) {
return result.exit_code;
}
Expand Down
72 changes: 72 additions & 0 deletions src/node_single_binary.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#include "node_single_binary.h"
#if defined(__POSIX__) && !defined(_AIX) && !defined(__APPLE__)
#include <link.h>
#include <stdlib.h>
#include <string.h>
#include <wordexp.h>
#endif // defined(__POSIX__) && !defined(_AIX) && !defined(__APPLE__)

namespace node {
namespace single_binary {

#define MAGIC_HEADER "JSCODE"
#define VERSION_CHARS "00000001"
#define FLAG_CHARS "00000000"

#if defined(__POSIX__) && !defined(_AIX) && !defined(__APPLE__)
static int
callback(struct dl_phdr_info* info, size_t size, void* data) {
// look for the segment with the magic number
for (int index = 0; index < info->dlpi_phnum; index++) {
if (info->dlpi_phdr[index].p_type == PT_LOAD) {
char* content =
reinterpret_cast<char*>(info->dlpi_addr +
info->dlpi_phdr[index].p_vaddr);
if (strncmp(MAGIC_HEADER,
content,
strlen(MAGIC_HEADER)) == 0) {
*(static_cast<char**>(data)) = content;
break;
}
}
}
return 0;
}
#endif // defined(__POSIX__) && !defined(_AIX) && !defined(__APPLE__)

struct NewArgs* checkForSingleBinary(int argc, char** argv) {
struct NewArgs* newArgs = new NewArgs;
newArgs->singleBinary = false;

#if defined(__POSIX__) && !defined(_AIX) && !defined(__APPLE__)
char* singleBinaryData = nullptr;
dl_iterate_phdr(callback, static_cast<void*>(&singleBinaryData));
if (singleBinaryData != nullptr) {
wordexp_t parsedArgs;
wordexp(&(singleBinaryData[strlen(MAGIC_HEADER) +
strlen(VERSION_CHARS) +
strlen(FLAG_CHARS)]),
&parsedArgs, WRDE_NOCMD);
newArgs->argc = 0;
while (parsedArgs.we_wordv[newArgs->argc] != nullptr)
newArgs->argc++;
newArgs->argc = newArgs->argc + argc;
newArgs->argv = new char*[newArgs->argc];
newArgs->argv[0] = argv[0];
int index = 1;
while (parsedArgs.we_wordv[index-1] != nullptr) {
newArgs->argv[index] = parsedArgs.we_wordv[index-1];
index++;
}
for (int i = 1; i < argc; i++) {
newArgs->argv[index++] = argv[i];
}

newArgs->singleBinary = true;
}
#endif // defined(__POSIX__) && !defined(_AIX) && !defined(__APPLE__)
return newArgs;
}

} // namespace single_binary
} // namespace node
22 changes: 22 additions & 0 deletions src/node_single_binary.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#ifndef SRC_NODE_SINGLE_BINARY_H_
#define SRC_NODE_SINGLE_BINARY_H_

#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS

namespace node {
namespace single_binary {

struct NewArgs {
bool singleBinary;
int argc;
char** argv;
};

NewArgs* checkForSingleBinary(int argc, char** argv);

} // namespace single_binary
} // namespace node

#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS

#endif // SRC_NODE_SINGLE_BINARY_H_

0 comments on commit fe4688e

Please sign in to comment.