Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making node_process and part of bootstrapper.cc an internalBinding #24961

Closed
joyeecheung opened this issue Dec 11, 2018 · 2 comments
Closed

Making node_process and part of bootstrapper.cc an internalBinding #24961

joyeecheung opened this issue Dec 11, 2018 · 2 comments
Labels
discuss Issues opened for discussions and feedbacks. process Issues and PRs related to the process subsystem.

Comments

@joyeecheung
Copy link
Member

Right now some bindings used to setup the process object are placed in bootstrapper.cc and are put onto a big object during bootstrap, then passed into other smaller lib/internal/process/something.js modules for further setup:

node/src/bootstrapper.cc

Lines 133 to 168 in b416daf

// The Bootstrapper object is an ephemeral object that is used only during
// the bootstrap process of the Node.js environment. A reference to the
// bootstrap object must not be kept around after the bootstrap process
// completes so that it can be gc'd as soon as possible.
void SetupBootstrapObject(Environment* env,
Local<Object> bootstrapper) {
BOOTSTRAP_METHOD(_setupTraceCategoryState, SetupTraceCategoryState);
BOOTSTRAP_METHOD(_setupNextTick, SetupNextTick);
BOOTSTRAP_METHOD(_setupPromises, SetupPromises);
BOOTSTRAP_METHOD(_chdir, Chdir);
BOOTSTRAP_METHOD(_cpuUsage, CPUUsage);
BOOTSTRAP_METHOD(_hrtime, Hrtime);
BOOTSTRAP_METHOD(_hrtimeBigInt, HrtimeBigInt);
BOOTSTRAP_METHOD(_memoryUsage, MemoryUsage);
BOOTSTRAP_METHOD(_rawDebug, RawDebug);
BOOTSTRAP_METHOD(_umask, Umask);
#if defined(__POSIX__) && !defined(__ANDROID__) && !defined(__CloudABI__)
if (env->is_main_thread()) {
BOOTSTRAP_METHOD(_initgroups, InitGroups);
BOOTSTRAP_METHOD(_setegid, SetEGid);
BOOTSTRAP_METHOD(_seteuid, SetEUid);
BOOTSTRAP_METHOD(_setgid, SetGid);
BOOTSTRAP_METHOD(_setuid, SetUid);
BOOTSTRAP_METHOD(_setgroups, SetGroups);
}
#endif // __POSIX__ && !defined(__ANDROID__) && !defined(__CloudABI__)
Local<String> should_abort_on_uncaught_toggle =
FIXED_ONE_BYTE_STRING(env->isolate(), "_shouldAbortOnUncaughtToggle");
CHECK(bootstrapper->Set(env->context(),
should_abort_on_uncaught_toggle,
env->should_abort_on_uncaught_toggle().GetJSArray())
.FromJust());
}
#undef BOOTSTRAP_METHOD

Note that not all the definitions of these methods are in bootstrapper.cc, some of those are placed in node_process.cc with declarations in node_internals.h, so for example, to bootstrap process.setgid, on the C++ side:

// 1. node_internals.h: declaration
void SetGid(const v8::FunctionCallbackInfo<v8::Value>& args);
// 2. node_process.cc: definition
void SetGid(const FunctionCallbackInfo<Value>& args) { ... }
// 3. bootstrapper.cc: SetupBootstrapObject() put it onto the bootstrap object
void SetupBootstrapObject(...) {
  ...
  BOOTSTRAP_METHOD(_setgid, SetGid);
}
// 4. node.cc: call SetupBootstrapObject() during bootstrap and pass the object into node.js
SetupBootstrapObject(env, bootstrapper);

On the JS side:

// 1. bootstrap/node.js: get this through a big bootstrap object passed from C++
const { _setgid } = bootstrappers;
// 2. bootstrap/node.js: Then it pass this to a method exposed by main_thread_only.js
mainThreadSetup.setupProcessMethods(... _setgid ...);
// 3. main_thread_only.js: wrap this binding with some validation, and
// directly write the method to the process object
function setupProcessMethods(_setgid) {
  if (_setgid !== undefined) { // POSIX
    setupPosixMethods(..._setgid..);
  }
}
function setupPosixMethods (..._setgid..) {
  process.setgid = function setgid(id) {
     return execId(id, 'Group', _setgid);
  };
}

There are several problems with the current setup:

  1. The code involved in bootstrapping these small methods span across too many files which make them hard to track down
  2. We write to the process object in a separate file, doing this everywhere makes it difficult to figure out the order of write access to the process object and the state of it, which creates difficulty for the v8 snapshot effort
  3. Methods like process.setgid is not that frequently used, we don't really need these code to be in the highlight of the bootstrap process

I propose we refactor these process methods to this setup:

// 1. Remove declaration in node_internals.h and declaration bootstrapper.cc
// 2. Make node_process.cc a binding available through `internalBinding('process')`,
//     then SetGid would be available to JS land as `internalBinding('process').setgid`
// node_process.cc: contains both definition and initialization
void SetGid(const FunctionCallbackInfo<Value>& args) { ... }
void Initialize(...) {
  env->SetMethod(target, "setgid", SetGid);
}
NODE_MODULE_CONTEXT_AWARE_INTERNAL(process, node::process::Initialize)

In JS

// 1. main_thread_only.js: return an implementation of setgid in a side-effect free manner, and load
// the binding directly in this file
const binding = internalBinding('process');
exports.hasPosixCredentials = !!binding.setgid;
if (exports.hasPosixCredentials) {  // TODO: this can be passed into the bootstrap script directly
  exports.setgid = function setgid(id) {
     return execId(id, 'Group', binding.setgid);
  };
}
// 2. bootstrap/node.js: re-export the implementation to process by writing to the process object
if (isMainThread) {
  const mainThreadSetup = NativeModule.require('internal/process/main_thread_only');
  if (mainThreadSetup.hasPosixCredentials) {
    process.setgid = mainThreadSetup.setgid;
  }
}

This way, the number of files involved in the implementation is reduced from 4(C++) + 2(JS) to 1(C++) + 2(JS), and the write to the process object are centralized in bootstrap/node.js so it's easier to figure out the state of the process object during the bootstrap.

WDYT?

cc @nodejs/process @addaleax @jasnell @devsnek

@joyeecheung joyeecheung added discuss Issues opened for discussions and feedbacks. process Issues and PRs related to the process subsystem. labels Dec 11, 2018
joyeecheung added a commit to joyeecheung/node that referenced this issue Dec 18, 2018
Expose the POSIX credential accessors through
`internalBinding('credentials')` instead of setting them on the
process or bootstrapper object from C++ directly. Also moves
`SafeGetEnv` from `internalBinding('util')` to
`internalBinding('credentials')` since it's closely related to
the credentials.

In the JS land, instead of wrapping the bindings then writing
to the process object directly in main_thread_only.js, return
the wrapped functions back to bootstrap/node.js where they get
written to the process object conditionally for clarity.

Refs: nodejs#24961
joyeecheung added a commit that referenced this issue Dec 18, 2018
Expose the POSIX credential accessors through
`internalBinding('credentials')` instead of setting them on the
process or bootstrapper object from C++ directly. Also moves
`SafeGetEnv` from `internalBinding('util')` to
`internalBinding('credentials')` since it's closely related to
the credentials.

In the JS land, instead of wrapping the bindings then writing
to the process object directly in main_thread_only.js, return
the wrapped functions back to bootstrap/node.js where they get
written to the process object conditionally for clarity.

Refs: #24961

PR-URL: #25066
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
joyeecheung added a commit to joyeecheung/node that referenced this issue Dec 19, 2018
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: nodejs#24961
joyeecheung added a commit to joyeecheung/node that referenced this issue Dec 21, 2018
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: nodejs#24961
joyeecheung added a commit that referenced this issue Dec 21, 2018
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: #24961

PR-URL: #25127
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
joyeecheung added a commit that referenced this issue Dec 24, 2018
PR-URL: #25163
Refs: #24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
joyeecheung added a commit that referenced this issue Dec 24, 2018
This patch:

- Moves the process.nextTick and promise setup C++ code into
  node_task_queue.cc which is exposed as
  `internalBinding('task_queue')`
- Makes `lib/internal/process/promises.js` and
  `lib/internal/process/next_tick.js` as side-effect-free
  as possible
- Removes the bootstrapper object being passed into
  `bootstrap/node.js`, let `next_tick.js` and `promises.js`
  load whatever they need from `internalBinding('task_queue')`
  instead.
- Rename `process._tickCallback` to `runNextTicks` internally
  for clarity but still expose it as `process._tickCallback`.

PR-URL: #25163
Refs: #24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
addaleax pushed a commit that referenced this issue Jan 14, 2019
Expose the POSIX credential accessors through
`internalBinding('credentials')` instead of setting them on the
process or bootstrapper object from C++ directly. Also moves
`SafeGetEnv` from `internalBinding('util')` to
`internalBinding('credentials')` since it's closely related to
the credentials.

In the JS land, instead of wrapping the bindings then writing
to the process object directly in main_thread_only.js, return
the wrapped functions back to bootstrap/node.js where they get
written to the process object conditionally for clarity.

Refs: #24961

PR-URL: #25066
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
refack pushed a commit to refack/node that referenced this issue Jan 14, 2019
Expose the POSIX credential accessors through
`internalBinding('credentials')` instead of setting them on the
process or bootstrapper object from C++ directly. Also moves
`SafeGetEnv` from `internalBinding('util')` to
`internalBinding('credentials')` since it's closely related to
the credentials.

In the JS land, instead of wrapping the bindings then writing
to the process object directly in main_thread_only.js, return
the wrapped functions back to bootstrap/node.js where they get
written to the process object conditionally for clarity.

Refs: nodejs#24961

PR-URL: nodejs#25066
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
refack pushed a commit to refack/node that referenced this issue Jan 14, 2019
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: nodejs#24961

PR-URL: nodejs#25127
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
refack pushed a commit to refack/node that referenced this issue Jan 14, 2019
PR-URL: nodejs#25163
Refs: nodejs#24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
refack pushed a commit to refack/node that referenced this issue Jan 14, 2019
This patch:

- Moves the process.nextTick and promise setup C++ code into
  node_task_queue.cc which is exposed as
  `internalBinding('task_queue')`
- Makes `lib/internal/process/promises.js` and
  `lib/internal/process/next_tick.js` as side-effect-free
  as possible
- Removes the bootstrapper object being passed into
  `bootstrap/node.js`, let `next_tick.js` and `promises.js`
  load whatever they need from `internalBinding('task_queue')`
  instead.
- Rename `process._tickCallback` to `runNextTicks` internally
  for clarity but still expose it as `process._tickCallback`.

PR-URL: nodejs#25163
Refs: nodejs#24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
addaleax pushed a commit to addaleax/node that referenced this issue Jan 14, 2019
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: nodejs#24961

PR-URL: nodejs#25127
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
addaleax pushed a commit that referenced this issue Jan 15, 2019
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: #24961

PR-URL: #25127
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>

Backport-PR-URL: #25496
addaleax pushed a commit that referenced this issue Jan 15, 2019
PR-URL: #25163
Refs: #24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
addaleax pushed a commit that referenced this issue Jan 15, 2019
This patch:

- Moves the process.nextTick and promise setup C++ code into
  node_task_queue.cc which is exposed as
  `internalBinding('task_queue')`
- Makes `lib/internal/process/promises.js` and
  `lib/internal/process/next_tick.js` as side-effect-free
  as possible
- Removes the bootstrapper object being passed into
  `bootstrap/node.js`, let `next_tick.js` and `promises.js`
  load whatever they need from `internalBinding('task_queue')`
  instead.
- Rename `process._tickCallback` to `runNextTicks` internally
  for clarity but still expose it as `process._tickCallback`.

PR-URL: #25163
Refs: #24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
BridgeAR pushed a commit to BridgeAR/node that referenced this issue Jan 16, 2019
Expose the POSIX credential accessors through
`internalBinding('credentials')` instead of setting them on the
process or bootstrapper object from C++ directly. Also moves
`SafeGetEnv` from `internalBinding('util')` to
`internalBinding('credentials')` since it's closely related to
the credentials.

In the JS land, instead of wrapping the bindings then writing
to the process object directly in main_thread_only.js, return
the wrapped functions back to bootstrap/node.js where they get
written to the process object conditionally for clarity.

Refs: nodejs#24961

PR-URL: nodejs#25066
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
BridgeAR pushed a commit to BridgeAR/node that referenced this issue Jan 16, 2019
Instead of:

- Writing methods onto the process directly in C++ during
  `SetupProcessObject()` and overwrite with argument checks later
- Or, wrapping and writing them in `internal/process/*.js`

Do:

- Move the C++ implementations in node_process.cc and mark them static
  wherever possible
- Expose the C++ methods through a new
  `internalBinding('process_methods')`
- Wrap the methods in `internal/process/*.js` in a
  side-effect-free manner and return them back to
  `internal/bootstrap/node.js`
- Centralize the write to the process object based on conditions
  in `bootstrap/node.js`

So it's easier to see what methods are attached to the process object
during bootstrap under what condition and in what order.

The eventual goal is to figure out the dependency of process methods
and the write/read access to the process object during bootstrap, group
these access properly and remove the process properties that should not
be exposed to users this way.

Also correct the NODE_PERFORMANCE_MILESTONE_BOOTSTRAP_COMPLETE milestone
which should be marked before code execution.

Refs: nodejs#24961

PR-URL: nodejs#25127
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>

Backport-PR-URL: nodejs#25496
BridgeAR pushed a commit to BridgeAR/node that referenced this issue Jan 16, 2019
PR-URL: nodejs#25163
Refs: nodejs#24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
BridgeAR pushed a commit to BridgeAR/node that referenced this issue Jan 16, 2019
This patch:

- Moves the process.nextTick and promise setup C++ code into
  node_task_queue.cc which is exposed as
  `internalBinding('task_queue')`
- Makes `lib/internal/process/promises.js` and
  `lib/internal/process/next_tick.js` as side-effect-free
  as possible
- Removes the bootstrapper object being passed into
  `bootstrap/node.js`, let `next_tick.js` and `promises.js`
  load whatever they need from `internalBinding('task_queue')`
  instead.
- Rename `process._tickCallback` to `runNextTicks` internally
  for clarity but still expose it as `process._tickCallback`.

PR-URL: nodejs#25163
Refs: nodejs#24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
BridgeAR pushed a commit to BridgeAR/node that referenced this issue Jan 17, 2019
PR-URL: nodejs#25163
Refs: nodejs#24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
BridgeAR pushed a commit to BridgeAR/node that referenced this issue Jan 17, 2019
This patch:

- Moves the process.nextTick and promise setup C++ code into
  node_task_queue.cc which is exposed as
  `internalBinding('task_queue')`
- Makes `lib/internal/process/promises.js` and
  `lib/internal/process/next_tick.js` as side-effect-free
  as possible
- Removes the bootstrapper object being passed into
  `bootstrap/node.js`, let `next_tick.js` and `promises.js`
  load whatever they need from `internalBinding('task_queue')`
  instead.
- Rename `process._tickCallback` to `runNextTicks` internally
  for clarity but still expose it as `process._tickCallback`.

PR-URL: nodejs#25163
Refs: nodejs#24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
@jasnell
Copy link
Member

jasnell commented Jun 26, 2020

@joyeecheung ... does this need to remain open?

@joyeecheung
Copy link
Member Author

@jasnell I think this is already done. I'll close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Issues opened for discussions and feedbacks. process Issues and PRs related to the process subsystem.
Projects
None yet
Development

No branches or pull requests

2 participants