module: add createRequireFunction method #19360

devsnek · 2018-03-14T23:43:46Z

useful method for all sorts of stuff such as relative require from project root or whatever else you can come up with

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

devsnek · 2018-03-14T23:44:23Z

CI https://ci.nodejs.org/job/node-test-pull-request/13690/

addaleax · 2018-03-14T23:46:04Z

doc/api/modules.md

+
+<!-- YAML
+added: REPLACEME
+-->


This should go directly below the heading

lol i have no idea how this got down here, will fix

addaleax · 2018-03-14T23:47:00Z

doc/api/modules.md

@@ -850,6 +850,21 @@ by the [module wrapper][]. To access it, require the `Module` module:
 const builtin = require('module').builtinModules;
 ```

+### module.makeRequire(filename)
+
+* `filename` {string|[URL][]}


No need to link here, the type parser should pick URL up automatically

addaleax · 2018-03-14T23:47:18Z

doc/api/modules.md

+### module.makeRequire(filename)
+
+* `filename` {string|[URL][]}
+* Returns: {[require][]} Require function


Code backticks for require?

addaleax · 2018-03-14T23:47:32Z

lib/module.js

+  m.filename = filename;
+  m.paths = Module._nodeModulePaths(path.dirname(filename));
+  const require = internalModule.makeRequireFunction(m);
+  return require;


Tiny nit: You could just return on the previous line :)

addaleax · 2018-03-14T23:47:56Z

test/parallel/test-module-make-require.js

+const Module = require('module');
+
+const p = path.resolve(__dirname, '..', 'fixtures', 'fake.js');
+const req = Module.makeRequire(p);


Maybe also test the URL functionality?

vsemozhetbyt · 2018-03-15T00:41:04Z

doc/api/modules.md

+```js
+// create a require function which will resolve relative
+// to the current ES Module file
+const require = module.makeRequire(new URL(import.meta.url));


module -> require('module') ?
(in sync with the example above and to not mix with module instances)

vsemozhetbyt · 2018-03-15T00:46:58Z

doc/api/modules.md

+-->
+
+* `filename` {string|URL} Filename for relative importing
+* Returns: {[`require`][]} Require function


Maybe it would be more consistent with other cases to add this to type parser between other Node.js "types". But I am not sure if we should do this for all solitary cases.

worth a separate pr? (also yay for them all being slightly different)

doc/api/modules.md 873:[`require`]: #modules_require doc/api/addons.md 1177:[require]: modules.html#modules_require doc/api/globals.md 168:[`require()`]: modules.html#modules_require

vsemozhetbyt · 2018-03-15T00:49:17Z

doc/api/modules.md

@@ -859,3 +874,4 @@ const builtin = require('module').builtinModules;
 [module resolution]: #modules_all_together
 [module wrapper]: #modules_the_module_wrapper
 [native addons]: addons.html
+[`require`]: #modules_require


Maybe this better be placed after the [`path.dirname()`] reference, ASCII-wise.

jdalton · 2018-03-15T05:39:04Z

doc/api/modules.md

+// to the current ES Module file
+const require = require('module').makeRequire(new URL(import.meta.url));
+```
+


👆 the code example is clunky because require doesn't exist in ESM and you're assigning a const named require while calling the require function.

jdalton · 2018-03-15T05:41:29Z

lib/module.js

@@ -24,7 +24,8 @@
 const { NativeModule } = require('internal/bootstrap_loaders');
 const util = require('util');
 const { decorateErrorStack } = require('internal/util');
-const { getURLFromFilePath } = require('internal/url');
+const { getURLFromFilePath, getPathFromURL } = require('internal/url');
+const { URL } = require('url');
 const vm = require('vm');


👆 URL is already exposed on the global now. No need to require it.

internally we still require it in case it's overridden

Ah, good catch, yes.

jdalton · 2018-03-15T16:16:52Z

doc/api/modules.md

+// create a require function which will resolve relative
+// to the current ES Module file
+import Module from 'module';
+const require = Module.makeRequire(new URL(import.meta.url));


Is there a way to support simply passing the url string around? Smoothing that over to just work without having to instantiate the URL constructor for things like this would be nice.

its done like that to mirror how fs allows URL objects, i would rather not diverge from that

vkurchatkin · 2018-03-15T16:27:14Z

What is the use-case for this?

devsnek · 2018-03-15T17:07:09Z

@vkurchatkin mostly just what the example has, but i've found myself creating require functions twice now for things not related to this (mostly very reflective/hacky stuff) and so i thought it might be useful to just make a utility function

devsnek · 2018-03-17T21:46:55Z

this could use a few more approvals

bnoordhuis · 2018-03-18T12:40:56Z

The bar for new APIs should be higher than 'might be useful,' IMO. The price of design mistakes is high.

devsnek · 2018-03-21T14:39:15Z

@bnoordhuis this is designed to alleviate some issues with cjs and esm interop, it's definitely something that will be useful and used, not just a random feature. if you're worried about design mistakes I encourage that you to review this in depth :)

jdalton · 2018-03-21T16:53:41Z

this is designed to alleviate some issues with cjs and esm interop

Still seems more inconvenient than needed.
I'm lukewarm on the API as is, but dig exposing makeRequireFunction in some way.

bmeck · 2018-06-14T22:26:04Z

CC: @nodejs/modules

I'd like to reopen this for discussion in light of some conversation going on in the modules WG nodejs/modules#130 (comment) and the comments leading up to it.

In particular, if exposing this is better than shipping import.meta.require. It does enable what appears to be better ergonomics than telling people to import a require() function and also is a means of keeping us from inevitably encouraging using CJS APIs into out initial ESM implementation.

It does have the ability for us to create new require() functions for other uses outside of the import.meta.require discussion and I'd like to see if any of those are problematic.

I'd note that you can create something akin to makeRequireFunction ~with :

function makeRequire(filepath) {
  const Module = require('module');
  const m = new Module(filepath, null);
  m._compile('module.exports=require', filepath);
  return m.exports;
}

I don't think using `m._compile` is problematic in my example and would refrain from that discussion.

I took a look at how much `m._compile` corresponds to in the wild usage and I see it used in a variety of places and don't think that it could easily be changed from current the CJS implementation : https://github.com/search?q=_compile+language%3Ajavascript&type=Code , there are false positives but plenty that look to be usage of it from Node's core.

In particular, this is an API ergonomics question I think.

benjamingr

+1 good idea I needed this before and had to use hacks.

jkrems

I have needed this at various times (mostly for "require relative to project directory"). I'm not sure this is a replacement for import.meta.require but I think it's a worthwhile feature even from a pure CJS perspective.

jasnell · 2018-06-15T15:34:28Z

test/parallel/test-module-create-require.js

+require('../common');
+const assert = require('assert');
+const path = require('path');
+const { URL } = require('url');


fwiw, URL should be a global now. There's no reason to require it.

are there any release lines where that hasn't been backported yet?

Yes, it's not on v8.x: #20306

jasnell · 2018-06-15T15:36:12Z

doc/api/modules.md

+import Module from 'module';
+const require = Module.createRequireFunction(new URL(import.meta.url));
+```
+


Some explanation about why this is useful would probably be a good thing in here. It will be non-obvious to many users and the lack of explanation could lead to confusion.

Can you also add an example using cjs and require? This is useful outside of ESM.

Notable changes: * fs * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If this option is set to `true`, non-existing parent folders will be automatically created. #21875 * Fixed fsPromises.readdir `withFileTypes`. #22832 * http2 * Added `http2stream.endAfterHeaders` property. #22843 * module * Added `module.createRequireFromPath(filename)`. This new method can be used to create a custom `require` function that will resolve modules relative to the `filename` path. #19360 * url * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These methods can be used to correctly convert between `file:` URLs and absolute paths. #22506 * util * Added `util.types.isBoxedPrimitive(value)`. #22620 * Windows * The Windows msi installer now provides an option to automatically install the tools required to build native modules. #22645 * Added new collaborators: * boneskull (https://github.com/boneskull) - Christopher Hiller * The Technical Steering Committee has new members: * apapirovski (https://github.com/apapirovski) - Anatoli Papirovski * gabrielschulhof (https://github.com/gabrielschulhof) - Gabriel Schulhof PR-URL: #22932

PR-URL: #19360 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Bradley Farias <bradley.meck@gmail.com> Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>

Notable changes: * fs * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If this option is set to `true`, non-existing parent folders will be automatically created. #21875 * Fixed fsPromises.readdir `withFileTypes`. #22832 * http2 * Added `http2stream.endAfterHeaders` property. #22843 * module * Added `module.createRequireFromPath(filename)`. This new method can be used to create a custom `require` function that will resolve modules relative to the `filename` path. #19360 * url * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These methods can be used to correctly convert between `file:` URLs and absolute paths. #22506 * util * Added `util.types.isBoxedPrimitive(value)`. #22620 * Added new collaborators: * boneskull (https://github.com/boneskull) - Christopher Hiller * The Technical Steering Committee has new members: * apapirovski (https://github.com/apapirovski) - Anatoli Papirovski * gabrielschulhof (https://github.com/gabrielschulhof) - Gabriel Schulhof PR-URL: #22932

targos · 2018-09-20T05:54:43Z

Depends on #21875. Marking dont-land-on-v10.x for now.

PR-URL: #19360 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jan Krems <jan.krems@gmail.com> Reviewed-By: Bradley Farias <bradley.meck@gmail.com> Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>

Notable changes: * assert * The diff output is now a tiny bit improved by sorting object properties when inspecting the values that are compared with each other. #22788 * cli * The options parser now normalizes `_` to `-` in all multi-word command-line flags, e.g. `--no_warnings` has the same effect as `--no-warnings`. #23020 * Added bash completion for the `node` binary. To generate a bash completion script, run `node --completion-bash`. The output can be saved to a file which can be sourced to enable completion. #20713 * crypto * Added support for PEM-level encryption. #23151 * Added an API asymmetric key pair generation. The new methods `crypto.generateKeyPair` and `crypto.generateKeyPairSync` can be used to generate public and private key pairs. The API supports RSA, DSA and EC and a variety of key encodings (both PEM and DER). #22660 * fs * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If this option is set to true, non-existing parent folders will be automatically created. #21875 * http2 * Added a `'ping'` event to `Http2Session` that is emitted whenever a non-ack `PING` is received. #23009 * Added support for the `ORIGIN` frame. #22956 * module * Added `module.createRequireFromPath(filename)`. This new method can be used to create a custom require function that will resolve modules relative to the filename path. #19360 * process * Added a `'multipleResolves'` process event that is emitted whenever a `Promise` is attempted to be resolved multiple times, e.g. if the `resolve` and `reject` functions are both called in a `Promise` executor. #22218 * **url** * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These methods can be used to correctly convert between file: URLs and absolute paths. #22506 * **util** * Added the `sorted` option to `util.inspect()`. If set to `true`, all properties of an object and Set and Map entries will be sorted in the returned string. If set to a function, it is used as a compare function. #22788 * The `util.instpect.custom` symbol is now defined in the global symbol registry as `Symbol.for('nodejs.util.inspect.custom')`. #20857 * **Windows** * The Windows msi installer now provides an option to automatically install the tools required to build native modules. #22645 * **Added new collaborators**: * digitalinfinity - Hitesh Kanwathirtha PR-URL: #23313

Notable changes: * assert * The diff output is now a tiny bit improved by sorting object properties when inspecting the values that are compared with each other. #22788 * cli * The options parser now normalizes `_` to `-` in all multi-word command-line flags, e.g. `--no_warnings` has the same effect as `--no-warnings`. #23020 * Added bash completion for the `node` binary. To generate a bash completion script, run `node --completion-bash`. The output can be saved to a file which can be sourced to enable completion. #20713 * crypto * Added support for PEM-level encryption. #23151 * Added an API asymmetric key pair generation. The new methods `crypto.generateKeyPair` and `crypto.generateKeyPairSync` can be used to generate public and private key pairs. The API supports RSA, DSA and EC and a variety of key encodings (both PEM and DER). #22660 * fs * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If this option is set to true, non-existing parent folders will be automatically created. #21875 * http2 * Added a `'ping'` event to `Http2Session` that is emitted whenever a non-ack `PING` is received. #23009 * Added support for the `ORIGIN` frame. #22956 * Updated nghttp2 to 1.34.0. This adds RFC 8441 extended connect protocol support to allow use of WebSockets over HTTP/2. #23284 * module * Added `module.createRequireFromPath(filename)`. This new method can be used to create a custom require function that will resolve modules relative to the filename path. #19360 * process * Added a `'multipleResolves'` process event that is emitted whenever a `Promise` is attempted to be resolved multiple times, e.g. if the `resolve` and `reject` functions are both called in a `Promise` executor. #22218 * url * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These methods can be used to correctly convert between file: URLs and absolute paths. #22506 * util * Added the `sorted` option to `util.inspect()`. If set to `true`, all properties of an object and Set and Map entries will be sorted in the returned string. If set to a function, it is used as a compare function. #22788 * The `util.instpect.custom` symbol is now defined in the global symbol registry as `Symbol.for('nodejs.util.inspect.custom')`. #20857 * Added support for `BigInt` numbers in `util.format()`. #22097 * V8 API * A number of V8 C++ APIs have been marked as deprecated since they have been removed in the upstream repository. Replacement APIs are added where necessary. #23159 * Windows * The Windows msi installer now provides an option to automatically install the tools required to build native modules. #22645 * Workers * Debugging support for Workers using the DevTools protocol has been implemented. #21364 * The public `inspector` module is now enabled in Workers. #22769 * Added new collaborators: * digitalinfinity - Hitesh Kanwathirtha PR-URL: #23313

nodejs-github-bot added the module Issues and PRs related to the module subsystem. label Mar 14, 2018

addaleax approved these changes Mar 14, 2018

View reviewed changes

addaleax added the semver-minor PRs that contain new features and should be released in the next minor version. label Mar 14, 2018

devsnek force-pushed the feature/module-makerequire branch from 7a0ccd0 to 915a294 Compare March 15, 2018 00:16

vsemozhetbyt reviewed Mar 15, 2018

View reviewed changes

devsnek force-pushed the feature/module-makerequire branch from 915a294 to 4927c0f Compare March 15, 2018 03:49

jdalton reviewed Mar 15, 2018

View reviewed changes

devsnek force-pushed the feature/module-makerequire branch 2 times, most recently from 4bd8eb4 to 2780483 Compare March 15, 2018 15:26

jdalton reviewed Mar 15, 2018

View reviewed changes

devsnek closed this Apr 22, 2018

devsnek deleted the feature/module-makerequire branch April 22, 2018 07:59

devsnek restored the feature/module-makerequire branch June 14, 2018 22:13

devsnek reopened this Jun 14, 2018

benjamingr approved these changes Jun 15, 2018

View reviewed changes

jkrems approved these changes Jun 15, 2018

View reviewed changes

devsnek force-pushed the feature/module-makerequire branch from fff0660 to a26ddc0 Compare June 15, 2018 14:49

devsnek changed the title ~~module: add makeRequire method~~ module: add creataeRequireFunction method Jun 15, 2018

devsnek changed the title ~~module: add creataeRequireFunction method~~ module: add createRequireFunction method Jun 15, 2018

jasnell reviewed Jun 15, 2018

View reviewed changes

targos mentioned this pull request Sep 18, 2018

Release proposal: v10.11.0 #22932

Merged

targos added the dont-land-on-v10.x label Sep 20, 2018

targos mentioned this pull request Sep 20, 2018

fs: implement recursive (mkdirp) functionality #21875

Closed

4 tasks

targos removed the dont-land-on-v10.x label Sep 25, 2018

targos mentioned this pull request Oct 7, 2018

Release proposal: v10.12.0 #23313

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

module: add createRequireFunction method #19360

module: add createRequireFunction method #19360

devsnek commented Mar 14, 2018

devsnek commented Mar 14, 2018

addaleax Mar 14, 2018

devsnek Mar 14, 2018 •

edited

Loading

addaleax Mar 14, 2018

addaleax Mar 14, 2018

addaleax Mar 14, 2018

addaleax Mar 14, 2018

vsemozhetbyt Mar 15, 2018

vsemozhetbyt Mar 15, 2018

devsnek Mar 15, 2018 •

edited

Loading

vsemozhetbyt Mar 15, 2018

jdalton Mar 15, 2018

jdalton Mar 15, 2018 •

edited

Loading

devsnek Mar 15, 2018

jdalton Mar 15, 2018

jdalton Mar 15, 2018 •

edited

Loading

devsnek Mar 15, 2018

vkurchatkin commented Mar 15, 2018

devsnek commented Mar 15, 2018

devsnek commented Mar 17, 2018

bnoordhuis commented Mar 18, 2018

devsnek commented Mar 21, 2018 •

edited

Loading

jdalton commented Mar 21, 2018 •

edited

Loading

bmeck commented Jun 14, 2018

benjamingr left a comment

jkrems left a comment

jasnell Jun 15, 2018

devsnek Jun 15, 2018

richardlau Jun 15, 2018

jasnell Jun 15, 2018

mcollina Aug 29, 2018

targos commented Sep 20, 2018

module: add createRequireFunction method #19360

module: add createRequireFunction method #19360

Conversation

devsnek commented Mar 14, 2018

Checklist

devsnek commented Mar 14, 2018

Choose a reason for hiding this comment

devsnek Mar 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devsnek Mar 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdalton Mar 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdalton Mar 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vkurchatkin commented Mar 15, 2018

devsnek commented Mar 15, 2018

devsnek commented Mar 17, 2018

bnoordhuis commented Mar 18, 2018

devsnek commented Mar 21, 2018 • edited Loading

jdalton commented Mar 21, 2018 • edited Loading

bmeck commented Jun 14, 2018

benjamingr left a comment

Choose a reason for hiding this comment

jkrems left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

targos commented Sep 20, 2018

devsnek Mar 14, 2018 •

edited

Loading

devsnek Mar 15, 2018 •

edited

Loading

jdalton Mar 15, 2018 •

edited

Loading

jdalton Mar 15, 2018 •

edited

Loading

devsnek commented Mar 21, 2018 •

edited

Loading

jdalton commented Mar 21, 2018 •

edited

Loading