Skip to content
This repository has been archived by the owner on Sep 2, 2023. It is now read-only.

Proposal: minimal esm implementation #141

Closed
MylesBorins opened this issue Jun 28, 2018 · 68 comments
Closed

Proposal: minimal esm implementation #141

MylesBorins opened this issue Jun 28, 2018 · 68 comments

Comments

@MylesBorins
Copy link
Contributor

MylesBorins commented Jun 28, 2018

Hey All,

Like everyone here I've been thinking about this a whole bunch. For a while I've been trying to come up with a minimal implementation that we can iterate on. Something that could offer a fast path to deflagging.

Features

  • esm works with flag --experimental-modules
  • must use .mjs as entry point for node binary
  • no transparent interoperability
  • import.meta.require for cjs interop in esm
  • dynamic import for esm interop in cjs
  • package-name-map compliance for esm specifier resolution
    • must provide full path to module
    • no support for importing directories

Try it out today

Repo: https://github.com/MylesBorins/node/tree/esm-kernel

Download: https://nodejs.org/download/test/v11.0.0-test2018070976df5841a1/

$ NVM_NODEJS_ORG_MIRROR=https://nodejs.org/download/test   nvm install v11.0.0-test2018070976df5841a1

warning super naive implementation, lots of room for improvement... but it shows off the desired UX and all the tests pass locally on my machine.

Upstream PRs

TODO

  • improve implementation
  • get tests working on all platforms
  • build consensus
  • ensure loaders works
  • ensure vm works

Note

I have not opened a PR upstream to remove transparent interoperability as I want to be respectful of the current conversations going on within the group. While import.meta.require and limiting extensions are not guaranteed to land, I do think that they are worth discussing independently on their own merits.

@WebReflection
Copy link
Contributor

WebReflection commented Jun 28, 2018

Quick review from user perspective.

Not only the binary

You wrote "must use .mjs as entry point for node binary" but I've noticed .mjs is mandatory as entry point regardless it's a binary file or not. Since there is a mechanism to start ESM, I'd love to have a flag that forces ESM as in node --experimental-modules --esm index.js.

Overall working ...

Overall everything seems to work as expected:

  • you can import fs from 'fs';
  • you can import rand from './module.js'; where module.js contains export default Math.random();
  • you can const {require} = import.meta; and bring in CJS as you go
  • you can require('./module.c.js') with module.exports = import('./module.js').then(module => module.default); in it and it works as expected, also same exact module
import fs from 'fs';
import rand from './module.js';

const {require} = import.meta;

const readFile = require('util').promisify(fs.readFile);

Promise.all([
  readFile('./package-map.json'),
  require('./module.c.js')
]).then(([data, crand]) => {
  console.log(rand === crand);
  console.log(data.toString());
});

... but !

I am not sure I am doing it right with the package-map.json file, but I've used this content:

{
  "path_prefix": "./node_modules",
  "packages": {
    "flatted": { "main": "esm/index.js" }
  }
}

and there's no way I can import flatted from the previous index.mjs file via import {parse} from 'flatted';

The ./node_modules/flatted/esm/index.js is a 100% valid ESM file already working on browsers and also NodeJS (but you need to use CJS otherwise it fails).

Last, but not least

  • I was incapable to bootstrap ESM via node --experimental-modules -e 'import("./index.js").catch(console.error)'
  • I was unable to use __filename or __dirname and the import.meta wouldn't provide any alternative

Wishes

  • the .mjs at this point is needed just as entry point, meaning it's not really needed if not to signal how an env should be parsed. I wish we could use a flag for that instead of an extension.
  • any alternative to __dirname or __filename would be great. I know it's relatively trivial to have both on user land but the following seems unnecessary boilerplate
import path from 'path';
const {
  pathname: __filename,
  __dirname = path.dirname(__filename)
} = new URL(import.meta.url);

console.log(__filename);
console.log(__dirname);

@bmeck
Copy link
Member

bmeck commented Jun 28, 2018

Can you define "transparent" interop here so that it doesn't get confused.

I remain strongly opposed to import.meta.require in light of my discussions on the related PR.

I would like to better understand the reasoning behind all of these choices rather than just seeing something that works. It would be nice to see a write up on what things were chosen not to be supported and why the use cases they provide can be served with this minimal implementation.

  • I have a particular concern about not being able to import CJS/C++ that I have brought up before and consider it necessary for any minimal implementation due to ordering concerns. Explanation of how those concerns are alleviated still in this minimal approach would be good.

  • This seems to imply a Major breaking change of requiring the entrypoint to be ESM once unflagged? How does this affect "bin" scripts that currently execute as CJS.

  • The package-name-map compliance seems to be at odds with current discussions about migration patterns in Patterns for interoperability #139 . It would be good to know how migration is expected to occur here.

@demurgos
Copy link

The package-name-map compliance seems to be at odds with current discussions about migration patterns in #139 . It would be good to know how migration is expected to occur here.

To clarify, the issue with the package-name maps is that it requires the specifier to have an extension. Unless there's is another way to branch on the module type used by the consumer, this rules out the patterns relying on "mjs + js" (see table in #139). This currently leaves only "Default Export" and its clunky API as a completely safe pattern.

@jkrems
Copy link
Contributor

jkrems commented Jun 28, 2018

I really like the direction of going for a more minimal solution.

I agree with @bmeck that there's some open questions. But I'd like to address them from a perspective of "what is the use case". So "Loading of polyfill or APM instrumentation that is implemented in CJS" instead of just "ordering concerns". Because that will most likely inform how we talk about it. When refactoring an application with "random" CJS files, the order is not as problematic in the general case.

@bmeck
Copy link
Member

bmeck commented Jun 28, 2018

@jkrems we have files for configuration and singleton modules at work that need to evaluate prior to others. They configure things like servers and database connection pools. These are outside of polyfill or APM concerns.

@jkrems
Copy link
Contributor

jkrems commented Jun 28, 2018

But I assume (hope) that they don't set random globals that other files then just assume are initialized..? If they do, I would treat them as "polyfills" (mutate global state to expose additional APIs).

@jkrems
Copy link
Contributor

jkrems commented Jun 28, 2018

For additional color, the following will "just work":

import fs from 'fs';
import randomLib from 'esm-lib';

const db = import.meta.require('db');

db.accessTable(); // db init will still happen before this

@devsnek
Copy link
Member

devsnek commented Jun 28, 2018

for even more color, the following will "just fail":

import.meta.require('x_polyfill'); // only available as cjs
import "something_expecting_x";   // only available as esm

I really don't like the term "just works" because it leaves out all the things that don't "just work"

@jkrems
Copy link
Contributor

jkrems commented Jun 28, 2018

One possible answer for "how would polyfills work in that world" would be "it requires an intermediate module", e.g.:

import.meta.require('polyfill-a');

await import('./run-app');

But that definitely reduce the abilities of modules that need to ensure a polyfill (no static import of actual code, pretty awkward exports).

@bmeck
Copy link
Member

bmeck commented Jun 28, 2018

I hate the term "just work"/"just fail" because it shuts down discussion of what/why things act a specific way under a guise of it being plainly apparent. We should try and explain why things serve or do not serve a use case.

I also want to bring up concerns that I have expressed elsewhere about introducing CJS permanently to all ESM modules under import.meta.require. As shown by @jkrems above, it can be used as a workaround and may require wrapping in other modules; however, it does introduce a large migration behavior of relying on CJS mechanisms that won't work on the web rather than allowing the migration paths described in other places. I am unclear on what actual value it provides vs just allowing imports of non-ESM module types. I see no value, and instead see major problems with exposing it. Topics like how to get people to purely use import which may have a different resolution algorithm and even cache algorithm from require become much harder if you don't have a single migration path towards one or the other. As it stands, without guaranteed ordering I would not suggest moving a large codebase to ESM because I don't think the benefits are worth. Wrapping modules and using different syntax/APIs depending on the module type being currently written/consumed is just not a beneficial path to me.

I also only see import.meta.require as a problematic API versus alternatives, including just exposing the ability to create a require function. If we put it into the migration strategy we are actively telling people to use APIs that are not web compatible. The ability to import non-ESM is not a compatibility concern and has well defined migration strategies to keep all new code using import, and moving towards ESM.

I am unswayed by the claims of web compatibility problems with importing non-ESM, especially in the face of browsers wanting to include non-ESM such as HTML modules. The arguments about compatibility have not made direct claims about how the ability to import CJS directly prevents support for some feature on the browser platform. All claims have been around codebases being unable to run in all platforms, however that claim is easily disproven if the codebase entirely is using ESM. Once browsers encourage non-supported formats such as HTML the same claim would be applied that their module system is no longer compatible with Node, but we can once again easily disprove this by creating applications that do not use HTML modules. What is the claim for compatibility problems in the face of the issues with not supporting importing CJS above? The claim that a codebase does not work on a platform is a codebase compatibility concern, please describe the platform compatibility concern.

@jkrems
Copy link
Contributor

jkrems commented Jun 28, 2018

The ability to import non-ESM is not a compatibility concern and has well defined migration strategies to keep all new code using import, and moving towards ESM.

I'm not sure I follow. import 'some-thing-that-happens-to-be-cjs' is just as incompatible with the web as import.meta.require('the-same-string'). One difference is that the latter makes it immediately obvious and sounds the alarm bells ("this files will not work in a browser").

@bmeck
Copy link
Member

bmeck commented Jun 28, 2018

@jkrems import 'some-thing-that-happens-to-be-cjs' can migrate to be ESM instead of CJS without causing consumers to break. That code itself is not incompatible but the underlying implementation of 'some-thing-that-happens-to-be-cjs' is. That is why I am saying it is a codebase incompatibility and absolutely not a platform incompatibility. The platform does not prevent you from writing code that is compatible in all environments, the code provided by the application is what is providing incompatible code. However, import.meta.require does not allow the same form of migration to occur. Per your claim that one is more obvious than the other, we can see errors from parsing and most likely also from linking when you import non-ESM, it has the same affect of showing errors in both those cases.

@ljharb
Copy link
Member

ljharb commented Jun 28, 2018

I really don’t understand why we’re seeing implementation proposals before we’ve finished discussing transparent interop (all of its definitions) and defaults.

@zenparsing
Copy link

I agree with @ljharb here. It's good to see this work, and @WebReflection 's feedback is valuable, but I think the waters are still pretty muddy. I think we should come back to looking at implementation alternatives after we've done some more work on our framework for evaluating implementations.

@michael-ciniawsky
Copy link

IMHO createRequire 'vs.' import.meta.require should be discussed further

createRequire

import module from 'module'

const url = new URL(import.meta.url)
const require = createRequire(url)

const cjs = require('./main.js')

import.meta.require

import esm, { ns } from './module.js'

const { require } = import.meta

const cjs = require('./main.js')

createRequire includes a small extra step to get CJS working in within a module, which is a good thing as it makes CJS usage less convenient and explicilty indicates that CJS usage is mainly thought of as a migration step and usage should ideally be temporarily

I second @WebReflection that a flag node --module main.js is preferable over a file extension (.mjs) to determine the parse goal of the entrypoint, especially if 'transparent' interop is removed for now and besides packages (bare specifiers) the parse goal is known then (import/import() (Module (ESM))/require() ('Script' (CJS)))

+ 1 on enforcing extensions and stop importing directories
- 1 on __filename and __dirname for import.meta.require as those likely will never be supported by browsers (get use/support outside of node) and are trivial to code

How would the story for packages look like (without relying on file extensions) ?

{
  name: '@scope/pkg',
  version: '1.0.0',
  main: 'lib', // node ...
  module: 'src/index.js', // node --module ...
  scripts: {
    "build": "babel src -o lib" // [ '@babel/preset-env', { modules: true } ]
  }   
}

?

Besides enforcing extensions and stop directory importing, is there a anything from the package.json (npm/node) side of affiars that may improve package-name-map compliance further (at least in the bsic case) ? Things like a default entry name (index|main).js e.g

<script src="module.js" type="module" packages="path/to/node_modules">

module.js

import pkg from 'pkg'
// Resolving
script.packages + pkg.name + (index|main).js

— node_modules
|— pkg
| |— (main|index).js 

or the like...

@devsnek
Copy link
Member

devsnek commented Jun 28, 2018

a flag node --module main.js is preferable over a file extension (.mjs) to determine the parse goal of the entrypoint

the author of the file should always have first say over what kind of file they authored. node can provide things to override defaults but the default behaviour must always defer to the author of the file.

@jkrems
Copy link
Contributor

jkrems commented Jun 28, 2018

which is a good thing as it makes CJS usage less convenient

I'd be cautious with that line of thinking. Making anything less convenient should never be a goal. Because people are really good to find a path of least resistance. So if you make something less convenient, people will find a more convenient solution. And it's almost never what you actually wanted them to do.

@zenparsing
Copy link

@MylesBorins In general, I think this is a good direction. How does resolution work when importing a package specifier?

  • Does it look for the "main" key in package.json and load that file as ESM?
  • Does it still support index at the package root?

@zenparsing
Copy link

There appear to be some bugs around package specifiers; it seems to be possible to import from a directory if the directory is contained within a package path (e.g. import "pkg/folder").

Other than that, the general idea seems to be: you can only import from a directory if you are using a root-level package specifier, and in that case the normal package.json and index rules apply, with the exception that the target file is assumed to be ESM.

Using this approach, an author could dual publish by including both "index.js" and "index.mjs", or by having two similarly named files and using an extensionless entry in package.json:main.

I think I would simplify things: make it possible to import from any directory, but only use package.json and not index. It's not web-compatible of course, but that's OK. The simplicity of the rule offsets that downside.

The other issue here is that dual-publishing still forces authors to use .mjs, and it forces the user to "overload" main with an extensionless path.

The community has already converged on package.json:module. Is there any reason that node's package.json lookup rules could not simply use "module" instead of "main" when attempting to resolve an imported entry point?

As @WebReflection mentioned, I would also add an executable flag for specifying the entry point format.

Summary:

  • Allow importing from all directories, but only use package.json
  • Use package.json:module for specifying ESM entry points instead of "main"
  • Add a --module format flag to the executable

With those changes, we can easily support dual-mode packages for interoperability and we allow users to keep using the ".js" extension if they choose.

@MylesBorins
Copy link
Contributor Author

MylesBorins commented Jun 28, 2018

If anyone would like to push commits on top of this to fix bugs, extend / change implementation, or anything else please do. Follow up with a comment and if I have issues with what's pushed we can talk through it.

edit: oh yeah this isn't a PR... lol whooops. If you post a link to a commit sha I'll cherry pick and push first and ask questions later 😄

@guybedford
Copy link
Contributor

guybedford commented Jun 28, 2018 via email

@GeoffreyBooth
Copy link
Member

In #137 there was a lot of discussion of initial entry points. I think the consensus was that we need a CLI flag to handle the --eval and STDIN cases, where there’s no filename to cue Node as to the parse goal. @bmeck made several good points about being careful to design the flag to avoid breaking WASM and other future module types.

@WebReflection
Copy link
Contributor

WebReflection commented Jul 2, 2018

FWIW, since it's actually super trivial to have filename and dirname in both node.js and browsers, I agree exposing those anyhow is unnecessary.

const {
  pathname: filename,
  dirname = filename.replace(/\/[^/]*$/, '')
} = new URL(import.meta.url);

that's already a cross env KISS approach.

@zenparsing
Copy link

@michael-ciniawsky

There should definitely be a feedback loop of some sort for the community being able to voice their opinions about this.

Absolutely! Perhaps @MylesBorins could merge my commits and provide an updated download link? 😺

@WebReflection
Copy link
Contributor

WebReflection commented Jul 4, 2018

Your server may choose not to do any guessing

I don't want guessing in my server. guessing means crawling, mean degraded performance, means complexity.

Which should I guess first, file.wasm or file.mjs or file.js or file.json or file/index.js, or maybe file/main.js and friends ?

This whole thread is about an MVP for ESM in modules and guessing is not part of an MVP, IMO.

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

@WebReflection are you referring to the debug binaries for engines? I don't think we should be considering those at all... they just exist to test the engines.

@ljharb
Copy link
Member

ljharb commented Jul 4, 2018

Migration is a required part of an MVP, and imo that includes guessing. In other words, we disagree on what "minimum" and "viable" mean.

As for which you would guess first, that'd be easy - you'd follow the order of require.extensions, as does (and will) every resolution tool in the ecosystem.

@WebReflection
Copy link
Contributor

you'd follow the order of require.extensions

it's not even an Array and last one I have just checked is .node while I expect native extension (including .wasm) to have privileged meaning.

I also write ESM daily and I've never had any issue in specifying the bloody extension of a file name.

Bundlers solve that, good for them, but if you write ESM for real, you better write that extension or it wont natively work on browsers/server 'cause nobody writes overkilling rewrite rules for a missing .js, specially not CDNs so I don't understand what is your migration issue: bundlers users have no issues, pure ESM users write extensions already and also have no issues 🤷‍♂️

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

package maps and bundling fix that. if you can somehow invalidate the existence of those then you have a point. no one expects have code written in node.js can be verbatim copied into a browser and just work beyond toy code and polyfills. idiomatic node lets you drop the file extension because what a package is written in is an implementation detail. we shouldn't give that up for misinformed browser compatibility.

@weswigham
Copy link
Contributor

weswigham commented Jul 4, 2018

Which should I guess first, file.wasm or file.mjs or file.js or file.json or file/index.js, or maybe file/main.js and friends ?
This whole thread is about an MVP for ESM in modules and guessing is not part of an MVP, IMO.

I'm pretty sure the DirectoryIndex command in a .htaccess file configures that in apache. Not sure what the equivalent is in IIS. Express's serve-static has an option for it.

Implied extensions and a priority list therein are really common among webservers. You may have never reconfigured it yourself, but it's usually there.

Bundlers solve that, good for them, but if you write ESM for real, you better write that extension or it wont natively work on browsers/server 'cause nobody writes overkilling rewrite rules for a missing .js, specially not CDNs

unpkg is a fairly popular CDN which does have that style rewrite rule. So they do exist.

@ljharb
Copy link
Member

ljharb commented Jul 4, 2018

Also cdnjs

@WebReflection
Copy link
Contributor

package maps and ...

if you use those, you won't write fully qualified path up to the extension. If you have package maps you import "module", you don't import "./some/path/to/file" omitting only the extension, so I keep saying there's no need to guess ever here, and it feels like many in here use bundlers and are solving problems that don't exist with bundlers.

Anyway, if you think omitting the extension is not just useless overhead and unnecessary for this MVP go ahead, all I was saying as daily ESM user is that explicit file names are not an issue, and never will be.

In JSC and SpiderMonkey these are also mandatory and ESM shipped like a year ago ... that is what I call a MVP: it works, it doesn't have much philosophy or guessing around file paths, it shipped ahead of time.

@michael-ciniawsky
Copy link

if you use those, you won't write fully qualified path up to the extension.

You surely do write it, in package-name-map.json :)

package-name-map.json

{
  "path_prefix": "/node_modules",
  "packages": {
    "moment": { 
       "main": "src/moment.js" 
     },
    "lodash": { 
         "path": "lodash-es", 
         "main": "lodash.js" 
     },
     "wrong": {
         "main": "src" // <= No
     }
  }
}

https://github.com/domenic/package-name-maps#basic-url-mapping

Note how unlike some Node.js usages, we include the ending .js here. File extensions are required in browsers; unlike in Node, we do not have the luxury of trying multiple file extensions until we find a good match. Fortunately, including file extensions also works in Node.js; that is, if everyone uses file extensions for submodules, their code will work in both environments.

👌

@WebReflection
Copy link
Contributor

@michael-ciniawsky it seems like we agree, or at least, I agree with everything written in that quote, and also JSC and SpiderMonkey already work like that (fully qualified names).

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

JSC and SpiderMonkey already work like that

you keep saying this, but what does it mean. are you referring to the cli debuggers they maintain? the APIs built into the engines? I have no idea that that means.

@WebReflection
Copy link
Contributor

WebReflection commented Jul 4, 2018

echo 'print(123)' > module.js
echo 'import "./module.js";' > entry.js
jsc -m entry.js
# prints 123

you can try omitting part of the name (extensions are irrelevant, imports resolve the path though)

echo 'import "./module";' > entry.js

and see that jsc -m entry.js now produces an error

Exception: Error: Could not open file './module'.
fetch@[native code]
requestFetch@[native code]
requestInstantiate@[native code]
requestSatisfy@[native code]
[native code]
promiseReactionJob@[native code]

Both JSC and SpiderMonkey, and browsers, uses out-of-the-box resolvable names, without any guessing involved, neither folder or extension crawling.

This is the meaning of (KISS and) an MVP, imo.

edit

and browsers

meaning, they ask for what you are importing, they don't crawl the server, they don't guess extensions, they ask for a module with that name, end of the story.

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

so you are referring to the cli debuggers. those only exist to test functionality of the engines. i would request that you do not base any behaviour in node.js on those, as node isn't a debug utility - its a production runtime with millions of users. i'm also slightly upset about that as i'm currently working on some examples of module loading for V8/D8 and i would dislike if my work was misrepresented like this.

@WebReflection
Copy link
Contributor

@devsnek you are ignoring SpiderMonkey, powering GJS and others, where it's the exact same. I am telling you how engines that shipped ESM already work, and also how the browser work.

If you don't want to listen, it's not me misrepresenting anything, it's you not wanting to listen, sorry.

I think we have done with this discussion.

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

@WebReflection how GJS resolves stuff isn't specific to spidermonkey, but it is a better example of your point. i wish you had brought it up earlier :)

@zenparsing
Copy link

I think we've probably reached the end of this line. I think we can all agree that file-extension searching is a web/node interop issue. On the other hand, we need to balance that against the need to provide a smooth transition path to native ESM modules on node. We can debate how to balance those concerns later when we have more feedback.

@WebReflection
Copy link
Contributor

how GJS resolves stuff isn't specific to spidermonkey

GJS uses js52 here and js52 behaves exactly the same. Swap jsc with js52 in previous example, omit the extension, then:

Error: can't open ./module: No such file or directory
Stack:
  fetch@shell/ModuleLoader.js:12:16
  loadAndParse@shell/ModuleLoader.js:18:22
  @shell/ModuleLoader.js:29:47
  Reflect.Loader<@shell/ModuleLoader.js:25:9

TL;DR I've used jsc 'cause I was on mac, not because it was anyhow more or less relevant to my point.

Also browsers do the exact same, they don't guess, they don't crawl, they ask one thing only (that might be changed by the map but that's another story, I have nothing against the usage of the map)

the need to provide a smooth transition path to native ESM modules on node.

AFAIK there is not much transition to smooth out since std esm already does that. This is what I mean when I say most developers here use bundlers or esm resolver and there's nothing to smooth out for them 'cause both bundlers and esm can guide developers to migrate start deprecating or warning in console without breaking.

Tools should be there to help migrating, following what Node.js ships.

This shouldn't be vice-versa, otherwise Node.js will be always doomed by tooling adoptions.

Sure thing though, we need feedbacks, but I am worried people that use tooling will create problems that don't really exists (and they'll keep using tooling anyway).

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

@WebReflection they don't use the cli debugger though... they're using their own implementation of resolution and evaluation which you can read through here: https://gitlab.gnome.org/GNOME/gjs/blob/master/gjs/module.cpp

edit: i'm not even sure they're using ESM... if anyone is familiar with mozjs's api please let me know

@WebReflection
Copy link
Contributor

WebReflection commented Jul 4, 2018

i'm not even sure they're using ESM

it was brought up already months ago in the mailing list, which is why I keep saying we should keep it simple. GJS has live bindings but a completely different module system which, AFAIK, is going to be integrated with ESM.

Right now though, the import is limited and it will always add a .js extension, which is why it'd be lovely to keep it simple and have ESM with explicit extensions and let loaders and package maps handle all other cases.

MVP of ESM: explicit extensions, never ambiguous, faster to ship, less to discuss. You have loaders to solve everything else. How cool is that? 🎉


actually: loaders can help smoothing out migration !!!

@devsnek
Copy link
Member

devsnek commented Jul 4, 2018

MVP of ESM: explicit extensions, never ambiguous, faster to ship, less to discuss. You have loaders to solve everything else. How cool is that? 🎉

super cool if we didn't have 750,000 cjs modules that are still first-class citizens in the node ecosystem.

if everyone has to use a loader (and they will have to use a loader) then why not make it the default...

@WebReflection
Copy link
Contributor

WebReflection commented Jul 4, 2018

super cool if we didn't have 750,000 cjs modules that are still first-class citizens in the node ecosystem.

those can be brought in via import.meta.require which works like a charm in here already. Have you tried this MVP? I suggest you do.

However, loaders can solve that too 🎉

if everyone has to use a loader (and they will have to use a loader) then why not make it the default...

Nobody has to use a loader here. I've tested this and I didn't need any loader. Also my code wouldn't need any loader. Package maps, require when needed, pure ESM straight forward for everything else.

Who's using ESM today is already behind bundlers or loaders, so they don't have to change anything 🎉

MVP

A minimum viable product (MVP) is a product with just enough features to satisfy early customers, and to provide feedback for future product development.

I see it this way:

  • is the extension guessing mandatory to ship this? No.
  • can everything related to the magic guessing be implemented regardless from users? Yes.

That's it. But I also stop now, since like I've said there's not much else to add from me.

Regards

@MylesBorins
Copy link
Contributor Author

MylesBorins commented Jul 9, 2018

I've updated my MVP branch based on some comments in this thread (updating original post as well).

Changes include

  • fix to import.meta.require for node_modules as found by @zenparsing
  • no longer enforcing extensions. Changes have been made to the resolution algorithm to no longer resolve extensions or directories for local modules.

Repo: https://github.com/MylesBorins/node/tree/esm-kernel

Download: https://nodejs.org/download/test/v11.0.0-test2018070976df5841a1/

$ NVM_NODEJS_ORG_MIRROR=https://nodejs.org/download/test   nvm install v11.0.0-test2018070976df5841a1

@WebReflection
Copy link
Contributor

@MylesBorins it looks like you dropped the need for the extension but you haven't put in the option to enforce ESM so that now we have ambiguity.

./module not found by import in ./index.mjs. Legacy behavior in require() would have found it at ./module.js

If I start from index.mjs and import from ./module.js, which is a perfectly valid ESM file, everything is OK. If I now import from ./module which is a perfectly valid ESM file, node assumes the .js version is not good, but it works without issues when explicitly written.

It is also impossible to bootstrap ESM via node --experimental-modules -e 'import("index.mjs").catch(console.error)' because the --module or --esm flag is missing, meaning an import always fails in --eval at this point.

Considering I don't care about extensions, since I've described why it's a footgun to omit those but I like the moment I don't everything works as expected, would you be so kind to bring in the --module or --esm or --type=esm or --type=module flag to this MVP ?

Thanks.

@GeoffreyBooth
Copy link
Member

Besides the above reasons, we need a CLI flag to enable ESM mode so that code like this works:

node --module --eval 'import path from "path"; console.log(path.sep);'

@WebReflection
Copy link
Contributor

WebReflection commented Jul 10, 2018

FWIW, other CLIs use --module shortcut'd as -m to force-enable ESM over JS.

Since other extensions don't really need any enforcement and have no differences between ESM env or not (i.e. .json or .wasm or even .css), I wonder what's stopping node to add such flag.

Behavior

Everything evaluated as JS, or with a JS oriented extension (both .js and .mjs), is loaded as ESM unless import.meta.require is used.

@MylesBorins
Copy link
Contributor Author

Closing as this is quite similar to what we've ended up doing for the minimal kernel

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests