Support .mjs output
#18442
Comments
|
@kitsonk |
|
Basically those say that TypeScript does not want to concern itself with managing extensions like that, as it is already complicated enough. In particular the output extensions run afoul of the TypeScript non-goal of:
If you want |
DanielRosenwasser
added
In Discussion
|
Thank you very much for clarifying your comment. The long term goal of the Node team is to allow users to author their code using ES modules without paying a cost due to the commonJs modules and script target. This is one of the reasons why the proposals to require Complex build tools have their place for assets management, dead-code elimination, bundling, minification, etc. These all fall out of scope for TS, but being able to produce runnable code does not. Big projects have their own workflows and use the TS library (either directly or through plugins for other task runners), adding the renaming step is not a big deal for them. On the other hand, a sizeable chunk of projects using Typescript are small/medium sized libraries. These libraries usually only need a single tool: It raises the barrier to entry because newcomers can no longer simply use So far, most of my arguments were about the build complexity: having it done once in TS is better than having each project coming up with its own solution. There is another important reason to have ES modules working out of the box, and I think that it aligns with the goals of TS:
ES modules are part of the spec and the TS syntax uses them. These modules have a specified runtime behaviour that cannot be fully replicated in commonJs. It affects among other things circular dependencies, early errors, mutation of exported namespace, etc. Many people won't care, until it bites them. When setting the One last argument is that even if browsers support native ES modules without requiring To summarize, here are my main points:
|
|
Just an Opinion As a very strong fan of TypeScript, and obviously a user of node (I guess even TypeScript is) I would really like to see some agreement between them on an issue that has had so much disagreements over many years of rational and sometimes awkward discussions. I would have loved it if node had a transition phase to make common js (the non-standard) become .cjs over a period of two years. That said, mjs seems to be the current direction, obviously that non-js file extension (either one) itself does not matter, but the ability for TypeScript to provide a way to control this inline as the originator of the transpiled files is something that really falls to TypeScript. That said (again), mjs as a locked extension is kind of very opinionated, someone must have realized that forcing js was okay, it was already called js, but forcing mjs, come on, even my spelling checker is nagging me on that one. I hope nodejs will become a little more flexible on that one. Proposal Please make it possible to specify mjs as a module format that simply means compiling es2015 modules and calling them mjs :) |
|
It's important to note here that If I'm understanding this right, renaming the files isn't a solution, TS would also need to compile module specifiers with the |
|
TypeScript already allows you to include a |
|
It's not about the source files including the extension, it's about the compiled files. If I compile var x = require("./x");If I compile it to ES2015/ESModules with an import x from "./x.mjs"If a library is trying to support both old and new node with CommonJS and ESM side-by-side (as |
|
There are some potential updates in the works… including a --loader option that would you to specify a file with a resolve hook that will take the specifier and parent module path, then it could handle extension prioritization as needed. It seems to be very flexible but their goal is to make it declarative and not have people overloading the actual loader functions. In terms of picking mjs over js, that is only the case in the CJS loader system which uses the now legacy Module prototype with it's hooks, the ones that everyone likes override all the time. In the current --experimental-modules release, the ESM loader is hard coded to '.mjs' or falls back to the CJS loader. What I've seen so far over the past few days makes me believe that the ESM loading system will take over with pluggable CJS loading (unless opting to use the legacy-modules by flag or based on the main file). |
|
@trxcllnt I guess regarding the part about baking the extension right into the import… this applies to import/export everywhere now as part of loader specs. When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks. So if TypeScript would continue to work without third-party tooling, it will need to find a strategy to write out standard (not ISO but platform-specific) out-of-the-"compiler"-box projects that will just work in either node or browsers at least depending on the compilerOptions intent specified by the user, and do so without asking the user to mockup some hack to get it to work. This is just an opinion, but honestly, it feels like it for TypeScript to address. Checkout this MDN reference but make sure you notice the part where it first said "excluding the .js extension" then in the examples included the extension anyway. |
|
I'd like to include here that the IANA has an Internet Draft which specifically adds @daflair can you expand on what
means? The next step after that PR is to setup per package loaders to be able to guard against global loader mutation. So, it might be going the direction you think.
Interestingly the WHATWG Loader Spec had a very early version in the ECMAScript spec that was removed at the last minute before ES2015! There are interesting other things like the original CommonJS Spec which mandated not to have extensions. We should probably avoid dwelling on the past so much since the different loaders vary so much on these opinions. Node's EP specced a superset of the WHATWG resolve algorithm that does do various Node idioms like file extension completion. However, the browser has a subset that is safe to use in Node. Still, even with that subset there are problems with dependency trees since "bare" imports are waiting on userland feedback (you can get involved in the hook here), but is mostly left up to intelligent servers and service workers for now. I'd recommend trying to compile down to the WHATWG compatible specifiers except for bare specifiers for now. |
|
@bmeck what I meant by:
I was making an assumption that at some point you will have to use something like --disable-es-modules or --legacy-modules which is really an option that simply does the opposite of --experimental-modules (when this flag is the default behaviour) to resort the current loader system and completely bypass anything related to the new loader (ie legacy applications that simply find ways to be incompatible with this existential change to their eco system). But now that I dug in a little deeper in your recent PR's this might not be your intent… So it is best to ask you about your intent here 😉? |
|
@daflair I'm still not understanding. ESM support in |
|
Okey, when we get to the point where --experimental-modules is no longer needed, at that point, will the CJS loading system that exists today still remain (mostly) unchanged? If so, at that point, without the above flag and without any other flags like --loader, …etc : a. Would running with a .mjs entry do what it does today with the flag and the CJS loader will simply pass it to the ESM loader? meaning that all dependencies will be marshalled by the ESM loader (even if it delegates something to the CJS loader). b. Would running with a .js entry always imply CJS (again no special flags) and this also means that ESM loader is essentially completely idle for the lifespan of the process (assuming there will be no require() support for es-modules in the future). |
Yes, it is unchanged except the
No, the 2 loaders are decoupled except when ESM defers to CJS. Loading via
Yes |
mhegazy
added
the
Domain: ES Modules
|
And people wonder why the TypeScript team wants to avoid getting into this area at the moment... 🙄 No it is, yes it isn't, no it is Even if TypeScript gets further into an extension mangling business it would wait until it was clear that |
can you clarify what is unclear in the messaging from Node? |
|
@bmeck so one thing that seems like a major foot-gun is how node doesn't automatically select the // node_modules/ix/Ix.mjs (no default export)
export class Iterable {};// node_modules/ix/Ix.js:
module.exports = function Iterable() {};// run with node --experimental-modules some_file.mjs
// works fine
import { default as Ix, Iterable } from 'ix/Ix.mjs';
assert(typeof Ix == 'undefined') // true, great
assert(typeof Iterable == 'function') // true, great
// broken
import { default as Ix, Iterable } from 'ix';
assert(typeof Ix == 'undefined') // false?
assert(typeof Iterable == 'function') // false :( |
|
@trxcllnt it is complete and has been laid out in depth in https://github.com/nodejs/node-eps/blob/master/002-es-modules.md
no.
Correct, because this code cannot be spec compliant. There is absolutely no way to link it ahead of time since you need to eval to get the shape of This style of eval based shape detection will never be possible. In the future there may be a pragma that allows parse time instead of eval time declaration of shape, but that would still be a breaking change. Simply put:
Is only true because those environments are not valid ESM |
|
@bmeck I'm afraid I wasn't clear. I'm not saying node shouldn't use the exports from a CommonJS module as the default when I'm saying that when I run with I have this expectation because I'm explicitly running node in ESModules mode, and using an ESModule import statement to import a package that exports an ESModule. But instead because node selects the CommonJS version (and then exports becomes default export), the code doesn't do what I expected it to do. And to clarify why this matters, we're compiling es2015+ features (generators, async generators) down to ES5/CommonJS as the |
|
@trxcllnt it does search for Your specifiers are different in meaning and you can see if you use |
|
@bmeck {
"name": "ix",
"main": "Ix.js",
"module": "Ix.mjs"
}We can't set |
|
@bmeck and also I want to clarify, you and I are on the same side of this issue. Considering the prevalence of importing libraries from node_modules in node, I was a bit surprised there wasn't clarity on this issue for ESModules. And also that it also did a bad/unexpected thing when we're explicitly trying to support both CJS and ESM in the IxJS project. I would even have preferred node to throw/print some sort of "ambiguous import" warning like "hey dummy, you tried to get an ESModule from a package.json file which doesn't have a way of specifying ESModule mains, here's the ES5 version but maybe be more explicit next time" |
|
Remove the extension from "main". Just "Ix"
On Oct 3, 2017 10:00 PM, "Paul Taylor" <notifications@github.com> wrote:
@bmeck <https://github.com/bmeck> and also I want to clarify, you and I are
on the same side of this issue. .mjs is just fine with me, as long as
things have reasonably normal default behavior. I'm in this thread arguing
that TS *should* generate imports with .mjs extensions, because it seems
all ESModule import specifiers are now spec'd to be valid URLs.
Considering the prevalence of importing things from node_modules in node, I
was a bit surprised there wasn't clarity on this issue for ESModules, and
it also did a bad/unexpected thing when we're explicitly trying to support
both CJS and ESM.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#18442 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAOUo4SRH5PQaknAohmUHU330-YDy7-mks5sovTkgaJpZM4PWbkF>
.
|
|
@bmeck HMMM yeah that sounds like a solution. sorry for hijacking the thread everybody. |
|
Awesome @trxcllnt and @bmeck, I guess we are all really trying to figure things out, so a resolution is always a great addition to a thread. By the way, many of us nagging at Node to support ESM with .js are also nagging at TypeScript to support "module": "mjs" that can run with node directly without a --loader. Different workflows, different requirements. So here is a silver lining: // hybrid package.json
{
…
"main": "index",
"files": [
"index.js",
"index.mjs",
"index.d.ts"
"index.es.or.whatever"
],
…
}Then if I do my work right, I would really appreciate being able to add this: // hybrid package.json
{
…
"scripts": {
"compile-cjs": "tsc -m commonjs",
"compile-mjs": "tsc -m mjs",
"compile-whatever": "boink plug and play box yes no okay --but-follow-es-standards-please",
"compile-all": "npm-run-all compile-cjs compile-mjs …"
},
…
}Instead of having to do this: …
"compile-mjs": "tsc; rename …; find-replace … from /.*?/ with $1.mjs --verbose"
…Does that sound about right? @bmeck so those index variants will resolve to mjs if imported and js (cjs) if required when things are stable without any special flags, right? 👍 Let's keep at it! |
|
adding 'they should use a bundler' is probably the practical answer, but there's a space between 'massive package' and 'single file package' that is underserved by that answer. |
Node is the only platform which cares about using the mjs (or cjs) extension (optionally). For anywhere else, just using .js is fine. In node, you should almost never compile your package into both formats; this causes two copies of your package (potentially) to get loaded into node at runtime, which, in addition to being performance inefficient, can cause issues with state, class nominality, and unique symbols. Instead, in node, you should really just provide a format-specific convenience wrapper around a single source-of-truth package format (eg, a handwritten esm wrapper that imports the cjs core). Ergo, you really shouldn't need to compile the same source to both a Worst case, if you really really badly want to make node load two full, separate copies of your library, you could always compile once into a |
|
Hm, if you can just have a package.json in a dist-esm folder with just |
|
@jamie-pate thats exactly my observation i am researcher in that fild and member of the nodejs package-maintainance group. We Really did break evolution at some points. But the Good thing is when everything would be offered ESM Only with .js extension even without a package.json we would make the whole ecosystem healty again. when using NPM packages you should dist only one flavor -esm -cjs -webpack everything a own package and consider even sharing packages the esm code for example could when it is a really old package you could start offering a ESM package that loads and depends on the CJS package do not even transpil that to ESM you gain nothing from that. to load a single file CJS package inside ESM it would be enough to know that both use the same globals so we can shim like this const resetLoader = () => {
globalThis.exports = {};
globalThis.module = { exports: globalThis.exports };
}
resetLoader()
await import('singleFile.cjs') // i do not want to over complicate it
const myCjsModule = globalThis.module.exports;
resetLoader()there is ongoing work on the import.meta.resolve feature that will complet the above shim with resolving capabilitys |
|
Trying to give a practical example here: I have been working on a deep dependency in our code that has been written in CommonJs a long time ago. I used it as a test to see if I could update it to be a package that is written in Typescript and supports both to be required as cjs (to not break when I drop it in the greater application) and to offer esm for the whole system to be step-by-step updated to ESM-only code. → @tradle/typeforce My plan was to use In the In order to be able to test the generated files I wanted to be able to load them directly in the CLI.
Of course in typescript I can not change The initial tests were written with To make sure that all code works same I originally I tried to put the tests in I think it would help quite a bit if typescript would build node esm modules with a |
@martinheidegger check out how https://github.com/jwalton/solox does this. The build process for solox is very similar to what you proposed with an esm tsconfig.json and a cjs tsconfig.json. These get used to compile the source into ./dist/esm and ./dist/cjs. There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains The result is that all files are ".js" files, but if you clone and build this project you'll find that My tests are in /test, but they import from ../src, and tests run fine out of the test folder using jest and ts-jest. (If you run into problems with tests, you might want to flip things so the tsconfig.json is commonjs, and have a tsconfig.esm.json that's Note that one potential downside of this approach (and your original approach) is that it's possible for projects to end up with two copies of your code in memory (if they both import your code and require it in different places). This approach should definitely be avoided if you have singleton instances or other global state. |
|
Thank you @jwalton for sharing notes.
Oh! Thank you for pointing this out. This will indeed make my solution a lot simpler. Wouldn't this be a good feature to add to the typescript compile process?
Actually, I also was under the requirement to support deep imports like In a simple experiment with solox (found here martinheidegger/solox@e0a5edd) I tried to add a simple dependency and ran into a catch-22 where lint would break without a |
This is a idiosyncracy of typescript, and it's not very intuative, but try making this change: -import foo from './test.ts';
+import foo from './test.js';(i.e. change the "ts" to a "js" in your import) and your experiment should build and run correctly. |
|
For deep imports in nodejs you can use the exports property of package.json "exports": {
"./*": "./dist/*.js",
"./*/*": "./dist/*/*.js"
},NodeJS does not recognize the "modules" property, only bundlers will recognize that. I'm using that with the same |
Interesting. This seems to be not yet supported by |
|
I use yarn add -D @swc/cli @swc/core
yarn run swc next.config.ts -o next.config.mjs -C module.type=es6 -C sourceMaps=inline |
@jwalton This proposal looks like a duplicate of probably the most insightful post on the topic to date cc @mobsense : https://www.sensedeep.com/blog/posts/2021/how-to-create-single-source-npm-module.html By the way, for those who may be interested in having a look at the rewrite strategy path, it also seems worth considering the 2 packages below : • tsukuru |
|
@gfortaine Excellent suggestion. By placing a |
…e), and CRA (webpack) (#38) **Investigation:** This PR attempts to fix the following error in alpha/sonic:  This was a result of adding `"type": "module"` to our esm bundle for our library to be compatible with vite/pagesJS. Vite, and bundlers such as webpack, follow node's module resolution algorithm. and NodeJS have certain rules on how it recognize files as ES modules (https://nodejs.org/api/packages.html#determining-module-system) -- which include adding `"type": "module"` as an option. The fix is to update all of our paths to be explicit in order to include the mjs/js extensions, including (e.g. `./components` to `./components/index.js` and `../icons/DualSync` to `../icons/DualSync.js`). This currently can't be done by typescript compiler (tsc) as it's strictly a nodejs behavior and they don't want to support that ([long issue thread here](microsoft/TypeScript#18442 (comment))), which is frustrating. So we would have to do it manually or have a script to update import/export paths in the final bundle generated by tsc. This is not ideal and can be error prone. **Solution:** I decided to **replace tsc with rollup to bundle our library and append .mjs extension to our final esm bundle**. Using `mjs` extension instead of `.js` also remove the need to do `"type": "module"` in our esm package.json, which previously introduce unnecessary caveats and one-off script to inject it into our esm bundle. NOTE: the default **interop** behavior for rollup is not compatible with alpha/sonic. As such, the rollup config in this PR uses `auto` to follow Typescript's `esModuleInterop` behavior in how it transpile named, default, and dynamic imports of external deps (the issue with alpha is related to how react-textarea-autosize was imported into ChatInput) NOTE: updated `tsconfig.json` to use `"jsx": "react"` instead of `"jsx": "react-jsx"` because of the way jsx-runtime is backported to react 16/17 without explicit exports field ([React github issue here](facebook/react#20235 (comment))). We would either need to export two separate bundles for latest and legacy versions in order to continue outputting the JSX syntactic sugar OR we could just output `React.createElement` directly. This is common for many React libraries that support older versions like React 16. (e.g. [ant design](https://github.com/ant-design/ant-design/blob/master/tsconfig.json#L15), [blueprint UI](https://github.com/palantir/blueprint/blob/develop/config/tsconfig.base.json#L9), [evergreen UI](https://github.com/segmentio/evergreen/blob/master/tsconfig.json#L6)) J=CLIP-520 TEST=manual see that the new build works with: - the local test-site of the component lib repo - pagejs/vite (released v0.6.0-alpha.38.6 to install and test) - sonic/alpha (released v0.6.0-alpha.38.6 to install and test)
and others
Experimental support for ES modules just landed in Node 8.5 (changelog, PR).
Since ES modules have some parsing and semantic differences, Node decided to use the
mjsextension for ES modules (whilejsis for the "script" target and commonjs modules).The current Typescript version (
2.5.2) supports ES modules emission but uses thejsextension by default. It means that to use it with Node, a post-compilation step is required to change the extension fromjstomjs. This adds undesirable complexity to use native ES modules support with Node.A solution would be to add a compiler option to output
*.mjsfiles when emitting ES modules.Edit (2018-03-22): The propositions below are a bit outdated. I recommend reading the issue to see the progression. See this comment for my current proposition.
Notes:
*.jsextension, many tools rely on thejsextension.*.mtsfiles would compile to*.mjs, this would be similar to*.tsxand*.jsx.The text was updated successfully, but these errors were encountered: