-
Notifications
You must be signed in to change notification settings - Fork 43
Feedback on extension resolution #323
Comments
Is there any information available on why file extension resolution was disabled? |
This comment has been minimized.
This comment has been minimized.
UPDATE: I think this is actually the biggest problem: #323 (comment) , #352 The below, while not super pleasant, could be worked around with smarter transpilers relatively easily. Before, it was possible to write ES modules (or TypeScript modules), and publish both CommonJS and ES modules with a simple module transformation via babel (I've actually started doing this already, so my modules can be treeshaken). Now we can't write: import {foo} from './bar';
console.log(foo); We have to write: import {foo} from './bar.mjs';
console.log(foo); Which means that now any transpiler also needs to rewrite the actual import path. I don't think any currently have this functionality, because until now this wasn't an issue. This is very different from how the ecosystem has worked so far. For example, TypeScript and Webpack both work just fine with the extension-less module resolution. I would guess rollup does too. I thought the previous design made much more sense. When I If I I get that some people don't like the new file extension, but I don't think it makes sense to create new problems and add ambiguity just to keep it. |
Some feedback in favor of not having extension searching on by default Twitter poll about compat: 65% of 1295 people favor Browser compat @jhnns on twitter: https://twitter.com/Jhnnns/status/1003201464716726272 @mhart on twitter: https://twitter.com/hichaelmart/status/1039529625100185600 Announcement tweet for PR had 150k impressions and no negative feedback Obviously none of this is scientific... but thought I could offer some balance to the feedback |
Again tho, enabling extension resolution does not in any way prevent “browser compat” - this is purely a question of whether you want the preferences of one group (browser compat folks) to oppress another (back compat folks) by making the feature off by default - because on by default causes no damage, but off by default does. |
Is browser compatibility really even possible? As soon as you import another module, isn't browser compatibility lost? import {something} from 'some-module';
console.log(something); In node, that would have to resolve to something in Am I missing something here? |
@AlexanderOMara Some browsers have shipped support for import maps which do make the code above work, assuming the appropriate import map is provided that tells the browser where |
@ljharb I disagree with your claim that turning it on by default has no undesirable outcomes. I also think we can avoid boxing people into categories and creating an us vs them narrative. We all care about our ecosystem and making a great developer experience. @AlexanderOMara the import map proposal Chrome platform status: https://www.chromestatus.com/feature/5315286962012160 |
I think it would help a lot if the proposed changes to the ecosystem were better fleshed out and documented.
I don't think the changes are just about browser-support vs backwards-compatibility. As I understand it, there would also be a loss of certain functionality, but potentially also a gain of new functionality. At this point, I'm honestly not sure which system I would prefer once all the pieces are in place, and currently I think it's a lot easier to see the negative implications of this change. |
Let’s be clear: we shipped automatic extension resolution in ESM. It’s As far as I can tell, to @AlexanderOMara’s point, the biggest consequence of having searching disabled by default is that public package authors can’t assume its availability—which is a big benefit, in the eyes of many in this group, as that encourages packages published to public registries to be cross-compatible with browsers by default (at least as far as resolution is concerned). That’s what would be lost if the default is flipped. To me, that benefit outweighs the cost of users needing to use a flag if they want this enabled in their projects.
They need to publish their packages with
None. Transpilers can add the ability to add extensions at compile time, so a specifier like
See above. They need to use a flag if they like this behavior, so that’s potentially a disadvantage. The advantage is that if they use any publicly published packages in a browser context, those packages are easier to work with in that they don’t require a build process or a specially configured server. |
They're already not cross-compatible with browsers because they use bare imports. import maps work the same way with bare imports as with extensions. In other words, unless we ban bare specifiers altogether, "work in browsers without extra transpilation or building" either is a) identically true with or without default extension resolution, or b) is identically false with or without it. |
Suppose I'm making a module that's explicitly node-only. What do I have to do to load a native module? Do users have to add a flag just to use it? Suppose I'm making a module that has a WASM component. Does that mean using node-only functionality, or is there a way to do that in a browser-compatible way? |
In general the ability to interpret different formats are not affected by resolution, so thats out of scope of this particular issue. However, the list of supported module formats could differ between Node and the browser and both have ways to intercept requests and translate to a different supported format (though I doubt the cost of doing so in the browser is worth it).
WASM requires all APIs be passed in via |
Re import maps, see https://github.com/WICG/import-maps#extension-less-imports:
|
I had thought the ability to import things that aren't JS modules was part of this issue. Maybe I was mistaken. |
@GeoffreyBooth sure. and i think it's a mistake that the import maps proposal isn't yet providing a more flexible means to handle that use case - but that's not something that should make node constrain itself. |
@ljharb I do not see this as a constraint, but rather a useful step in iterating so that we can ensure the end result of our process is what we desire step by step. I think going conservative with our ecosystem forwards compatibility is better than diverging for backwards compatibility. This is an issue about feedback but saying we are being constrained makes it sound like feedback about why extension resolution is useful wouldn't be accepted. We can iterate and improve our modules implementation over time :). |
@AlexanderOMara importing things that are not js is not the issue. We have experimental support for JSON and a PR is open for WASM. We are also doing work with browser vendors and the wasm spec authors to standardize the same modules for browsers 🎉. The bigger issue is that browsers will never do multiple network calls to resolve a file extension, so having specifiers throughout source text without file extensions creates a universe of "node only" code. It could be transpiled, but why should it have to be? Yes, people can always choose to write code with a subset if they wish to, but my feelings on the matter is that we will over time have a more stable cross platform ecosystem if we minimize the places where things diverge. If you choose to use Node.js apis, you will find yourself in a place where your diverge... but it is possible to polyfill those specific APIS (see browserify). |
@bmeck because of extensionless flies and type module, I’m not convinced that it won’t be a breaking change to add default extension resolution later. The module system is not like other things; it can’t really be iterated on over time. Just like cjs, whatever we initially ship may be effectively the entirety of the system for the foreseeable future. |
If this is considered breaking, it just means people could opt-in via a flag, like package.json or w/e. However, we can be more concrete here with an example. The addition of an extension-less file in a place that it collides requires an invalid specifier:
Which would error without resolution. Moving from error to non-error seems ok to me and often is not considered breaking when adding features (particularly when some feature is missing like an API).
I disagree, CJS has evolved (slowly) over time. ESM can do the same. Whatever initially ships might want to setup forward compatibility paths if desirable, but has no clear reason it is unable to match CJS' evolution over time. Part of the problem with CJS is the outstanding number of features and dynamic behavior that it exposes publicly, and we are not looking to duplicate that to my knowledge in the ESM implementation. If there are clear paths we want to reserve, we can do so; however, without clear reasons conservative iteration seems a good path forward. |
@weswigham my comment wasn't entirely focused on you but more to anyone here who feels they have been sidelined in this process. Specifically what I take issue with is the claim of being left powerless while at the same time taking no action or making no contributions towards the goals of the group. It would have been far more efficient and less of a waste of time for this not to have been an open process. This was very specifically was a fully open process though, which is an absolutely tremendous thing to have accomplished. I don't think pursuing any directions, even when against what might seem like a majority, would ever lead to lost social capital here. I'm personally glad you've been part of the process to date and that despite your frustrations you have remained in attendance. I'm also really sorry if you feel that the process hasn't met your ideals for what you were looking to accomplish. The things that likely to lead to lost social capital are unprofessional tone, personal attacks, attempts at strong-arming or being unable to compromise on deadlocks. Blaming the process is certainly not a good look though, and if there's a specific problem with the process - then work to change that in the process! If you really want extension searching, the onus is on you to create a proposal and add an agenda item. The ideal timeframe for that would have been a few months ago, so that this point the bar is very high indeed to get such changes through. It would likely have a few blockers (yes likely including myself), but nothing a vote couldn't ultimately overrule with enough consensus building between those who care about it. Extension searching has always very clearly been a feature that would require a vote either way, so if you want to bring such a thing back to the table now is the last call certainly. |
I think what @weswigham rightfully called out is that by keeping extension searching out of the minimal implementation, there was a de-facto bias towards not having it. So there were two positions in the working group: Having browser-like behavior and having What would be fair, I think, and also consistent with past discussions (IIRC), would be that the |
Of note, and called out way upthread, browser-like specifier behavior is pretty much out the window at this point, what with export/import map support. The only leg left standing in this area is "the complexity of tools designed to translate from |
I meant it as a short-hand for relative and absolute ("non-bare") URLs, specifically. There are currently no rules for resolving bare specifiers in the browser, so that's kind of out of scope (import maps aren't a final standard yet). And for relative and absolute ("non-bare") URLs, it's very much exactly what the browser does. Unless I'm missing some piece there? |
The browser, absent import maps, doesn’t support bare specifiers at all, only URLs - and node’s resolution only overlaps with data URLs, and arguably relative file URLs (which are really paths, not URLs, despite file: being a URL protocol). I very much agree that “browser-like” is never going to be fully obtainable unless a more fully featured proposal than the current import maps proposal ends up landing in browsers; and i strongly agree with wes’ comments above. What should have required consensus was deviating from require, and it’s highly unfortunate that those of us advocating for require-like behavior got put into a position where we were scary bad blockers, and where those advocating for no extension lookup were the “good guys” who just want to see ESM shipped. This issue definitely shouldn’t be closed. |
The distinction doesn't matter as far as what tooling is needed to get an arbitrary Additionally, I keep saying this, but it's not specified in the browser for relative specifiers - the browser just fetches whatever resource is provided at the URL by the webserver, extension or no (and provides an |
Like, node requiring extensions on relative imports is strictly less capable than what an arbitrary browser/webserver combo is capable of! |
It seems to me that the real failure here wasn't in our inability to reach a consensus on this issue, as it's clear that there are diverging opinions that simply won't be reconciled; our failure was in not coming up with a definitive way to resolve this question. We created the Based on comments on this thread and elsewhere, and based on the one vote we took regarding unflagging without automatic extension resolution a few years ago, it seems clear to me that if we had taken this to a vote, the current extensions-required implementation would have prevailed in a vote within the modules group. And the lack of a deluge of issues on the Node repo implies to me that the user base isn't too upset by the current implementation. So even if we didn't get here through a process we can all be happy with, it seems to me that the extensions-required implementation is the preference of both a majority of the modules group and of the Node-using public. I think what would make me feel better about this conclusion, and would hopefully help the automatic-resolution proponents better accept it, would be more concrete indicators from our stakeholders (package authors, tool authors, general developers, etc.) that they agreed. The survey that @SMotaal tried to put together had the potential to give us this broader data, but that effort fell apart due to infighting within the group over what questions the survey should contain (because even deciding what to ask the public influences what answers you'll get). Ultimately this speaks to me of the dysfunction of this group: we can't even agree on a way to settle our disagreements. So I guess where do we go from here? I think if the automatic-extension proponents are still interested in pursuing this, we can try to find consensus on a way to settle this question definitively, such as through some kind of democratic process like a survey of users or a vote of the modules group or of all Node contributors or all Node groups. Speaking for myself, that's what I would need to convince me that I'm wrong and that this should be changed. Likewise, if such a vote or survey reaffirms the current behavior, I would expect the automatic side to accept the election results and move on. And it's not for nothing that the situation today is much better than it was when this issue was opened. Not only does |
So, if you want feedback, I still get weekly pings asking me to modify TS to paper over the lack of extension searching, and to "automatically append" |
I've been following that specific issue for a while, and I also expected |
given that this is not a correctness issue, I think it is somewhat inappropriate to take a prescriptive approach to this. if you have some moral objection to code written for nodejs not working in browser when served with a static web server then you should convince people of that, not force it on them. |
As I've explained countless times, that's a leaky transform, which is why we won't do it. We do not, generally, override the runtime resolver (like webpack or parcel may do); what you see as your specifiers, is what you get in your output, and we're smart enough to know that a reference to We're not even being asked to support the "new way" of resolving esm specifiers, because we already do, because it's a subset of the old way (sans exports)! People are asking us to be able to write |
@weswigham I fully understand the reasoning and I'm not by any means wanting to inform TypeScript what to do. I'm simply stating my opinion that seems to be shared by others that this behaviour could be seen as unintuitive and there are possible alternative designs that could be seen to be more intuitive. For example, treating relative specifiers only (as defined in HTML) as permitting extension mapping through configuration could be one mechanism. The reason I'm stating this opinion is only to counter your assertion that Node.js put TypeScript in this situation, when we must look at the fact that TypeScript is making design decisions here, and the blame cannot be directed to Node.js, when making those design decisions without adequate user education is the underlying reason for confusion. |
We are refraining from designing something to cover up what the runtime does not do, but users still want. That is very different than designing something in conflict with the runtime. The runtime doesn't support any kind of extension mapping thing, nor does it have a great need to. We do not want to be extending the resolver at compile time. |
I must still admit I don't understand why. Also, isn't |
Also, in theory a file mapping is not a resolver extension it is a file system remapping. |
Let it be known that people who do not use typescript also dislike the lack of extension resolution. |
No, we reimplement the resolver wholesale; we run in browser runtimes where node's resolver isn't available, but we'll still analyze code intended for |
Putting these two arguments aside, the main argument to consider is that there is a file system mapping happening. The module at |
Hold it there; those tools bundle their own resolver into the bundle. They no longer care about extensions at all - webpack even goes so far as just just use IDs to refer to each module anywhere it's precalculated the links. Point is, we don't bundle a big runtime like that. |
I specifically did not mention Webpack - I'm referring to code splitting outputs from RollupJS and esbuild, which only rely on relative specifiers to exact file extensions as the basic primitive (with externals of course). |
|
Because it is safe to say that nobody use native ESM in node. None of my projects at work or private can be converted. Most of the tooling lack proper support: berry, typescript, webpack, eslint, jest. Support is either partial or would require massive amount of work with little to no benefit. Lots of people have transpillers in theirs toolchain that compile ESM to CommonJS and are by large unaware. That why you do not see a lot of feedback. It is more visible in corresponding tools repositories where people are harassing maintainers. I think many people like me just wait for things to "Just work". I am observing ESM progress in node for the last 3 years now from initial work in node-eps. This is moving incredibly slowly because of changes in resolution. You cannot expect adoption if a an migration is to change every relative import statement and every index.js. That why sooner or later all tools will be forced to support re-writing paths. node.js was so late in the ESM party but want to change everybody assumptions about resolution. |
I'm sorry, but I don't find any of these arguments persuasive; nor do I find this ongoing debate a productive use of our time. @ljharb at least opened this issue to try to collect feedback from across the web although it's been hijacked; and @rauschma opened a Twitter poll: Excluding the “just show me the results” folks, that’s 39% for mandatory extensions, 30% for automatic resolution, and 31% that don’t care; from a sample of 294 votes. I think efforts like this poll are the way forward to try to build support for any potential change, if one is desired. |
people who prefer providing extensions can provide them regardless of what the default here is. what i see from that poll is 30% of devs saying they miss a functionality, that's pretty huge. |
Seriously; if 30% of your users said they wanted something that didn't affect what the rest of your users could do, I don't know why you wouldn't consider it... |
Again, another poll that is just misleading. Workaround only applies to external dependencies. Not only you suggested workaround is bad because lead to fragmentation but do not address core issue. As result your node.js application written in typescript can end up with something like this import module form 'main-module-package'; // with single main
import cjsSubmodule from 'main-module-cjs/submodule' //cjs modules still search for extension ?
import submoduleMapping from 'es-module-package/submodule'; // with subpath mapping
import submoduleNoMapping from 'es-module-package/submodule.js'; // without subpath mapping
import base form '../base/index.js' // index.js as no longer supported
import local from './local-module.js' // relative imports need to have extension Now in my typescript code I need to know how modules can be imported. For relative imports I need to import using transpilled extension and for existing code I need to refactor or import index.js. |
It seems like a good idea to capture somewhere the feedback we receive on extension resolution.
The text was updated successfully, but these errors were encountered: