mem intellisense broken for JS

[sandersn]

This issue matches PR #30. I think it's better to have a discussion here.

I believe this issue applies to all your packages that recently added types via d.ts files. I assume that the purpose was to improve intellisense. Right now, it works fine for Typescript users who have --esModuleInterop turned on, but for normal JS use it's broken:

const mem = require('./mem')

isn't callable, even though the primary export of this package is a function.

Instead, people have to change their code to

const mem = require('./mem').default

which seems like it goes against the normal commonjs usage.

The d.ts file seems like it exists solely to provide better intellisense, and it consists solely of typescript syntax, so I think using the typescript syntax for commonjs exports make sense. That way people who use mem in editors with intellisense will not have to add .default to all their existing require('./mem') calls.

[sindresorhus]

I'm preparing for the future. I'm not interested in rewriting all my type definitions when I move my packages to ES2015 module syntax, so my type definitions assume it's module and I also export a .default to make it ES2015-module-like. I rather optimize the experience for TypeScript and ESM users. My type definitions are not intended for JS users, that's just a bonus that happens to work.

I don't see the problem with having to use .default. You have to do that already if you use any package compiled with Babel from ESM to CommonJS.

[weswigham]

With the current designs under consideration require("esm") isn't even going to work (you need to use dynamic import) - so what future is this preparing for?

[weswigham]

In chalk you have a .flow definition file that correctly declares the shape of the module as an export assigned thing, but the .d.ts file is likewise incorrect, as in mem. Why have intentionally inaccurate TypeScript metadata, but correct Flow metadata?

[weswigham]

I'm unsure how this is different than this issue about how the same problem used to exist in chalk's flow types, which did get fixed.

[sindresorhus]

When I switch my Node.js modules to ESM, I won't have to inconvenience TypeScript users to update their imports. If I use export = , TS users needs to import with import * as mem from 'mem'; now and later switch to import mem from 'mem'; (unless they use allowSyntheticDefaultImports). In the past, I got many issues of people not understanding why importing didn't work. I have used (faked) default imports for a long time now and you are the first to complain.

[sindresorhus]

With the current designs under consideration require("esm") isn't even going to work (you need to use dynamic import) - so what future is this preparing for?

The future where I don't use require at all.

[sindresorhus]

In chalk you have a .flow definition file that correctly declares the shape of the module as an export assigned thing, but the .d.ts file is likewise incorrect, as in mem. Why have intentionally inaccurate TypeScript metadata, but correct Flow metadata?

I didn't add the Flow declaration.

[sindresorhus]

// @BendingBender

[sandersn]

I think we were the first to notice the problem because the people most affected by the problem are JS users of VS Code, who don't even know they're using index.d.ts for type information. In JS mode, we don't report errors, so the difference between working and broken is Intellisense vs No Intellisense.

We detected it because @BendingBender is a major contributor to Definitely Typed and we took notice when he started removing packages instead of adding them. :)

[BendingBender]

I think what @weswigham tries to say here is that the current state of affairs according importing CJS modules from ESM modules is that the whole module.exports definition is made available as a default import. So for example, if we have a CJS module like this:

const myModule = {prop: 'foo'};

module.exports = myModule;
module.exports.default = myModule;

then in an ESM module, this would look like this:

import myModule from 'my-module';

console.log(myModule.prop);
// => 'foo'
console.log(myModule.default.prop);
// => 'foo'

There is, however, a note that named imports should be supported when importing CJS modules from ESM modules.

[BendingBender]

@sandersn Intellisense interop was not on my radar. I started upstreaming the definitions because it became a burden to keep them up-to-date regarding the pace in which @sindresorhus' modules keep changing.

[weswigham]

When I switch my Node.js modules to ESM, I won't have to inconvenience TypeScript users to update their imports. If I use export = , TS users needs to import with import * as mem from 'mem'; now and later switch to import mem from 'mem'; (unless they use allowSyntheticDefaultImports). In the past, I got many issues of people not understanding why importing didn't work. I have used (faked) default imports for a long time now and you are the first to complain.

Having the default in addition to the namespace assignment should be sufficient here. Generally people will need to be swapping from import mem = require('mem'); to import mem from 'mem'; in such a future, and we have a compilation mode (esModuleInterop with the module: "es6" flag set) where only the default import style will end up working, so for users who care about that kind of future compat, the compiler can check it.

But even that has problems, and not with TS. If mem and mem's consumers swap to esm at the same time, you'd be fine... But if a user updates to esm first (or depends on an older mem), then the mem import would be backed by cjs, and instead of getting a callable default you'd get the entire module as the default. So you'd need to say import mem from 'mem'; mem.default() for it to actually work (without a namespace assignment). The default-default is something I hope I don't actually need to see in shipped code, but is ultimately the outcome of a preemtive cjs-namespace-assignment-with-default-but-only-reports-a-default declaration.

When I switch my Node.js modules to ESM, I won't have to inconvenience TypeScript users to update their imports

You're going to inconvenience your TS users less than everyone else, since all your cjs users, under current proposals, will just be completely broken without a reasonable way to use your module. At least your TS users will have the option of running a quick fix (or already be correct) on their import and upgrading their codebase - your cjs users are just dead under the current esm/cjs interop scheme (because replacing require with a promise-returning dynamic import isn't really a practical thing for most places). :(

[BendingBender]

With the current designs under consideration require("esm") isn't even going to work (you need to use dynamic import) - so what future is this preparing for?

As you said, current designs specify that using

(async () => {
	await import('esm');
})();

will be possible from CJS modules. Which will import such modules exposing them the way they are authored for ESM, mostly meaning that there will be a default property. Which is the way @sindresorhus' modules are exported in TS.

[weswigham]

As you said, current designs specify that using

You can't seriously update any large program using the pattern. For one, is forces you to defer all of your own module's exports because your module now has an async dependency, and you somehow need to report that (eg, by exporting a promise to the real exports namespace), and so the async taint transitively spreads to every dependency. If I were using, eg, pad-left and it updated to esm and required a sweeping change like that to use that would also force me to break all my consumers (since now all my modules need to export promises), I'd just not update.

[weswigham]

What I'm getting at here, in the abstract, is that there's no clean migration path in node's esm plan right now. Don't try to craft your declarations for that future, because it doesn't work as well as you think it does, is still in flux, and will only hurt your consumers today.

[BendingBender]

You can't seriously update any large program using the pattern.

Is there any kind of discussion on this topic? I did a quick search but I seem to miss it.

... instead of getting a callable default you'd get the entire module as the default. So you'd need to say import mem from 'mem'; mem.default() for it to actually work ...

Why wouldn't mem be directly callable provided @sindresorhus moves his modules to ESM, which he plans to do?

[BendingBender]

What I'm getting at here, in the abstract, is that there's no clean migration path in node's esm plan right now.

@sindresorhus I think this is a very valid point to consider. What do you think?

[weswigham]

Why wouldn't mem be directly callable provided @sindresorhus moves his modules to ESM, which he plans to do?

As I said, if both consumer and producer are esm, it'd work fine, but if mem is still cjs (eg, because a consumer's dependency is locked to this version), the default member of the default import is what's going to be needed to work (according to the type definitions).

Lemme be blunt with y'all: DO NOT WRITE DECLARATION FILES TODAY FOR TOMORROW'S NODE ESM IMPLEMENTATION.

1. How it works is still in flux. What you think works today might not tomorrow, or there may be a better way tomorrow.
2. Because of how it's likely to work at present, we (as in TS) will likely need to augment declaration files with some kind of "node-esm-only" marker (since it behaves very differently than the transpiled esm es syntax in a declaration file represents today, and we'll need to flag it's usage from commonjs). We'll be deciding how that works once everything interop wise is decided and it's about to unflag, which is optimistically October. So any declaration file written today will, ofc, not have that marker.
3. Declarations that don't accurately reflect (current) reality are dangerous. Ultimately that means incorrect intellisense results, and a drop in type coverage for a program.

[sindresorhus]

I just wish someone had told us all this a month ago before we made 100+ declaration files...

So let me just get this straight:

1. export default mem; has to be changed to export = mem;, right?
2. How do we do named exports using this syntax?
3. Can we continue to export interfaces? export interface CacheStorage {}. If not, what do we use?
4. Does the user have to use the awkward import mem = require('mem') syntax if they don't set allowSyntheticDefaultImports?

Anything else?

(Please look closely at the declaration file, I don't want another round of rewrites)

[BendingBender]

@sindresorhus

1. export default mem; has to be changed to export = mem;, right?

Correct.

2. How do we do named exports using this syntax?

3. Can we continue to export interfaces? export interface CacheStorage {}. If not, what do we use?

We have to use the following construct:

declare namespace foo {
	function namedExportFn(): void;
	// importable via: import {namedExportFn} from 'foo';

	interface Foo {}
	// importable via: import {Foo} from 'foo';
}

declare function foo(fooArg: foo.Foo): void;
// importable via: import foo = require('foo');
// or with allowSyntheticDefaultImports/esModuleInterop: import foo from 'foo';

export = foo;

4. Does the user have to use the awkward import mem = require('mem') syntax if they don't set allowSyntheticDefaultImports?

Yes, if they don't set either allowSyntheticDefaultImports or esModuleInterop. I've co-authored an FAQ section on DT on this topic.

[sindresorhus]

We have to use the following construct:

How would it look like to import namedExportFn and Foo in TS? (Without setting allowSyntheticDefaultImports or esModuleInterop).

[BendingBender]

There are 2 options:

// 1st
import foo = require('foo');

const fooVal: foo.Foo = ...;
foo.namedExportFn();

// 2nd
import foo = require('foo');
import {namedExportFn, Foo} from 'foo';

If your default export is a class the whole definition becomes much more awkward:

declare namespace Foo {
	// Only interfaces may be declared here. If you have a value,
	// you'll need to declare it as a static member on the class
	// or via an intersection type.
}

declare class Foo {}

export = Foo;

[BendingBender]

All in all, this is considered more of a hack than a solution because combining export = ... exports with namespace declarations leads to the effect that the following becomes possible even without setting allowSyntheticDefaultImports or esModuleInterop:

import * as foo from 'foo';
foo(); // shouldn't be primitive or callable as per ES spec

Only in case of very simple modules that don't have interfaces to export and thus where you can omit the namespace declaration will the compiler enforce the import foo = require('foo') kind of imports.

[weswigham]

Only in case of very simple modules that don't have interfaces to export and thus where you can omit the namespace declaration will the compiler enforce the import foo = require('foo') kind of imports.

Or if you set esModuleInterop which was introduced to help close that hole~

[sindresorhus]

@BendingBender Alright. Let's use the correct CommonJS-compatible syntax going forward. I will try to update existing definitions when I have time, but I could use some help with that too.

[BendingBender]

@sindresorhus I will happily help you with this, most of that stuff comes from me after all. Just for me to know how you plan to change existing declarations: do you want to keep the default export to keep breaking changes for TS users as minimal as possible or do you want to tidy up with this topic?

[sindresorhus]

do you want to keep the default export to keep breaking changes for TS users as minimal as possible or do you want to tidy up with this topic?

Is there a way to move to CommonJS-syntax while still not doing a breaking change? I was under the assumption we had to completely move over and do a major bump.

[BendingBender]

We could still leave the default export, this would minimize the amount of breaking changes. This isn't clean, however. This default export would have nothing to do with compatibility with ES modules, it just happens to have the same name.

This would be possible:

declare namespace foo {
	interface Foo {}
}

declare const foo: {
	(fooArg: foo.Foo): void;
	namedExportFn(): void;

	default: typeof foo;
}

export = foo;

[sindresorhus]

Alright, let's do that for the packages where we already released the type definitions, with a TODO comment about removing it in the next major release.