Add flag to not transpile dynamic import() when module is CommonJS

[dummdidumm]

Suggestion

🔍 Search Terms

List of keywords you searched for before creating this issue. Write them down here so that others can find this suggestion more easily and help provide feedback.

dynamic import, commonjs, esm, node

✅ Viability Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion

Add a tsconfig flag to not transpile dynamic imports when module is CommonJS. Something like transpileDynamicImport which would be true by default and only take effect when "module": "CommonJS".

📃 Motivating Example

Since Node 12, it is possible to use dynamic imports. Yet I'm not able to tell the TS compiler to not transpile this into Promise.resolve().then(() => __importStar(require('..')));. This prevents users from importing ES modules into CommonJS, which will become increasingly common now that Node is transitioning to ES modules.

💻 Use Cases

The main use case is to be able to import ES modules into CommonJS, which is possible to do with import(). The workaround today involves an additional build step which does replacements to hide the import() statement from TS so it does not get transpiled and re-add it later on, which is suboptimal.

[marvinhagemeister]

Just ran into this myself and noticed that this breaks ESM interop in node. Per node's documentation it is possible to load esm files from commonjs via import(). Therefore the dynamic import statement must not be transpiled to require calls. This is currently broken when using TypeScript.

[RyanCavanaugh]

@weswigham any additional context you might provide here?

[orta]

I can as I've been asked about this in a few contexts. Today we assume that all requires and imports have the same end result post-transpilation, as either commonjs or esmodules.

However, it's a bit more nuanced because node supports the same keywords working differently depending on if you're in an ESM context or CommonJS context. So in a commonjs context, you normally can't use import/export, but you can use await import to import an esm module:

// index.cjs
let module = await import('/modules/my-module.js');

What TypeScript thinks today is that this import should be switched to a require

const a = Promise.resolve().then(() => __importStar(require("/modules/my-module.js")));

Which is what you want for existing cases because that was a normal feature of require statements. It's a signal that the thing you're about to grab is a ESM module.

What's tricky is that TypeScript has no way to disambiguate whether you want await import in a cjs context to continue being a = require( or now stay as a = await import(. We'd either need a flag like the above which handles it app wide, or a pragma at the import call-site.

This seems to be hitting a few big projects because people want to have config files in ESM but let the app stay in cjs.

[typhonrt]

Greets... Thanks @dummdidumm for filing this issue. I was about to post the same thing about a week ago.

@orta not transpiling dynamic import is an important aspect to enable for Node v12+ in regard to loading ESM for CommonJS targeted TS efforts. It will be used for a lot more than loading configuration files. I am an outside contributor attempting (ESM support was added see - oclif/core#130) to get Heroku / Salesforce to add ESM loading capabilities to their Oclif v2 CLI framework that is due for launch in the coming months. I have already worked out the essential changes with a proof of concept that is discussed in the issue linked above, but it required a workaround. Oclif is a TS project which targets CommonJS for release. One can build CLIs in CommonJS or TS; my proposed changes adds ESM to the mix which is quite relevant as the Node ecosystem moves to ESM.

The least worst workaround I could come up with is the following:
const _importDynamic = new Function('modulePath', 'return import(modulePath)') which is used internally to an encapsulated addition for loading ESM via dynamic import or require for non-ESM code - module-loader.ts.

As an outside contributor I certainly could not touch the build process, so the above seems to do the trick until a flag can be added to Typescript which is certainly desirable.

[aomarks]

What's tricky is that TypeScript has no way to disambiguate whether you want await import in a cjs context to continue being a = require( or now stay as a = await import(. We'd either need a flag like the above which handles it app wide, or a pragma at the import call-site.

If node style module resolution is enabled, and you encounter a bare module specifier in a dynamic import, would it make sense to check the imported package's package.json for "type": "module" and preserve the dynamic import if it is present?

[yinzara]

If node style module resolution is enabled, and you encounter a bare module specifier in a dynamic import, would it make sense to check the imported package's package.json for "type": "module" and preserve the dynamic import if it is present?

If you also checked the extension of the file being imported for ".mjs", then yes I think that would be appropriate.

This has become a big problem for using some packages in a CommonJS app that are MJS only.
sindresorhus/meta#15

[styfle]

The beauty of await import() is that it can import both ESM and CJS:

// index.cjs

const one = await import('./new.mjs');
const two = await import('./old.cjs');
const three = await import('./unknown.js');

This is highly desirable to maintain interop as the ecosystem slowly moves towards ESM because I can import a file without knowing the module system. The current all-or-nothing approach is too painful.

We'd either need a flag like the above which handles it app wide, or a pragma at the import call-site.

I would be okay with either solution, but the pragma would give the most flexibility if it can be configured per call site.

[orta]

This is going to be coming in natively and without a flag/pragma I expect, when more of the node ESM support starts rolling after #44501

[voxpelli]

For future reference: A workaround for this issue can be had by moving the import() to a non-compiled dependency, such as inclusion.

Biggest drawback of this, apart from it being confusingly non-standard, is that types won't get imported. That can be worked around by manually importing the types.

Complete example:

import inclusion from 'inclusion';

export async function foo(): Promise<void> {
  const pMap: typeof import('p-map')['default'] = (await inclusion('p-map')).default;
}

[NoyHanan]

This is brilliant. But I like this:

Function('return import("node-fetch")')() as Promise<typeof import('node-fetch')>

Apparently it's much(?) faster, not sure but maybe even safer.

Hello everyone, has anyone else encountered a problem while using this workaround with Jest?

[fcano-ut]

Unfortunately, on our use case none of the proposed workarounds work, because they are incompatible with a strict Content Security Policy. We cannot add unsafe-eval to the CSP policies due to client requirements, so doing await eval or new Function('') is not possible for us.

If you are asking why would I transpile to CJS even though I'm using the module in the browser: a) I'm already using webpack so I don't care about module formats, and b) Jest doesn't support ESM and I don't want my users to be messing up with Jest and babel transforms just to run some tests.

Is there any workaround that doesn't require to eval code, or is there a flag planned to solve this issue? Thanks

[kth-tw]

It is work for me after set tsconfig.json compilerOptions to node16

 "compilerOptions": {
   "moduleResolution": "node16",
 }

[Zielak]

It is work for me after set tsconfig.json compilerOptions to node16

 "compilerOptions": {
   "moduleResolution": "node16",
 }

That doesn't work with module CommonJS:

Option 'module' must be set to 'Node16' when option 'moduleResolution' is set to 'Node16'.

[elovin]

I also had to enable "skipLibCheck": true:

		"lib": ["ES2022"],
		"target": "ES2022",
		"module": "Node16",
		"moduleResolution": "Node16",
		// currently necessary due to ESM only imports
		"skipLibCheck": true,

[shirakaba]

Solution to a challenging case

I solved a particularly difficult case this way, by writing a module in CommonJS outside of the TypeScript source tree (so that it doesn't get compiled) that simply re-exports the desired ESM library. Here's the setup:

• The project is an npm package named @expo/cli.
• It transpiles the TypeScript files in src to CommonJS files in build.
• We want to import the ESM package globby, but TypeScript is unfortunately transpiling all our dynamic imports into require() statements.
• Not actually relevant to this issue, but does complicate the solution: We can't use relative imports that climb outside of the src folder, because the build system outputs a directory structure that's not symmetrical with the source tree (it's nested one level deeper).

Here's the project tree visually (I added the lib directory to solve the issue at hand):

  .
  ├── build
  │   ├── bin
  │   │   └── cli.js
  │   └── src
  │      ├── index.js.map
  │      └── index.js
+ ├── lib
+ │   ├── importGlobby.d.ts
+ │   └── importGlobby.js
  ├── package.json
  ├── src
  │   └── index.ts
  └── tsconfig.json

File contents

lib/importGlobby.js

First, you make a CommonJS module that dynamically imports and then returns the globby ESM module:

/**
 * A re-export of globby.
 *
 * globby is an ESM module, and so can only be imported into a CommonJS project
 * by using dynamic import(). However, TypeScript transpiles import() to CommonJS.
 * @see https://stackoverflow.com/questions/65265420/how-to-prevent-typescript-from-transpiling-dynamic-imports-into-require
 * @see https://github.com/microsoft/TypeScript/issues/43329
 */
async function importGlobby() {
  return await import('globby');
}
exports.importGlobby = importGlobby;

lib/importGlobby.d.ts

Optionally, you can write some typings alongside it.

export declare async function importGlobby(): Promise<typeof import('globby')>;

src/index.ts

If this issue didn't exist (i.e. if globby were a CommonJS module), we'd be writing something as simple as this:

// If this issue didn't exist
import { globbyStream } from 'globby';

export async function demo(){
  for await (const path of globbyStream('*.tmp')) {
    console.log(path);
  }
}

... but as that's not the case, see our workaround code below.

If you have a symmetrical build tree

For simple projects with a symmetrical build tree, you can consume the module as easily as this:

const { importGlobby } = require('../../lib/importGlobby');

export async function demo(){
  // Hurrah, we've imported the `globbyStream` function from the `globby` module!
  const { globbyStream } = await importGlobby();

  // Now just use `globbyStream` as usual:
  for await (const path of globbyStream('*.tmp')) {
    console.log(path);
  }
}

If you have an asymmetrical build tree

However, if, like me, you have an asymmetrical build tree, then your best option to get both typings and implementation may be to import relative to the package itself, or just leave it with any type.

Otherwise, you could just use it untyped.

// Do a little dance to grab the typings
import type importGlobbyModule from '../../lib/importGlobby';

// Import the module relative to the project (our project is @expo/cli)
const { importGlobby } = require('@expo/cli/lib/importGlobby') as typeof importGlobbyModule;

export async function demo(){
  // Hurrah, we've imported the `globbyStream` function from the `globby` module!
  const { globbyStream } = await importGlobby();

  // Now just use `globbyStream` as usual:
  for await (const path of globbyStream('*.tmp')) {
    console.log(path);
  }
}

[juliolopeztorres]

This is brilliant. But I like this:

Function('return import("node-fetch")')() as Promise<typeof import('node-fetch')>

Apparently it's much(?) faster, not sure but maybe even safer.

Hello everyone, has anyone else encountered a problem while using this workaround with Jest?

I did but not exactly your case. I faced an error using the eval() workaround.

At the end, I decided to add --experimental-vm-modules option to NODE_OPTIONS before calling Jest. For reference, I am using Jest 29.5 at the moment with node 20.13.

My script line in package.json looks like:

...
    "jest": "NODE_OPTIONS=\"$NODE_OPTIONS --experimental-vm-modules\" jest ..."
...

[JinYuSha0]

I'm dying of this problem.
I tried using tsconfig.json to exclude file but it didn't work
In the end, I chose to just make a simple npm package await-import-dont-compile to prevent compilation

This is how he uses it

import awaitImport from "await-import-dont-compile";
import { pathToFileURL } from "url";

export const dynamicImport = async (filepath: string) => {
  return await awaitImport(pathToFileURL(filepath).href);
};

Hope it helps!!!