v4 can't be imported in node

[g012]

Hello,

I'm not knowledgeable at all in all the module formats / import mess of javascript / node, but I'm using KeystoneJS and I import nanoid in it. Since version 4 (2 days ago), I have this error :

Error [ERR_REQUIRE_ESM]: require() of ES Module /node_modules/nanoid/async/index.js from schema.ts not supported. Instead change the require of index.js in schema.ts to a dynamic import() which is available in all CommonJS modules.

This is how I've always imported nanoid :

import { nanoid } from 'nanoid/async';

Reverting to 3.3.4 works.
I see in the commit comment that you removed CJS support, but as I understand, using this import statement, I'm using the new thingy module stuff, not CJS which uses require instead, and you have "type: module" in your package.json, so I really don't know what's wrong here.

Any idea what I'm doing wrong ? Thanks !

[PoProstuWitold]

I'm also having this issue

[marekkobida]

Me too.

[danielweck]

This is a feature, not a bug. See the changelog for details of the breaking change in version 4.0 (semantic versioning major increment): https://github.com/ai/nanoid/blob/main/CHANGELOG.md#40

[danielweck]

Also see the author's feedback in this previously-closed issue: #364 (comment)

[ai]

Check your TS config.

Seems like TS convert import in source TS sources to require. I use this option in tsconfig.json: "module": "esnext" (buy you may need to convert the whole app to TS).

[g012]

This is a feature, not a bug. See the changelog for details of the breaking change in version 4.0 (semantic versioning major increment): https://github.com/ai/nanoid/blob/main/CHANGELOG.md#40

This doesn't help at all, we saw the comment in the commit on main page of code repo, as stated.

Also see the author's feedback in this previously-closed issue: #364 (comment)

In this one the guy says he was using require, and is told to move to import. As stated, I (we ?) are already using import since long.

Check your TS config.

Seems like TS convert import in source TS sources to require. I use this option in tsconfig.json: "module": "esnext" (buy you may need to convert the whole app to TS).

Thanks, I've tried changing module: "commonjs" to module: "esnext", but it made no difference. Could you share your whole tsconfig file ? Here's mine:

  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "strict": true,
    "noEmit": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "jsx": "react"
  }
}

[ai]

Maybe you should remove "esModuleInterop": true,. Try to call tcs file.ts to be sure that you have import in the compiled results.

Here is one of my examples https://github.com/hplush/slowreader/blob/main/tsconfig.json

[danielweck]

Hello @g012 ,
have you seen this doc?
https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c

Support for ESM depends heavily on tooling, for example you may have different tsconfig.json settings for your unit tests, your dev builds, and your production release ... depending on whether your codebase is managed with WebPack, Rollup, Vite, Jest, Vitest, etc.

The author of NanoID said that he will kindly continue to offer off-the-shelf compatibility with CommonJS by maintaining v3 in parallel to ESM v4, but I am concerned that noise/signal ratio will increase in this issue tracker due to v4 support queries.

As far as I can tell there are no problems with NanoID's published package.json export map etc. so I think it would make sense to continue such general ESM/CJS conversations in a GitHub discussion instead.

:)

[amir-arad]

I'm having the same issue in two flavors. first one was described above. second one is:

 Details:

    /home/circleci/project/node_modules/nanoid/index.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { randomFillSync } from 'crypto'
                                                                                      ^^^^^^

    SyntaxError: Cannot use import statement outside a module

    > 1 | import { customAlphabet } from 'nanoid';
        | ^
      2 |
      3 | const letters = customAlphabet('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ', 1);
      4 | const alphanumeric = customAlphabet('0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ', 5);

in short, I need to turn on an experimental feature in node to enable ESModules import in Jest.
I can't wait for ESModules to become a standard, but right now it's not yet frictionless.

this is my update PR: starwards/starwards#847

[picardplaisimond]

I think this decision was made far too early. ESM has just started to be supported by NodeJS. I'm not sure if I want to refactor my whole "production application" for ESM right now.

I just wanted to point that out.

[danielweck]

. I'm not sure if I want to refactor my whole "production application" for ESM right now.

"We will support 3.x branch with CommonJS for users who can’t migrate to ESM"

https://github.com/ai/nanoid/blob/main/CHANGELOG.md#40

[g012]

Hello @g012 , have you seen this doc? https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c

Support for ESM depends heavily on tooling, for example you may have different tsconfig.json settings for your unit tests, your dev builds, and your production release ... depending on whether your codebase is managed with WebPack, Rollup, Vite, Jest, Vitest, etc.

The author of NanoID said that he will kindly continue to offer off-the-shelf compatibility with CommonJS by maintaining v3 in parallel to ESM v4, but I am concerned that noise/signal ratio will increase in this issue tracker due to v4 support queries.

As far as I can tell there are no problems with NanoID's published package.json export map etc. so I think it would make sense to continue such general ESM/CJS conversations in a GitHub discussion instead.

:)

Hi Daniel,

Yes I think you're right, the Keystone JS toolset converts typescript to CJS, and if I disable that, I get lint warnings for k6 core and some others, so I've decided to stick with branch 3 until everything migrates.

Thanks for the help !

[garretthogan]

This is a feature, not a bug. See the changelog for details of the breaking change in version 4.0 (semantic versioning major increment): https://github.com/ai/nanoid/blob/main/CHANGELOG.md#40

Really terrible decision. Was gonna use this in place of shortid because apparently this is more secure? But since I literally can't import the latest I'm just gonna keep using shortid. Good luck building a product that people can't use!

[danielweck]

... since I literally can't import the latest I'm just gonna keep using shortid.

"We will support 3.x branch with CommonJS for users who can’t migrate to ESM"

https://github.com/ai/nanoid/blob/main/CHANGELOG.md#40

[weyert]

Are you planning any security updates for 3.x? Can you clarify what you mean with 'support 3.x branch'?

[ai]

Are you planning any security updates for 3.x? Can you clarify what you mean with 'support 3.x branch'?

Of course. All fixes will be backported.

[ayushsharma82]

Use this version while the author fixes commonjs issue:

npm i -S nanoid@^3.0.0

[weyert]

Whatever works for the maintainer. I am happy the person still maintains the CommonJS version for security updates :)

[ai]

Theres no harm in supporting commonJS neither does it has any performance ,build or maintainability problems. Its too early to move into ESM imo

Dual ESM/CJS packages is not a good thing. As a developer of dual-publish I can say it without a doubt.

1. You need to duplicate every file in npm package which increase node_modules size.
2. It leads to expected problems in some non-standard systems (for instance, Jest is using custom ESM resolver and dual packages could increase the error rate).
3. Dual packages requires extra build step (like dual-publish or babel). With ESM-only you can publish your repository directly.
4. It also make more complex another tools. For instance, Size Limit will require extra plugin.
5. There is no simple way to use version from repository (for instance, to test PR) without some painful bugs. Because dual packages requires build step.

Current state of ecosystem when we need to support both ESM and CJS is very hard for maintainers. Our community need to migrate to ESM-only future. It was already a few years since ESM announce.

[Julusian]

Current state of ecosystem when we need to support both ESM and CJS is very hard for maintainers.

That is true, but esm only packages can be very hard for users ;)

Im perfectly happy to keep using v3 until I can switch things to ESM (there is already a pretty long list of other dependencies that cant be updated), but that switch is currently impossible due to electron not supporting ESM in the backend at all, and jest needing nodejs to finish some features before they can have feature parity with CJS.
I personally would have no problem with libraries pushing users to ESM if nodejs & electron were ready for it. But they aren't, so even though ESM has been around for years, the push from library authors feels premature to me.

[ai]

I personally would have no problem with libraries pushing users to ESM if nodejs & electron were ready for it

Electron and Node.js will not be ready for years if there will not be push from the community.

It is better to support ESM issues in Electron and Node.js.

Im perfectly happy to keep using v3 until I can switch things to ESM

You do not need to always use the latest version of the dependency. You need to use the latest supported version. Both branches 3.x and 4.x are supported. Don’t worry that you are on 3.x branch.

[weyert]

I don't really mind maintainers of packages switching to ESM. It's their project. Only I wish they don't forget their old CommonJS packages when vulnerabilities in dependencies or similar occur. So I was glad to hear this is the case for this super useful package :)

I really need to find out how I can get ESM modules working with Jest. Or is there a better alternative for Jest that has better ESM support? Anyone tried Vitest? Looks like its compatible with Jest's expect?

[ai]

Or is there a better alternative for Jest that has better ESM support?

For simple Node.js libraries uvu is better.

For React projects Vitest is the best option.

[Julusian]

You do not need to always use the latest version of the dependency. You need to use the latest supported version. Both branches 3.x and 4.x are supported. Don’t worry that you are on 3.x branch.

Exactly. Im using 4.x in some projects and 3.x in others, depending on whether they are ESM or not. Hopefully 3.x will get updates for a few years, as I expect ESM wont be usable for everyone for at least a couple more years

I really need to find out how I can get ESM modules working with Jest. Or is there a better alternative for Jest that has better ESM support? Anyone tried Vitest? Looks like its compatible with Jest's expect?

I believe that Jest does mostly work, but they don't have feature parity with CJS. jestjs/jest#9430
The main blocker for me is the lack of mock support, which I believe is blocked on the required nodejs features still being experimental.

[dannyglovory]

Use this version while the author fixes commonjs issue:

npm i -S nanoid@^3.0.0

Thank you, it's works for me

[ai]

Just stay on Nano ID 3 if you have CJS project. We will support Nano ID 3 for a few years.

I delete a very hacky and dangerous advice of crazy wraps to load ESM to CJS.

[jpike88]

No, I'm going to use v4 using the 'hacky and dangerous' method because I don't want to worry about being stuck on an obsolete version which will not be supported soon.

[ai]

I don't want to worry about being stuck on an obsolete version which will not be supported soon.

Nano ID 3 is not obsolete. It is a parallel version. We will support it for a few years.

Just use Nano ID 3.x. There are no rational problems with this version.

[jpike88]

If a version of an open source node library has got an end of life announced for it, I consider it already on its way out. It's how I and many other devs out there work, and it's all about minimising technical debt.

That will be the only dependency of over 50 in my package.json I have to worry about pinning, which I've managed to avert worrying about by doing this 'super dangerous hack' (which isn't really that dangerous at all, I have an integration test that covers usage of it and continuously audit my packages).

The problem here isn't whether or not people should stay on v3 or not, it's about understanding the reality of many projects out there that do not have the resources or priorities around upgrading to ESM. ESM isn't a magical solution to dependency security, you can still do plenty of mischief whether your lib is used in CJS or ESM mode.

[ai]

@vdjurdjevic if you like to keep everything fresh, you have an another reason to move to ESM. ESM is the future of imports in JS. Keep your project fresh too. 😊

[vdjurdjevic]

Trust me I tried :) But everything started to break, spent few hours trying to make it work and then decided to go back. It's medium size project, with 70 dependencies, it's not that easy to switch

[garretthogan]

@vdjurdjevic if you like to keep everything fresh, you have an another reason to move to ESM. ESM is the future of imports in JS. Keep your project fresh too. 😊

I admire the attempted prescience but please stop making these decisions for developers that take pride in their work. We will decide the future of our own stack whether it includes this dep or not.

[markhughes]

@vdjurdjevic if you like to keep everything fresh, you have an another reason to move to ESM. ESM is the future of imports in JS. Keep your project fresh too. 😊

I admire the attempted prescience but please stop making these decisions for developers that take pride in their work. We will decide the future of our own stack whether it includes this dep or not.

v3 is still supported

[jgillick]

You should be able to wrap this with a dynamic import:

For example:

src/lib/nanoid.ts

import type NanoidLib from 'nanoid';

let _lib: typeof NanoidLib;

/**
 * Wrapper to import the ESM module into this project
 *
 */
export async function nanoid(size?: number) {
  // Load library
  if (!_lib) {
    _lib = await import('nanoid');
  }

  // Create ID
  return _lib.nanoid(size);
}

src/usage.ts

import { nanoid } from 'lib/nanoid';
async function doingStuff() {
  const id = nanoid();
}

[cstavaru]

Some funny stats from npmjs:

nanoid weekly downloads: 37 million.

v4 and v5 combined combined weekly downloads (ESM): 1.9 million.
v3 weekly downloads (CJS): the rest up to 37 million.

"ESM is the future". Ok, but how many centuries into the future ? :)

[ai]

@pBread just move your project on ESM. You will have problems with CJS project not only because of Nano ID, but because of all Sandre's packages, etc. Even Vite deprecated CJS API.

Open source is about community work, not about demanding something from people whom you are not paying.

[ai]

Is there a reason you don't just transpile it w/a microbundle or something similar?

It is exactly what we did in 3.x.

It has three problems:

1. it increase library size in node_modules size which is against the philosophy of nano-library.
2. It could not fix all compatibility problems. I still need to process issues from Jest and other edge cases.
3. Any build step requires more work to maintain the library. Check ChangeLog, some of releases was because build step did something wrong.

[jsardev]

lol, this thread keeps going. @ai I admire that you still have the strength to write everyone back 😅

[markhughes]

Alrighty, I'll fork my own version.

@pBread are you forking nanoid? 3.3.7 was released not too long ago.. 3.x is still fine if you need CJS, and don't want to for whatever reason move to ESM.

..

@ai I admire that you still have the strength to write everyone back 😅

I'm impressed as @jsardev is. So incredibly patient.

[Petersdavis]

For people coming here because of Jest specifically you can setup a mock for nanoid which worked to solve our issues:
mock.js

jest.mock('nanoid', () => {
  return { nanoid: () => '1234' };
});

and then call it in global setup:
jest.config.js

{
  ...
  setupFiles:["./mock.js"]
};

[ai]

Node.js 22 allowed to require() ESM dependencies (like nanoid@4).

Just run your app with experimental-require-module flag:

node --experimental-require-module app.js

[HaTiIn]

Is there no way to have a single source that exports ESM and commonjs?

[ai]

There are many ways, but none of them works in 100% cases of big user base of Nano ID.

For instance, Jest replaces Node.js loader with own implementation which kills most of hacks.

And all methods bloats node_modules size.

The best way is to use latest Node.js where you can require ESM in CJS (they are going to remove that limit) or just migrate your app to ESM.

[dcsan]

maybe its worth creating a new library which is ESM only to avoid this breaking change breaking people's apps?

FWIW I only use typescript now and getting ESM to work there seems extremely fragile. switching the compiler output then breaks other things.

[ai]

We removed CJS because we had many issues from Jest users because of dual CJS/ESM system.

Just move your app to ESM. And easy and future proof.

[HaTiIn]

I have tried to migrate my application to ESM. After several days I gave up because there were so many hurdles for which I could not find a solution. From my point of view, ESM is stillborn and many other developers have probably had the same experience as me, otherwise your package in version 3 would not have so many daily downloads.

[aggarwaldev]

@ai Hey, thank you for the efforts on this library but please consider the following:

• Community: Someone suggested forking the library as it's MIT licensed to support the commonJS module, I firmly believe it defeats the purpose of the community. A fork wouldn't receive as many fixes as the original library and would divide the efforts/attention given to a particular library.
• Ecosystem: I'm not kidding when I say I've been facing issues when dealing with multiple frameworks and bundlers. I'm using NestJS which relies on webpack by default and uses TS. TS/Webpack gave me different errors while trying to "fix" tsconfig.json. It's simply a headache and after a day of trying various solutions, I'm forced to reach out.
• Feedback: Based on the above discussion, please consider the feedback shared by different developers above. They wouldn't try to pursue you if things were streamlined or if there was a way to resolve this without major refactoring. There's no point in discussing so much. It feels like adamant behavior.
• Versioning: While the comments above claim that 3.x has no EOL and security patches will be addressed, please let me know what should I think of this comment: 4.0 doesn't support require in nodeJS #364 (comment)
  It clearly states that 3.x will be supported for a year (limited time).
• New features: No one wants to use an outdated library because obviously, they might miss out on some critical feature. 3.3.7 was published 8 months ago, guess what 3.x node version didn't receive the web crypto API update.

I understand that the goal of 4.x was to reduce the size for browsers, is it not possible to use monorepo and publish a separate node.js version which will support both ESM and CommonJS?

P.S. I'm not against ESM, just unable to put it to good use till now

[ai]

@aggarwaldev thanks for get reasoning.

My current 2 reasons to keep ESM-only:

1. I want to have nano size not only in JS bundle, but also in node_modules
2. Node.js will soon support require() ESM files (like Bun already does). See --experimental-require-module.

Your arguments do not change my arguments, so I still not convinced (but I open for discussion).

A few words about why your arguments did not convince me:

Community

I do not believe in front-end community anymore. I am mostly creating tools for myself and my project in new fields.

Front-end community doesn’t give enough back to maintainers (not only in terms of money), but demanding a lot.

Ecosystem

Since:
— I do not believe in main-stream community (which destroyed ecosystem in my opinion because of hype-based choice and one popular tool instead of having tool diversity)
— In modern ecosystem there is no way how we can make CJS + ESM works. We will always break something. 3.x doesn’t work in any case too (the main problem for 3.x is mostly Jest).

I prefer to works with “indie” ecosystem. For me, it is ESM-only.

In my indie ecosystem everything works out-of-box and even better than with 3.x. Why I should care about main-stream community ecosystem if the community behind this ecosystem doesn’t care about my need?

Feedback

Since I do not believe in main-stream community choices, I care about feedback from people important to me.

It was too many mistakes in front-end stack to believe in every feedback.

Versioning

There is no strict 3.0 end-of-time.

Even if I do not believe in the main-stream community, I will support 3.x as much it is needed (for instance, until we will have stable require() for ESM files in LTS version).

New features

We do not plan any new features for 5.x or 6.x.

Web Crypto API update is not a new feature. Nothing will be changed in public API by moving to Web Crypto API internally (correct me if I am missing something).

[cstavaru]

The only thing that saves ESM is that NodeJS 22 and higher will finally allow mixing ESM and CJS modules. Otherwise we would keep seeing ESM packages downloaded 25 times less than CJS for years. As I have always said, allowing coexistence of CJS and ESM in software was the only way to save ESM. So basically after NodeJS 22 (and especially after that "experimental" flag will be default") this kind of thread will disappear.

As for me, I find CommonJS to be powerful, flexible, with clear rules, basically meeting all my needs, I never felt the need for anything else, but I am also not opposed to it as long as it allows me to deliver software into real, money-making production (and not only to embrace a new technology just because it's new and shiny).

[Leizhenpeng]

You can try this package to support the latest version of nanoid and compile it into the format supported by cjs.

https://github.com/Leizhenpeng/nanoid-cjs

import { nanoid } from 'nanoid-cjs'
model.id = nanoid() //=> "V1StGXR8_Z5jdHi6B-myT"

[ai]

New Node.js 20 supports --experimental-require-module flag too.

This flag allow you to use Nano ID 5.x with CJS apps.

https://nodejs.org/en/blog/release/v20.17.0