Discuss: Should Highlight.js version 11 npm release be ESM only?

[aduh95]

Is your request related to a specific problem you're having?

I'm always frustrated when starting a project, I don't want to set up a whole build chain and just want to get started:

import highlight from './node_modules/highlight.js/lib/core.js'

And I get an error Error: 'default' is not exported by /node_modules/highlight.js/lib/core.js...

The solution you'd prefer / feature you'd like to see added...

I'd much rather prefer having standard ES6 modules in the npm package, and being able to use them as advertised in the README.md.

Any alternative solutions you considered...

s/module.exports =/export default/ on the node_modules/highlight.js/ files, but that's not very convenient.

Additional context...

Support for ESM has landed in Node.js current and LTS lines recently, with features such as Conditional Exports (useful in case we want to keep the CJS version around, still required for Node.js v10 support).

Having ESM files would also make the library compatible with Deno for free, which would be nice.

[joshgoebel]

Support for ESM has landed in Node.js current and LTS lines recently

Landed landed vs still experimental? Do you have a link/announcement?

Is there a graph or stats anywhere of who is still using Node v10 (it's popularity still)? Is it considered "very old"? We have very limited time - I'd really rather not maintain both CJS and ESM if we can avoid it. I'd much rather say drop CJS with the next major release and switch completely to ESM if it works for the majority, etc.

@egor-rogov @allejo Thoughts?

---

Also worth noting the internal libs are already ESM, but our npm packages (built for Node primarily) are packaged as CJS. If you started from just the GitHub repo source you'd already have ESM all the way down.

[joshgoebel]

@aduh95 Do you know if the built-in support can import from JSON files? We currently do this (via rollup) to pull in the version from the package.json file, though I suppose we could change that if we had to.

[aduh95]

Do you have a link/announcement?

https://nodejs.org/en/blog/release/v12.17.0/

It used to be behind a flag and a warning, it's still officially experimental though. If you want to ditch CJS, it might be wiser to wait for the feature to be officially stable (the goal of Node.js is to have it stable by the time v14 reaches LTS in October).

Node.js v10 is in LTS maintenance mode until April 2021, so we can expect people still using it until its end of life. I don't think there are any official download stats for Node.js versions 😕

What is the timeframe for Highlight.js 11.x?

If you started from just the GitHub repo source you'd already have ESM all the way down.

That's a very good point, although it's worth noting that the source doesn't specify file extension in the imports (E.G.: import './file' instead of import './file.js'), which makes it not suitable for Node.js, Deno, or browser consumption without transformation. Maybe that something we can discuss refactoring?

Do you know if the built-in support can import from JSON files?

Importing JSON using ESM syntax is still experimental on current Node.js version. But there are workaround for this (example).

[joshgoebel]

What is the timeframe for Highlight.js 11.x?

Early 2021 prolly. I don't like to push breaking releases too often... so my idea is a roughly yearly cycle... but I'm also not in a rush if we don't NEED to break anything - 10 could live on longer.

it's still officially experimental though

Yeah, I'd say it's not a super high priority for us while it's still experimental.

it's worth noting that the source doesn't specify file extension in the imports (E.G.: import './file' instead of import './file.js')

If Rollup doesn't mind I'd have no issue with adding .js everywhere.

[joshgoebel]

Is anyone using .jsm? Or a special extension for modules vs normal CJS?

s/module.exports =/export default/ on the node_modules/highlight.js/ files, but that's not very convenient.

Perhaps I should have looked at this a bit more carefully. Is this really ALL you were looking for here? If we shipped those 2-3 build assets (or variants of) with ONLY this change... would that resolve this? I was imagining publishing all the "raw" source files as their own or alternative NPM package... which is a whole other level of complexity.

Just shipping ESM "entry points" for core and index sounds perhaps a lot more doable.

[aduh95]

Is anyone using .jsm? Or a special extension for modules vs normal CJS?

.mjs is the one recognized by Node.js.

would that resolve this?

Yes, that would be totally fine! I think Rollup can handle that, right?

highlight.js/tools/build_config.js

Line 10 in b1bce6e

output: { format: "cjs", strict: false },

Haven't tried it, but I think you can replace it with:

output: [
  { format: "cjs", strict: false },
  { format: "esm" },
]

Would you interested in a PR?

[joshgoebel]

Actually that will mess with all the individual languages we require also, no? Those would then also need to also be modules, yes? That's what I was fearful of... I'm not sure we want to ship +180 additional/duplicate JS files just because of ESM... is that the only way to do this - or am I thinking about it wrong? I wonder if there are any other large modular libraries and how they are dealing with this problem?

Building a "monolith" (like we do for the browser) would solve some of the issues [for some people] but others really prefer to load just core and the languages they need one at a time, and it seems to allow that we'd still need 100s of separate modules.

[joshgoebel]

but I think you can replace it with:

I doubt it. We have a pretty custom build process. It'd probably be easier to build a list of the individual CJS files and then feed them thru a simple text transform... then spit out new files beside them.

But we'd also preferably want our CI/test suite to test these new "build products" and I'm not sure that's even even possible with Mocha. Do you know? Last time I looked at using modules with it things seemed pretty rocky... but perhaps that has improved?

[joshgoebel]

@aduh95 If your actual problem is truly just "get started fast" why not just use the browser build and change it's single CJS export to an ESM export instead?

[aduh95]

Thanks for the detailed explanation, I think I got your point. This discussion would be more productive in a few months when Node.js ESM support has stabilised.

Maybe we can wait and see if the situation evolves on Node.js side, and come back to the "drop CJS with the next major release and switch completely to ESM" mindset when it makes sense?

But we'd also preferably want our CI/test suite to test these new "build products" and I'm not sure that's even even possible with Mocha. Do you know?

Yes Mocha is able to interact with ES modules, you just have import() them instead on require() them.

If your actual problem is truly just "get started fast" why not just use the browser build and change it's single CJS export to an ESM export instead?

I want the best of both world: get started fast and cherry-pick the languages I'm actually highlighting. I know I'm bad ^^