"Recipes" and "Quick Start:" add a general project config example

[cspotcode]

Spinning off from conversation started here:
#1007 (comment)

I think what's missing in recipes is a general recipe for projects in general - you have examples for all these tools that have special requirements, and that's great, but I'm just looking for a basic, general setup for a project that uses some npm packages. I think that ought to be the first recipe on the menu? 🙂

[cspotcode]

@mindplay-dk we can capture requirements for this more general project configuration example here.

I'm copying over some of my questions from the other thread. Getting a second opinion on these ideas will be very helpful in confidently creating the necessary docs updates. Please take a look and let me know what you think.

• Where should the example live?
  • I can create a new TypeStrong/ts-node-examples repository, with multiple subdirectories for different examples
  • I often create working examples as branches in ts-node-repros, and then I link to them from Discussions. For example, this example using mocha and ts-node with native ESM: mocha and native ESM #1268
• Our docs have a "Recipes" section with configuration examples for specific situations. Should we add a recipe for this?
• The docs have dedicated pages about ESM: https://typestrong.org/ts-node/docs/imports Should they link to this example?
• Should we add a "Quick Start" page to the website, immediately after "Overview"?

[cspotcode]

@mindplay-dk friendly reminder to create a project configuration that does not work, as a starting point for this discussion.

• node version
• package.json file w/ts-node, typescript dependencies
• tsconfig.json file
• index.ts file
• launch invocation (ts-node ./index.ts or other) that demonstrates the failure

Can be posted to a new git repository

[mindplay-dk]

Alright, here's a first draft to get the ball rolling. 🙂

https://github.com/mindplay-dk/ts-node-example

I'm targeting Node 16, the current LTS release. I don't know how common it is to target the next version of Node? It may even be more common than targeting the LTS? I also don't know if there are any special considerations or configuration specific to the next version of Node. But let's get a working example for the version recommended by the Node project itself, and maybe we can think about an example for the next version later.

I've kept all the ts-node settings in tsconfig.json as opposed to scripts, to keep them all collected and visible in one place.

I've kept the package.json and tsconfig.json as minimal as possible. I'd prefer not to have anything "opinionated" in there - just the bare minimum settings required to make things work.

I've put a comment next to each setting in tsconfig.json to explain why these are needed. This is just from my understanding, which may be wrong or poorly explained on some points - please feel free to critique or correct these. (We might should have longer explanations or links for some of these?)

I'm using @tsconfig/node16 explicitly to make this visible in both package.json and tsconfig.json, so it's completely clear that this is designed to work with Node 16. (I'm unsure if this is the best decision? It might be better to just copy the settings we need, so that everything is visible and documented in tsconfig.json?)

I'm approaching this sort-of like a "test suite" - so it just makes a simple test for each of my expectations and prints the result to the console as OK or FAIL.

I'm not sure about all the expectations here. There may be things that can't or shouldn't be supported for some reason? But I would find it confusing if it's valid according to TS and doesn't work with ts-node - in my opinion, if TS can validate or auto-complete something, it should work in ts-node. (Again, there may be reasons it can't or shouldn't - if so, we should probably uncover and document those.)

Interestingly, loading both ESM and CommonJS modules does work for some types of imports in this project - it's the first time I've managed that. I've compared the settings against one of my own projects where it doesn't work, and they seem identical - yet, for some reason, baseUrl and absolute imports in my local project does work, and in this example project they don't.

Documentation mentions "files": true, which we might need for local .d.ts files? Unsure how to test this.

We should probably include an example of importing .json and/or other module types as well? This might require "files": true, so might make a test case for that as well? Unsure.

Documentation also mentions NODE_OPTIONS, which we might need to support somehow? I might prefer node with --register ts-node/register (or is it --loader ts-node/esm??) over ts-node here to keep things simple and familiar for people who regularly use node? Unsure.

I've left some TODOs in test.ts for other things I couldn't figure out.

Let me know if you can think of anything else not covered by the example or my notes here?

[mindplay-dk]

@cspotcode I've added you as a collaborator, feel free to go nuts. 😄

[mindplay-dk]

Also, not clear if tsconfig-paths is even necessary?

What I'm after here is absolute imports, with a / prefix in the paths, to avoid the mess of ../../../ paths everywhere.

It's not clear to me if this package is required only to support path mappings - or if ts-node needs this in order to support absolute paths?

This feature is not important to me, personally - I actually think this does more harm than good. But it probably should be supported and needs an example/test for those who insist on it, so there are as few unpleasant surprises as possible. It's (hopefully) harmless to have this enabled even if it goes unused?

[cspotcode]

Great, thanks. Responding to questions in no particular order:

tsconfig "paths" and "baseUrl"

Path mapping will soon be added to ts-node by #1664. Also, tsconfig-paths does not support ESM, only CJS. (probably why it's not working in the example) So to avoid wasted effort, I think we should focus this example on the near future where ts-node will natively do path mapping.

This feature is not important to me, personally - I actually think this does more harm than good.

I'm glad you said this. Yeah, it's a tad frustrating for people to assume this should work. tsconfig "paths" was never intended by the typescript team to be used in this way. Rather, you're supposed to use the features of your runtime environment or bundler to handle import specifiers, and then tell typescript about it using paths. We're not meant to be doing the opposite: bending the runtime to match tsconfig "paths."

Node has a bunch of built-in features, such as package.json "exports" and package-local imports, which can achieve similar things. I'm pretty sure those will be fully supported with tsconfig module=nodenext #1727

Node version

Node version is less important than ts-node version. Using node lts is totally fine; especially if it's the latest minor&patch version of it. Very important to use latest ts-node version to ensure you have the latest bugfixes and features.

NODE_OPTIONS

The benefit of NODE_OPTIONS is that it gets passed into node processes even when you're not invoking them by typing node in your terminal or a script. Hypothetical example, NODE_OPTIONS="--loader ts-node/esm" fancy-cli-tool ./fancy.config.ts We're not calling node directly, but we are running a tool written in node, and we do want it to have TS support in both commonjs and ESM. So we pass it the loader flag.

Import from ansi-colors looks wrong

// TODO named imports from NPM packages: doesn't work for some reason
//import { red } from "ansi-colors";

Where is the red export? https://unpkg.com/ansi-colors@4.1.1/index.js

[cspotcode]

Took another look at ansi-colors. The issue you are hitting is a node issue, not a typescript nor a ts-node issue.

https://nodejs.org/dist/latest-v18.x/docs/api/esm.html#interoperability-with-commonjs

Notably:

Named exports may be available, provided by static analysis as a convenience for better ecosystem compatibility.

"static analysis" not "runtime analysis," meaning node is looking at the source code and guessing.

In the case of ansi-colors, the source code is https://unpkg.com/ansi-colors@4.1.1/index.js

module.exports = create();

Static analysis cannot discover any of the properties like red, so named exports are not available. Thus you're stuck with:

When importing CommonJS modules, the module.exports object is provided as the default export.

You're stuck using the default export. If tsc is not happy with that, then ts-node in typechecking mode will raise the same error.

[mindplay-dk]

You're stuck using the default export. If tsc is not happy with that, then ts-node in typechecking mode will raise the same error.

Oh, but it type-checks - the .d.ts file is declared correctly, and tsc is happy. It only fails at run-time.

And look:

https://codesandbox.io/s/proud-thunder-zp79ud?file=/src/index.js

If that's not supposed to work, why does this work?

CSB uses Parcel. Maybe it has better support for CommonJS than Node does?

[mindplay-dk]

Ahh, now I see what you're talking about:

https://www.typescriptlang.org/tsconfig/#node12nodenext-nightly-builds

Available in nightly builds, the experimental node12 and nodenext modes integrate with Node’s native ECMAScript Module support. The emitted JavaScript uses either CommonJS or ES2020 output depending on the file extension and the value of the type setting in the nearest package.json.

I just assumed that of course TS would support this, but it's not a stable feature yet.

I guess for this we'll just have to wait, yeah.

[mindplay-dk]

Are there any plans to support import { a } from "./mymodule" without the .js file extension?

I absolutely despise the fact that TS won't let you use the actual .ts file-extension, as we do for every other type of module throughout the ecosystem - being forced to put the wrong file-extension .js there is even more vexing. At least putting no file-extension doesn't seem quite as misleading to me. 🤔

[cspotcode]

We already do with --experimental-specifier-resolution=node. And honestly, I bet when you turn off the typechecker, it'll work with the .ts extension in ts-node. I recommend trying both of those options.

Remember that typechecking is controlled by the compiler, not ts-node. We delegate typechecking to the compiler. Even if we could change the rules in ts-node, your code would still break in tsc.

If you're curious to learn more, I recommend googling for why the TS team has decided that the compiler will never rewrite import specifiers in any way. If you want the file extension thing to change, you'll need to convince the TypeScript team, not me. You can ask about this on the TypeScript Discord if you want to discuss with others; it's a great community.

[mindplay-dk]

We already do with --experimental-specifier-resolution=node.

Is that a ts-node option? I don't see that option in the docs.

And honestly, I bet when you turn off the typechecker, it'll work with the .ts extension in ts-node.

Right, but as you said:

Remember that typechecking is controlled by the compiler, not ts-node.

So that's not an option.

If you're curious to learn more, I recommend googling for why the TS team has decided that the compiler will never rewrite import specifiers in any way.

<rant>

Oh, yes - I've come across this thread dozens of times over the years, and I've attempted to debate it more than once. I mean, they don't have to apply the transform - they just have to allow the file extension, so other compilers can support it. I mean, they do this for plenty of things already - most notable NPM packages, which you can't actually use in code that targets the browser, without adding a bundler to actually link packages to make it work.

I don't know why allowing yet another thing that isn't actually supported by tsc is a problem. When it comes to compilation, the TS compiler is mostly a toy anyway - only beginners use it for toy projects while learning, and of course some projects like ts-node use it as a component internally. It seems like the TS team have always been content to let the community build actually useful compilers, while leaving every newbie completely in the dark with a toy compiler that can't even link a module. 😞

</rant>

Anyhow.

Where can I find information about this --experimental-specifier-resolution flag?

I want to get that into the example project and make it use imports without the .js extension.

(If you think that makes sense?)

[mindplay-dk]

Oh, it's a node option - and it basically says, "do not use". 🤨

Do not rely on this flag. We plan to remove it once the Loaders API has advanced to the point that equivalent functionality can be achieved via custom loaders.

Sounds like something ts-node might be able to leverage internally?

[mindplay-dk]

Ah, and you meant this would work for the .ts extension, which TS doesn't allow, right?

It wouldn't work without the file-extension?

So there is no way to do that at the moment with ts-node in ESM mode, correct?

[cspotcode]

Regarding the CommonJS interop, I forgot about a nifty thing that Typescript's node16/nodenext mode does:

You can do import ansiColors = require('ansi-colors'); in any file, even ESM files. TS will emit a bit of boilerplate to gin up a require function by calling createRequire. Then it'll use that require function to grab the module. You can use this to grab CommonJS modules from within ESM.

[mindplay-dk]

How is that nifty?

Ideally, you shouldn't need to think about how modules are implemented. Conceptually, there are modules, and they have members - this is true whether they're implemented as CommonJS or ESM modules.

What you want is to reference those members - how they were defined or implemented should be an implementation detail.

With bundlers, there are concerns about tree shaking and bundle size and so on - these things don't really matter under Node.JS, so I don't know why the consumer of a package should even need to know or care if the package is internally CommonJS or ESM.

import is how we write imports now - TS allows it, and it should "just work", as it does with e.g. Parcel, and most bundlers. (through the use of some plugins.)

This shouldn't be out of reach for ts-node, should it?

[cspotcode]

I recommend taking some time to spec this out:

Write down exactly what your TS should be transformed into so that your imports can be written in exactly the way you are hoping. Use the ansi-colors import as an example.

Then think about how that transformation can be done reliably for all kinds of TS code, think about the performance implications (if any). Remember that tricks which work for ansi-colors may actually break for other modules, so keep that in mind.

Remember that this will need to be compatible with the code transformers we support: both tsc and swc.

Think about the complexity tradeoffs.

Think about the rules node imposes upon us, how node behaves at runtime, remembering that we cannot get access to node's internals.

[cspotcode]

Can repurpose this for the examples repo:
https://github.com/TypeStrong/ts-node-npx-example
Note to self: when I rename it, find references in docs and discussions, adjust those links if reasonable to do so
Also can bring in zx example from: cefn/zx-tsnode-repro#4