Node.js require() works as envisioned, without requiring symlinks

[pauldraper]

Is your feature request related to a problem? Please describe.

The semantics of the Node.js require() function were designed to be general enough to support reasonable directory structures.

https://nodejs.org/api/modules.html#modules_addenda_package_manager_tips

Yes, but no.

The problem is the require() algorithm essentially requires symlinks. Examples are in the docs, but another example:

Example

The entry point is server@1.0.0.

• server@1.0.0 depends on collections@1.0.0, http@1.0.0, json@1.0.0, and xml@1.0.0.

• http@1.0.0 depends on collections@1.0.0.

• json@1.0.0 depends on collections@2.0.0.

• xml@1.0.0 depends on collections@2.0.0.

The only way to satisfy package expectations is to

• Solution A: Install multiple copies. This can pose functional issues (multiple functions, classes), and performance issues.

• Solution B: Use a symlinking feature from the OS.

Solution B, requiring a particular symlink structure is a problem because:

1. Symlinks/junctions are in general problematic on Windows.

2. Symlinks are often used already for storage and adminstration concerns: placing a file on a different media device, putting the file in a different tree for different management. However doing so, changes Node.js behavior.

   • There is the --preserve-symlinks feature, but again, the fundamental design of module resolution precludes this without making other compromises.

3. Look no further than the dozens of Node.js issues around symlinks (and hundreds in other projects) to conclude that symlink behavior is torturous for Node.js users in way that it isn't for Java, Python, Ruby users.

Symlinks should not be required to use Node.js features. Node.js -- like any other runtime -- should use OS concerns like file systems and symlinks transparently.

Describe the solution you'd like

Node.js resolution works, without mandating how symlinks are used.

For example, a node_modules or node_modules.json file would describe the dependencies.

{
  "collections": "../collections@1.0.0"
}

In conjunction with the existing --preserve-symlinks options, this would provide a mechanism for efficient, deduplicated libraries, mucking with overloaded file system features.

Describe alternatives you've considered

This problem has plagued the Node.js ecosystem for a decade; I am certain there are some other proposed solutions I am ignorant of.

• Yarn PNP, though the Node.js project has issues with that approach.

• Tink, but it's a bit of a hack, and essentially dead.

[aduh95]

Duplicate of #37001.