Skip to content

Proposal for correct version lookup in deeply embedded non-TS external modules #4673

Closed
@poelstra

Description

@poelstra

Update 20150908: applied feedback from comments. Thanks @jbrantly! Also renamed "Legacy mode" to "Mixed mode" and clarified what it's about.
Update 20150909: more feedback from @jbrantly.

Now that #2338 is implemented, we have nice resolution of typings for 'native TS modules'.
However, exporting typings for non-TS modules can be problematic in case of deeper dependency trees.

Issue #2839 (by me) contains a proposal to solve this, but does not talk about deeper-than-one dependencies in case of conflicting versions. Issue #4665 (by @mhegazy) also addresses non-TS resolution, but does not explicitly mention the lookup logic.

Example dependency tree

  • myprogram (native TS module)
    • mylib (native TS module, with 'proper external' typing)
      • foolib@1.0 (plain JS module)
        • utils@3.0 (plain JS module)
      • barlib@1.0 (plain JS module)
        • utils@4.0 (plain JS module)
    • myotherlib (native TS module, with 'proper external' typing)
      • foolib@2.0 (plain JS module)
        • utils@4.0 (plain JS module)

Assumptions about this tree:

  • Mainly talking about NodeJS (CommonJS) environment here, not trying to solve the bower problem
  • None of the JS modules pollute the global namespace, i.e. they are 'normal' CommonJS modules

Problem description

In this case, because mylib is the 'first' package that knows about TS, it somehow needs to provide the typings for all non-TS stuff it exposes: foolib@1.0, barlib@1.0 but also both utils@3.0 and utils@4.0.

In an ideal world, using 'proper external modules' (see #4665 and #2338) would solve this if they were all TS modules (i.e. they provide their own typings in their npm package).

In this case though, the import statements need to resolve to e.g. DefinitelyTyped typings, typically located in e.g. mylib/typings/.

So, mylib/typings/ needs to have a utils.d.ts for version 3.0 and 4.0, and the compiler needs to know how to find them, and which version to use.

Especially note the difference between the concept of the current JS module versus current TS module in the following proposal.

Proposed algorithm

Naming (taken from #4665):

  • 'ambient external' module typing means it contains declare module "Y" { ... } declarations
  • 'proper external' module typings don't have declare module "Y" { ... }, but directly export their classes, variables, etc.

When compiling a package X, and looking for typings of an external module Y/Z, let:

  • CurrentTSModule = X
  • CurrentJSModule = X
  • Z = "index" if module is imported without a path (i.e. import "Y" instead of import "Y/Z")

Now:

  • Use the logic of External module resolution logic #2338 to locate Y's package.json (starting atCurrentJSModule`)
    • i.e. also traversing node_modules of parent packages
  • If Y provides its own typing (either index.d.ts or by following typings property in package.json in the package directory)
    • Parse that typing as a 'proper external module' typing
    • Set CurrentTSModule and CurrentJSModule variables to Y, i.e. any external modules used by Y should be resolved by looking in <Y>/node_modules and <Y>/typings/, no longer in <X>/typings/
    • Done
  • Otherwise, search for typings using either:
    • ALGORITHM A: searches for Y in X's typings folder (let's call it typings/):
      • typings/<Y>@<majorY.minorY.patchY>/<Z>.d.ts (proper mode)
      • typings/<Y>@<majorY.minorY>/<Z>.d.ts (proper mode)
      • typings/<Y>@<majorY>/<Z>.d.ts (proper mode)
      • typings/<Y>/<Z>.d.ts (proper mode)
      • typings/<Y>/<Y>.d.ts (mixed mode)
      • typings/<Y>.d.ts (mixed mode)
    • ALGORITHM B: match against .d.ts files with <library ... /> tags
      • For all .d.ts files passed on commandline/tsconfig.json that have a <library ... /> tag, match Y's package.json name and version against the name and semver specification in the <library> tag
  • When a match is found:
    • Keep CurrentTSModule as X, but set CurrentJSModule to Y
      • This ensures that the correct version of any sub-package is found
    • If 'proper mode': parse as 'proper external module', done
    • If 'mixed mode': determine whether it's a 'proper external' or 'ambient external' typing:
      • If 'proper external': use it as-is, done
      • If 'ambient external':
        • Don't expose any ambient declarations (modules, namespaces, variables, types), nor recursive
          <reference>'d declarations, to the global module namespace, and
        • Look for the declare module "Y/Z" { ... } and use its contents as the result (i.e. 'convert to proper external')

Notes:

  • <majorY> etc. are based on the version as found in Y's package.json
  • In contrast to the lookup in node_modules, lookups in typings do not traverse typings/ dirs of parent packages. Only those of (TS-)package X are searched.
  • This algorithm correctly determines that foolib uses utils@3.0, but barlib uses utils@4.0 (i.e. CurrentTSModule stays mylib, but CurrentJSModule switches from foolib to barlib)

Mixed mode

Mixed mode (previously called "legacy mode"), is intended to allow existing DefinitelyTyped typings
(which use 'ambient external' scheme) to basically be used as external modules (CommonJS) without making a lot of 'accidental' globals available (e.g. the Promise type in bluebird typings), while still also allowing them to be used in AMD and plain script modes ('isomorphic typings').

These isomorphic typings typically declare lots of things as globals (perfect for plain script mode),
but these are usually not made globally available when loaded as CommonJS.

So, the idea is to wrap the whole typing into its own private space, then only make the actually requested external module part of it available.

Note that if an isomorphic typing includes a 'proper external' typing, and the proper external typing
<reference>'s another typing, that typing is still allowed to declare globals (they will not be 'isolated').

Having two packages declare the same global (e.g. when a proper external typing references a .d.ts, which explicitly marks a variable, class, etc as being globally available) currently leads to a compiler error ("Duplicate identifier").
One idea I had was to 'merge' the types of such globals instead (e.g. if two packages both declare a Promise, but they are in fact of different types, the resulting global will be typed as e.g. declare var Promise: PromiseType1|PromiseType2;
This way, the compiler can error when the global is actually used (as opposed to when declared) in an incompatible way (see #4673 (comment))..

Discussion items

  • I'm not sure whether I prefer Algorithm A or B, yet.
    Algorithm A has the advantage of being 'filesystem based', just like node's environment.
    But B supports more complex semver matches and prevents bikeshedding over e.g. the name and structure of typings/.
  • When using Algorithm A (files), @jbrantly suggested to have an extra check for ES6 typings, i.e. preferring e.g. typings/<Y>.es6.d.ts over typings/<Y>.d.ts when available. Or possibly typings/<Y>.<target>.d.ts where <target> is the --target passed to tsc. See comments below for discussions on pros/cons.
  • How to 'override' the type a global actually will have, in case of conflicting types (see Proposal for correct version lookup in deeply embedded non-TS external modules #4673 (comment))

Metadata

Metadata

Assignees

Labels

@typesRelates to working with .d.ts files (declaration/definition files) from DefinitelyTypedIn DiscussionNot yet reached consensusSuggestionAn idea for TypeScript

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions