Adds 'lib' reference directives #23893

rbuckton · 2018-05-04T17:50:36Z

This change adds support for a /// <reference lib="name" /> directive, allowing a file to explicitly include an existing built-in lib file.

When a built-in lib is referenced via the lib reference directive, any no-default-lib reference directive in the lib file is ignored. This is so that a lib reference in an imported package won't cause your current project to lose its default lib if you haven't specified one.
Built-in lib files are referenced in the same fashion as the "lib" compiler option in tsconfig.json (e.g. use lib="es2015" and not lib="lib.es2015.d.ts", etc.).

In the long term this can help polyfill/shim packages like core-js or es6-shim.

NOTE: Apparently we were very close to running out of heap memory when running gulp tests after running gulp clean as gulp is holding onto too much memory. Individually each task is fine, but in concert they add up to a large amount of uncollected memory. To address this, I've added a script to run some of our builds out-of-process. This is not an issue in the jake builds.

Fixes #15732.
Supersedes #15780

ghost · 2018-05-04T21:43:50Z

src/compiler/parser.ts

                    forEach(toArray(entryOrList), (arg: PragmaPsuedoMap["reference"]) => {
                        if (arg.arguments["no-default-lib"]) {
                            context.hasNoDefaultLib = true;
                        }
                        else if (arg.arguments.types) {
                            typeReferenceDirectives.push({ pos: arg.arguments.types.pos, end: arg.arguments.types.end, fileName: arg.arguments.types.value });
                        }
+                        else if (arg.arguments.lib) {


#23904 Could avoid repeated code here

Do you want me to do this or do you want to do in your other PR once this is merged?

I'll close the other PR since it doesn't have much benefit with only 2 branches, but maybe it would have a benefit with 3 branches, or there's a better way to share the code.

ghost · 2018-05-08T22:03:58Z

tests/baselines/reference/arrayFrom.types

 >Array : ArrayConstructor
->from : { <T>(iterable: Iterable<T> | ArrayLike<T>): T[]; <T, U>(iterable: Iterable<T> | ArrayLike<T>, mapfn: (v: T, k: number) => U, thisArg?: any): U[]; <T>(arrayLike: ArrayLike<T>): T[]; <T, U>(arrayLike: ArrayLike<T>, mapfn: (v: T, k: number) => U, thisArg?: any): U[]; }
+>from : { <T>(arrayLike: ArrayLike<T>): T[]; <T, U>(arrayLike: ArrayLike<T>, mapfn: (v: T, k: number) => U, thisArg?: any): U[]; <T>(iterable: Iterable<T> | ArrayLike<T>): T[]; <T, U>(iterable: Iterable<T> | ArrayLike<T>, mapfn: (v: T, k: number) => U, thisArg?: any): U[]; }


Shouldn't arrayFrom.ts have a Set test? Then if would break due to this change since Set is Iterable but not ArrayLike.

I'll add a test, but this shouldn't break because there is still an overload that supports Iterable. The order just changed because I cleaned up the default order of libs. Previously if you had a "lib": ["es5", "es2015"] in your tsconfig.json we would actually end up inserting them in reverse order because we were doing files.unshift for default libs.

NN--- · 2018-05-20T18:36:12Z

Will this allow fixing #16077 ?

ghost · 2018-05-29T21:25:13Z

src/compiler/core.ts

@@ -1749,7 +1749,7 @@ namespace ts {
     * Case-sensitive comparisons compare both strings one code-point at a time using the integer
     * value of each code-point after applying `toUpperCase` to each string. We always map both
     * strings to their upper-case form as some unicode characters do not properly round-trip to
-     * lowercase (such as `ẞ` (German sharp capital s)).
+     * lowercase (such as `áºž` (German sharp capital s)).


Was this intentional?

Not sure what caused that to change, but I'll revert.

mhegazy · 2018-05-30T17:38:25Z

src/compiler/commandLineParser.ts

+    // Internally we add some additional lib references that we only support when used as part of a
+    // "lib" reference directive. They are not available on the command line or in tsconfig.json.
+    /* @internal */
+    export const libMap = cloneMap(commandLineLibMap)


i would just add this to the map. chances are no one will ever include it.. and if they did.. it does not break anything.

mhegazy · 2018-05-30T17:44:34Z

src/lib/es2015.full.d.ts

@@ -0,0 +1,14 @@
+/// <reference lib="es5" />


should not this just be:

/// <reference lib="es2015" /> /// <reference lib="dom" /> /// <reference lib="dom.iterable" /> /// <reference lib="webworker.importscripts" /> /// <reference lib="scripthost" />

mhegazy · 2018-05-30T17:47:11Z

src/lib/esnext.full.d.ts

+/// <reference lib="dom" />
+/// <reference lib="webworker.importscripts" />
+/// <reference lib="scripthost" />
+/// <reference lib="dom.iterable" />


nit. i would move this just after dom

mhegazy · 2018-05-30T17:52:01Z

src/lib/webworker.generated.d.ts

@@ -1,3 +1,5 @@
+/// <reference lib="webworker.importscripts" />


this is not needed here, the file is autogenerated, and it already has the correct definition. we add it to lib.d.ts to partially allow the lib to be used in webworkers for simple cases..

the two files dom and webworker have too many conflicts in between them, and can not be loaded together in the same context anyways..

so I would just revert the changes to this file, and leave it as is.

mhegazy · 2018-05-30T17:54:53Z

src/lib/libs.json

@@ -46,57 +47,5 @@
        "webworker.generated": "lib.webworker.d.ts",
        "es5.full": "lib.d.ts",
        "es2015.full": "lib.es6.d.ts"
-    },
-    "sources": {


can we remove the support for sources from jake and gulp as well.

mhegazy · 2018-05-30T18:06:08Z

src/compiler/commandLineParser.ts

@@ -1,6 +1,55 @@
 namespace ts {
    /* @internal */
    export const compileOnSaveCommandLineOption: CommandLineOption = { name: "compileOnSave", type: "boolean" };
+
+    // NOTE: The order here is important to default lib ordering


i would expand on this comment and say that the order of the list here is the same order the lib files have in the generated program, and add a note to see use in createProgram

mhegazy · 2018-05-31T19:40:05Z

src/services/goToDefinition.ts

@@ -89,6 +89,7 @@ namespace ts.GoToDefinition {
    }

    export function getReferenceAtPosition(sourceFile: SourceFile, position: number, program: Program): { fileName: string, file: SourceFile } | undefined {
+        debugger;


mhegazy

one debugger statement that needs to be removed

styfle · 2018-06-04T23:40:22Z

Great job with this PR!

The diff won't load for me so I'll just ask here 🤔 ...

Does this PR actually implement <reference lib> in such a way to make the TS engine smaller? Or is this PR just implementing the feature itself and the usage will be implemented in a different PR?

For more context see: #23339 (comment)

Thanks! 👍

mhegazy · 2018-06-05T00:07:43Z

lib folder should go down by 4 MB or so.

RyanCavanaugh · 2018-06-05T22:35:54Z

@mhegazy @rbuckton This is going to be a big break to API consumers loading just lib.d.ts from the TS and expecting things to work - what's the mitigation here for APIs loading e.g. a fixed set of files from the network on load?

weswigham · 2018-06-05T22:53:18Z

You could always just use an older copy of the lib, if you're one of those API consumers. We ship the latest one with the latest features for consumers, but if you have constraints that force you to, usually you can just include an older copy of the lib in your project - a bunch of our older RWC tests do that.

mhegazy · 2018-06-06T00:38:11Z

If we get feedback on that we can ship an all in one file as well. The biggest issue is tool authors really, like TS-loader for instance. Most users use the built in lib with no changes.

Bnaya · 2018-06-09T15:51:23Z

Will it add the library to the entire program or only from the reference point and downward?

mhegazy · 2018-06-11T17:44:05Z

not sure i understand the question. the only difference is instead of on file lib.d.ts you will have a set of them in your program.

RyanCavanaugh · 2018-06-13T17:24:07Z

@rbuckton @mhegazy this has broken our own build because buildProtocol builds up its program with --noResolve. Checking out fbeb58a and running jake LKG twice fails.

Now noResolve effectively means you don't load any standard library types at all, which will be visible to more than just API consumers.

mhegazy · 2018-06-13T17:32:10Z

Looks like we need to ignore noResolve when we are resolving lib files as well.. we already have a ignoreNoDefaultLib flag, so we can extend this to mean resolvingLibReference and ignore noResolve as well as noDefaultLib

RyanCavanaugh · 2018-06-13T17:33:50Z

--disableIgnoreSkipNoResolve

rbuckton · 2018-06-13T18:31:30Z

I'll take a look. Lib refs are supposed to ignore noResolve.

rbuckton · 2018-06-13T23:45:57Z

This had nothing to do with noResolve and more to do with buildProtocol.ts looking only ignoring "lib.d.ts". It also looks like its already fixed in master.

rbuckton requested review from weswigham, mhegazy and a user May 4, 2018 17:50

rbuckton mentioned this pull request May 4, 2018

Add support for 'lib' reference directive #15780

Closed

ghost mentioned this pull request May 4, 2018

Share code between typeReferenceDirectives and referencedFiles parsing #23904

Closed

ghost reviewed May 4, 2018

View reviewed changes

ghost mentioned this pull request May 4, 2018

Remove unused interface #23905

Merged

rbuckton added 6 commits May 4, 2018 15:51

Move getSpellingSuggestion to core

b5540a5

add libMap and libs exports

c05923b

Add 'lib' reference support

a089297

Fork gulp-typescript tasks to run out-of-process

f4ca7ee

Fix typo in comment

8d230fc

Accept new symbol baselines, ensure expected ordering for default libs

68c0e52

rbuckton force-pushed the libReference branch from ce2f777 to 68c0e52 Compare May 5, 2018 00:24

ghost reviewed May 8, 2018

View reviewed changes

Add additional Array.from test case

6c35abe

rbuckton force-pushed the libReference branch from 7d5c1bc to 99cb2ad Compare May 9, 2018 18:40

Merge branch 'master' into libReference

698f229

rbuckton force-pushed the libReference branch from 99cb2ad to 698f229 Compare May 9, 2018 18:41

Merge branch 'master' into libReference

8548fb8

rbuckton mentioned this pull request May 25, 2018

Remove dependency on types not in ES5 DefinitelyTyped/DefinitelyTyped#25882

Merged

Merge branch 'master' into libReference

ec0af20

ghost reviewed May 29, 2018

View reviewed changes

mhegazy reviewed May 30, 2018

View reviewed changes

rbuckton added 4 commits May 31, 2018 09:50

PR Feedback

7de55fb

Merge branch 'master' into libReference

b4e113b

Fix preProcessFile tests

cee2920

Updated baselines

159e73c

mhegazy reviewed May 31, 2018

View reviewed changes

mhegazy approved these changes May 31, 2018

View reviewed changes

rbuckton added 2 commits June 4, 2018 13:42

Remove debugger statement

fb09aeb

Merge branch 'master' into libReference

6668412

rbuckton merged commit fbeb58a into master Jun 4, 2018

mhegazy deleted the libReference branch June 5, 2018 00:03

ghost mentioned this pull request Jun 5, 2018

runtests-browser cause heap out of memory error #24697

Closed

ericdrobinson mentioned this pull request Jun 13, 2018

Add es3.d.ts #16077

Closed

ghost mentioned this pull request Jun 19, 2018

Compiler options have different effects on command line vs in fourslash test #25081

Closed

timocov mentioned this pull request Jun 29, 2018

Fix detecting TypeScript default lib files timocov/dts-bundle-generator#48

Merged

brikou mentioned this pull request Sep 12, 2018

feat: remove dom dependency graffle-js/graffle#108

Merged

ghost mentioned this pull request Oct 8, 2018

@types/node@10.10.1 throws Error: Cannot find name 'Set' DefinitelyTyped/DefinitelyTyped#28969

Closed

4 tasks

This was referenced Feb 8, 2019

Availability of Node Slack SDK v5 (timeline for end of support on v4) slackapi/node-slack-sdk#689

Closed

Update to TypeScript 3.3 slackapi/node-slack-sdk#693

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds 'lib' reference directives #23893

Adds 'lib' reference directives #23893

rbuckton commented May 4, 2018

ghost May 4, 2018

rbuckton May 5, 2018

ghost May 7, 2018

ghost May 8, 2018

rbuckton May 9, 2018

NN--- commented May 20, 2018

ghost May 29, 2018

rbuckton May 30, 2018

mhegazy May 30, 2018

mhegazy May 30, 2018 •

edited

Loading

mhegazy May 30, 2018

mhegazy May 30, 2018

mhegazy May 30, 2018

mhegazy May 30, 2018

mhegazy May 31, 2018

mhegazy left a comment

styfle commented Jun 4, 2018

mhegazy commented Jun 5, 2018

RyanCavanaugh commented Jun 5, 2018

weswigham commented Jun 5, 2018

mhegazy commented Jun 6, 2018

Bnaya commented Jun 9, 2018

mhegazy commented Jun 11, 2018

RyanCavanaugh commented Jun 13, 2018

mhegazy commented Jun 13, 2018

RyanCavanaugh commented Jun 13, 2018

rbuckton commented Jun 13, 2018

rbuckton commented Jun 13, 2018

		@@ -1,3 +1,5 @@
		/// <reference lib="webworker.importscripts" />

Adds 'lib' reference directives #23893

Adds 'lib' reference directives #23893

Conversation

rbuckton commented May 4, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NN--- commented May 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhegazy May 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhegazy left a comment

Choose a reason for hiding this comment

styfle commented Jun 4, 2018

mhegazy commented Jun 5, 2018

RyanCavanaugh commented Jun 5, 2018

weswigham commented Jun 5, 2018

mhegazy commented Jun 6, 2018

Bnaya commented Jun 9, 2018

mhegazy commented Jun 11, 2018

RyanCavanaugh commented Jun 13, 2018

mhegazy commented Jun 13, 2018

RyanCavanaugh commented Jun 13, 2018

rbuckton commented Jun 13, 2018

rbuckton commented Jun 13, 2018

mhegazy May 30, 2018 •

edited

Loading