From 457db8c233bff2931575a6f1494bbbf705e1652c Mon Sep 17 00:00:00 2001 From: Orta Therox Date: Tue, 7 Apr 2020 11:27:03 -0400 Subject: [PATCH 1/6] RFC for types in packument --- ...0026-add-types-metadata-to-package-json.md | 102 ++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 accepted/0026-add-types-metadata-to-package-json.md diff --git a/accepted/0026-add-types-metadata-to-package-json.md b/accepted/0026-add-types-metadata-to-package-json.md new file mode 100644 index 000000000..459f2d87c --- /dev/null +++ b/accepted/0026-add-types-metadata-to-package-json.md @@ -0,0 +1,102 @@ +# Add Types Metadata to the Registry [Packument](https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#package-endpoints) + +## Summary + +Add an unambiguous notation to the derived package.json whether a dependency includes types. + +## Motivation + +It would be great to know from the NPM API if a library included type definitions. This would allow the npm website, and other tooling to be able to help people make decisions about which dependencies they would like to use and support for types plays into that. + +## Detailed Explanation + +Working with one of my libraries which ships with both Flow and TypeScript types: + +[Package.json](https://github.com/danger/danger-js/blob/e7d9b6a515eabe0488a5f0ec196319c7fbe479b0/package.json) + +```json5 +{ + "name": "danger", + "version": "10.1.0", + "description": "Unit tests for Team Culture", + "main": "distribution/danger.js", + "typings": "distribution/danger.d.ts", + "bin": { + // + } + "prettier": { + // ... + }, + "scripts": { + // ... + }, + "repository": { + "type": "git", + "url": "git+https://github.com/danger/danger-js.git" + }, + "keywords": [ + "danger", + "ci" + ], + "author": "Orta Therox", + "license": "MIT", + "bugs": { + "url": "https://github.com/danger/danger-js/issues" + }, + "homepage": "https://github.com/danger/danger-js#readme", + "devDependencies": { + // ... + }, + "dependencies": { + // ... + }, +} +``` + +This `package.json` unambiguously declares that it has TypeScript types. However, a flow user has no idea that I put the time in [to maintain](https://unpkg.com/browse/danger@10.1.0/distribution/danger.js.flow) support for them! + +I propose that [the packument version of this package](https://registry.npmjs.org/danger/10.1.0) have a new field `"_types"` which would look like: + +```json5 +{ + // ... + "dependencies": { + // ... + }, + "gitHead": "66e18daae5286ea64ed49484fd5835685684c156", + "_id": "danger@10.0.0", + "_types": { + "ts": "included", + "flow": "included" + } +} + +``` + +The shape of this copies the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346). + +This RFC would explicitly _exclude_ information on 3rd party types hosted on [DefinitelyTyped](https://github.com/DefinitelyTyped/) and [FlowTyped](https://github.com/flow-typed/flow-typed) - the assumption is that anyone wanting to provide comprehensive type information for _any_ package would check their registries if no type information is present in the packument. + +### "Unambiguous" + +Both flow and TypeScript types work on projects which don't declare that they have types inside them. I could remove `"typings"` in the above `package.json` and TypeScript would exhibit the same behavior. That would mean that the current package.json would not have any indication that there is typed support for either tools. + +### Downside of Recommendation + +1. This would mean that only newly shipped packages with the latest CLI would have that information. In theory, you could backport the logic and re-run it across the whole registry... That's quite a bit of work though. +1. The naive recommendation of "check for *.d.ts / *.js.flow" could mean that projects which have types setup incorrectly could be triggered as a false positive. Personally, I think this is a tiny edge case and one likely a human would intervene on with fixing their types. The project could "have types", just not in the right places. + +## Rationale and Alternatives + +1. Don't have that metadata on the site. Both TS/Flow represent unofficial extensions of JavaScript and who knows what the future holds. +1. Ping the algolia npm index for that information. I would expect you to want to keep your server side dependencies low, and including this information in the registry would allow you to keep one source of information. + +## Implementation + +Personally, I think a simple check for the inclusion in the package tarball for any `.d.ts` file should be enough for TypeScript, and _I believe_ that a check for `.js.flow` would be enough for Flow typings. + +These can happen during `npm package` in the CLI. + +## Prior Art + +[The algolia npm-search index](https://github.com/algolia/npm-search/) - which I believe powers the search on https://www.npmjs.com From 21fc7860deced0927b408f39721a31f6806846fd Mon Sep 17 00:00:00 2001 From: Orta Date: Thu, 30 Apr 2020 12:40:57 -0400 Subject: [PATCH 2/6] Update 0026-add-types-metadata-to-package-json.md Update from the last RFC meeting --- ...0026-add-types-metadata-to-package-json.md | 65 ++++++++++--------- 1 file changed, 34 insertions(+), 31 deletions(-) diff --git a/accepted/0026-add-types-metadata-to-package-json.md b/accepted/0026-add-types-metadata-to-package-json.md index 459f2d87c..2d20a1e35 100644 --- a/accepted/0026-add-types-metadata-to-package-json.md +++ b/accepted/0026-add-types-metadata-to-package-json.md @@ -21,29 +21,7 @@ Working with one of my libraries which ships with both Flow and TypeScript types "description": "Unit tests for Team Culture", "main": "distribution/danger.js", "typings": "distribution/danger.d.ts", - "bin": { - // - } - "prettier": { - // ... - }, - "scripts": { - // ... - }, - "repository": { - "type": "git", - "url": "git+https://github.com/danger/danger-js.git" - }, - "keywords": [ - "danger", - "ci" - ], - "author": "Orta Therox", - "license": "MIT", - "bugs": { - "url": "https://github.com/danger/danger-js/issues" - }, - "homepage": "https://github.com/danger/danger-js#readme", + //... "devDependencies": { // ... }, @@ -55,7 +33,7 @@ Working with one of my libraries which ships with both Flow and TypeScript types This `package.json` unambiguously declares that it has TypeScript types. However, a flow user has no idea that I put the time in [to maintain](https://unpkg.com/browse/danger@10.1.0/distribution/danger.js.flow) support for them! -I propose that [the packument version of this package](https://registry.npmjs.org/danger/10.1.0) have a new field `"_types"` which would look like: +I propose that [the packument version of this package](https://registry.npmjs.org/danger/10.1.0) have a new field `"_typesRoot"` which would look like: ```json5 { @@ -65,17 +43,36 @@ I propose that [the packument version of this package](https://registry.npmjs.or }, "gitHead": "66e18daae5286ea64ed49484fd5835685684c156", "_id": "danger@10.0.0", - "_types": { - "ts": "included", - "flow": "included" + "_typesRoot": { + "ts": "distribution/danger.d.ts", + "flow": "distribution/danger.js.flow" } } ``` -The shape of this copies the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346). +The shape of this info is based on the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346). The information can be generated via: + +- Detecting explicit `"types"` or `"typings"` (for TS explicitly, from reading [their docs](https://flow.org/en/docs/declarations/), Flow does not have a field like this) in the manifest, and just moving that field's value across directly +- Resolving the `main` with both TypeScript and Flow's extra file resolvers (e.g. when `distribution/danger.js` look for the files `distribution/danger.js.flow` and `distribution/danger.d.ts`) +- If we got here then there are no TS/Flow files in the package, and we could do nothing or leave an empty `"_typesRoot": {}` -This RFC would explicitly _exclude_ information on 3rd party types hosted on [DefinitelyTyped](https://github.com/DefinitelyTyped/) and [FlowTyped](https://github.com/flow-typed/flow-typed) - the assumption is that anyone wanting to provide comprehensive type information for _any_ package would check their registries if no type information is present in the packument. +This RFC would explicitly _exclude_ information on 3rd party types hosted on [DefinitelyTyped](https://github.com/DefinitelyTyped/) and [FlowTyped](https://github.com/flow-typed/flow-typed) - the assumption is that anyone wanting to provide comprehensive type information for _any_ package would check their registries if no type information is present in the packument. Both registries can make it easy for others to grab the listing e.g. the [types-registry](https://www.npmjs.com/package/types-registry) package. + +
+ Note: The original version of this proposal used boolean values... + +Which assumed that API clients would do some of the work themselves to get the root of the types: + +```json +"_types": { + "ts": "included", + "flow": "included" +} +``` + +We can already do the work ahead of time, and `_typeRoots` is less likely to have conflicts in the registry I imagine. +
### "Unambiguous" @@ -88,8 +85,14 @@ Both flow and TypeScript types work on projects which don't declare that they ha ## Rationale and Alternatives -1. Don't have that metadata on the site. Both TS/Flow represent unofficial extensions of JavaScript and who knows what the future holds. -1. Ping the algolia npm index for that information. I would expect you to want to keep your server side dependencies low, and including this information in the registry would allow you to keep one source of information. +1. Use the `npm package` command to make maintainers explicitly fill out the `"types"` field when a `.d.ts` can be inferred from `"main"` via a warning for a few releases, then a fail. + + This could work, however it would favour TypeScript today. There isn't a similar manifest field for Flow, though they could definitely pick one and roll with it also. + + This would mean that we don't need to add reserved fields to the packument, and these fields becomes a cultural norm. The downside is that this puts work on existing maintainers to amend their package.jsons. + +1. Don't have that metadata on the site. Both TS/Flow represent unofficial extensions of JavaScript and who knows what the future holds. +1. Ping the algolia npm index for that information. I would expect you to want to keep your server side dependencies low, and including this information in the registry would allow you to keep one source of information. ## Implementation From fe8ffe0fb1429bab47c9fdadfb4cc5a05b5b8126 Mon Sep 17 00:00:00 2001 From: Orta Date: Thu, 30 Apr 2020 12:43:50 -0400 Subject: [PATCH 3/6] Change the prior art because algolia doesn't power npm search --- accepted/0026-add-types-metadata-to-package-json.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/0026-add-types-metadata-to-package-json.md b/accepted/0026-add-types-metadata-to-package-json.md index 2d20a1e35..b363cf527 100644 --- a/accepted/0026-add-types-metadata-to-package-json.md +++ b/accepted/0026-add-types-metadata-to-package-json.md @@ -102,4 +102,4 @@ These can happen during `npm package` in the CLI. ## Prior Art -[The algolia npm-search index](https://github.com/algolia/npm-search/) - which I believe powers the search on https://www.npmjs.com +[The algolia npm-search index](https://github.com/algolia/npm-search/) - which is currently the best index of what packages have types included in their modules From 2adf14d7615e2779beaf6bd2b7c3a15cea9c8379 Mon Sep 17 00:00:00 2001 From: Orta Therox Date: Wed, 27 May 2020 15:45:04 -0400 Subject: [PATCH 4/6] Minor tweaks from last RFC meeting --- accepted/0026-add-types-metadata-to-package-json.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/accepted/0026-add-types-metadata-to-package-json.md b/accepted/0026-add-types-metadata-to-package-json.md index b363cf527..0ea1b10b3 100644 --- a/accepted/0026-add-types-metadata-to-package-json.md +++ b/accepted/0026-add-types-metadata-to-package-json.md @@ -51,9 +51,9 @@ I propose that [the packument version of this package](https://registry.npmjs.or ``` -The shape of this info is based on the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346). The information can be generated via: +The shape of this info is based on the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346) but different. The information can be generated automatically via: -- Detecting explicit `"types"` or `"typings"` (for TS explicitly, from reading [their docs](https://flow.org/en/docs/declarations/), Flow does not have a field like this) in the manifest, and just moving that field's value across directly +- Detecting explicit `"types"` or `"typings"` (for TS explicitly, from reading [their docs](https://flow.org/en/docs/declarations/), Flow does not have a field like this) in the manifest, and just moving that field's value across directly. - Resolving the `main` with both TypeScript and Flow's extra file resolvers (e.g. when `distribution/danger.js` look for the files `distribution/danger.js.flow` and `distribution/danger.d.ts`) - If we got here then there are no TS/Flow files in the package, and we could do nothing or leave an empty `"_typesRoot": {}` @@ -96,7 +96,7 @@ Both flow and TypeScript types work on projects which don't declare that they ha ## Implementation -Personally, I think a simple check for the inclusion in the package tarball for any `.d.ts` file should be enough for TypeScript, and _I believe_ that a check for `.js.flow` would be enough for Flow typings. +Personally, I think a check for the inclusion in the package tarball for any `.d.ts` file should be enough for TypeScript, and _I believe_ that a check for `.js.flow` would be enough for Flow typings. These can happen during `npm package` in the CLI. From 4d5c903618f0eb0b16141f0b3cd419f3782d57cb Mon Sep 17 00:00:00 2001 From: Orta Therox Date: Wed, 17 Jun 2020 14:54:13 -0400 Subject: [PATCH 5/6] Updates from June 17th feedback --- accepted/0026-add-types-metadata-to-package-json.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/accepted/0026-add-types-metadata-to-package-json.md b/accepted/0026-add-types-metadata-to-package-json.md index 0ea1b10b3..6d8fa90ab 100644 --- a/accepted/0026-add-types-metadata-to-package-json.md +++ b/accepted/0026-add-types-metadata-to-package-json.md @@ -44,8 +44,8 @@ I propose that [the packument version of this package](https://registry.npmjs.or "gitHead": "66e18daae5286ea64ed49484fd5835685684c156", "_id": "danger@10.0.0", "_typesRoot": { - "ts": "distribution/danger.d.ts", - "flow": "distribution/danger.js.flow" + "ts": "./distribution/danger.d.ts", + "flow": "./distribution/danger.js.flow" } } @@ -54,11 +54,16 @@ I propose that [the packument version of this package](https://registry.npmjs.or The shape of this info is based on the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346) but different. The information can be generated automatically via: - Detecting explicit `"types"` or `"typings"` (for TS explicitly, from reading [their docs](https://flow.org/en/docs/declarations/), Flow does not have a field like this) in the manifest, and just moving that field's value across directly. -- Resolving the `main` with both TypeScript and Flow's extra file resolvers (e.g. when `distribution/danger.js` look for the files `distribution/danger.js.flow` and `distribution/danger.d.ts`) +- Resolving the `main` with both TypeScript and Flow's extra file resolvers (e.g. when `distribution/danger.js` look for the files `./distribution/danger.js.flow` and `./distribution/danger.d.ts`) - If we got here then there are no TS/Flow files in the package, and we could do nothing or leave an empty `"_typesRoot": {}` This RFC would explicitly _exclude_ information on 3rd party types hosted on [DefinitelyTyped](https://github.com/DefinitelyTyped/) and [FlowTyped](https://github.com/flow-typed/flow-typed) - the assumption is that anyone wanting to provide comprehensive type information for _any_ package would check their registries if no type information is present in the packument. Both registries can make it easy for others to grab the listing e.g. the [types-registry](https://www.npmjs.com/package/types-registry) package. +##### Path handling + +When handling a custom `main`, `types`, or `typings`: if the path is a relative path to the root, the `_typesRoot` version should verify that the file exists and add a `./` as a prefix. +This allows for downstream consumers to be able to differentiate between a module hosting their types and a file available inside the package. +
Note: The original version of this proposal used boolean values... @@ -76,7 +81,7 @@ We can already do the work ahead of time, and `_typeRoots` is less likely to hav ### "Unambiguous" -Both flow and TypeScript types work on projects which don't declare that they have types inside them. I could remove `"typings"` in the above `package.json` and TypeScript would exhibit the same behavior. That would mean that the current package.json would not have any indication that there is typed support for either tools. +Both flow and TypeScript types work on projects which don't declare in the `package.json` that they have types. I could remove `"typings"` in the above `package.json` and TypeScript would exhibit the same behavior. That would mean that the current `package.json` would not have any indication that there is typed support for either tools. ### Downside of Recommendation From 784edcd815740edbc7cc1edd2d09c58136a2af38 Mon Sep 17 00:00:00 2001 From: Orta Therox Date: Wed, 15 Jul 2020 15:06:00 -0400 Subject: [PATCH 6/6] Update 0026-add-types-metadata-to-package-json.md --- ...0026-add-types-metadata-to-package-json.md | 34 +++++++++---------- 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/accepted/0026-add-types-metadata-to-package-json.md b/accepted/0026-add-types-metadata-to-package-json.md index 6d8fa90ab..7dce8695c 100644 --- a/accepted/0026-add-types-metadata-to-package-json.md +++ b/accepted/0026-add-types-metadata-to-package-json.md @@ -33,36 +33,30 @@ Working with one of my libraries which ships with both Flow and TypeScript types This `package.json` unambiguously declares that it has TypeScript types. However, a flow user has no idea that I put the time in [to maintain](https://unpkg.com/browse/danger@10.1.0/distribution/danger.js.flow) support for them! -I propose that [the packument version of this package](https://registry.npmjs.org/danger/10.1.0) have a new field `"_typesRoot"` which would look like: +I propose that [the packument version of this package](https://registry.npmjs.org/danger/10.1.0) we emulate the file resolvers for TS + Flow and ensure their fields are filled if they are not present. For example, when publishing a field for "flow" would be added: -```json5 +```diff { // ... "dependencies": { // ... }, - "gitHead": "66e18daae5286ea64ed49484fd5835685684c156", - "_id": "danger@10.0.0", - "_typesRoot": { - "ts": "./distribution/danger.d.ts", - "flow": "./distribution/danger.js.flow" - } ++ "gitHead": "66e18daae5286ea64ed49484fd5835685684c156", ++ "flow": "./distribution/danger.js.flow" } - ``` -The shape of this info is based on the work we did in [Algolia's npm search index](https://github.com/algolia/npm-search/pull/346) but different. The information can be generated automatically via: +- When there is an explicit `"types"` or `"typings"` (for TS explicitly, from reading [their docs](https://flow.org/en/docs/declarations/), do nothing. Flow does not have a field like this) in the manifest, and so this would _add_ that field. + +- If there is not the explicit fields: Resolving the `main` with both TypeScript and Flow's extra file resolvers (e.g. when `distribution/danger.js` look for the files `./distribution/danger.js.flow` and `./distribution/danger.d.ts`) -- Detecting explicit `"types"` or `"typings"` (for TS explicitly, from reading [their docs](https://flow.org/en/docs/declarations/), Flow does not have a field like this) in the manifest, and just moving that field's value across directly. -- Resolving the `main` with both TypeScript and Flow's extra file resolvers (e.g. when `distribution/danger.js` look for the files `./distribution/danger.js.flow` and `./distribution/danger.d.ts`) -- If we got here then there are no TS/Flow files in the package, and we could do nothing or leave an empty `"_typesRoot": {}` +- If we got here then there are no TS/Flow files in the package, do nothing -This RFC would explicitly _exclude_ information on 3rd party types hosted on [DefinitelyTyped](https://github.com/DefinitelyTyped/) and [FlowTyped](https://github.com/flow-typed/flow-typed) - the assumption is that anyone wanting to provide comprehensive type information for _any_ package would check their registries if no type information is present in the packument. Both registries can make it easy for others to grab the listing e.g. the [types-registry](https://www.npmjs.com/package/types-registry) package. ##### Path handling -When handling a custom `main`, `types`, or `typings`: if the path is a relative path to the root, the `_typesRoot` version should verify that the file exists and add a `./` as a prefix. -This allows for downstream consumers to be able to differentiate between a module hosting their types and a file available inside the package. +This RFC would always add the relative path to an existing file if the fields are empty. +This allows for downstream consumers to be able to differentiate between a module hosting their types and a file available inside the package.
Note: The original version of this proposal used boolean values... @@ -81,7 +75,11 @@ We can already do the work ahead of time, and `_typeRoots` is less likely to hav ### "Unambiguous" -Both flow and TypeScript types work on projects which don't declare in the `package.json` that they have types. I could remove `"typings"` in the above `package.json` and TypeScript would exhibit the same behavior. That would mean that the current `package.json` would not have any indication that there is typed support for either tools. +Both flow and TypeScript types work on projects which don't declare in the `package.json` that they have types. + +For example, I could remove `"typings"` in the above `package.json` and TypeScript would exhibit the same behavior. That would mean that the current `package.json` would not have any indication that there is typed support for either tools. + +This ambiguity makes it hard for someone wanting to build tools which offer information about whether the package offers types. ### Downside of Recommendation @@ -101,7 +99,7 @@ Both flow and TypeScript types work on projects which don't declare in the `pack ## Implementation -Personally, I think a check for the inclusion in the package tarball for any `.d.ts` file should be enough for TypeScript, and _I believe_ that a check for `.js.flow` would be enough for Flow typings. +Personally, I think a check for the inclusion in the package tarball for any `.d.ts` file should be enough for TypeScript, and the check for `.js.flow` would be enough for Flow typings. These can happen during `npm package` in the CLI.