Separate types from code and make types composable/extendable

[gr2m]

Currently all JS Octokit modules depend on @ocotkit/types. @octokit/types include definitions for all 600+ REST API endpoints for api.github.com and primitives such as types for Requests and responses that are not extendable.

Problems

1. All endpoint types are for api.github.com. They cannot be replaced for GHAE or a specific GHES version. They cannot be imported selectively.
2. The types cannot be extended by plugins. Plugins often time introduce new Octokit options such as throttle, extend request options and/or responses. Authentication strategies are a special kind of plugins that provide custom auth options, but they cannot define the options in a way that would provide IntelliSense to users of the authentication strategy. Plugins also hook into the request life cycle and amend requests/responses, but the types for requests/responses provided by @octokit/types cannot be extended at this point.

Solution

Take advantage of declaration merging and global augmentation to make it possible for imported code to amend the global Octokit Types.

For the endpoint types, I see two approaches:

1. Create entirely separate types for api.github.com, GHES-3.0, GHES-3.1
2. Use types for api.github.com as the starting point, then create types for the GHES/GHAE versions as an extension of that. Endpoints that do not (yet) exist in the target version can be set to never with an explanation. If the endpoint exists in both environments but differs in its request parameters or response, create a union of both types. We could introduce a new option such as request.version which defaults to "api.github.com" but which can be set to whatever version of the REST API endpoint types were loaded. So when it's set to "ghes-3.1" then the types from that version are used explicitly, instead of the union between the types of api.github.com and ghes-3.1.

I'd prefer the 2nd approach as I'd expect it to be a better developer experience.

For developers who write code that is meant to run against both api.github.com and a minimal version of GHES, we could create virtual versions such as ghes-3.0-compatible, which will either only include types that are covered by both environments, or make clear which endpoints/properties do not exist in either.

Benefits

• Make developing for GHAE and GHES a first-class experience

Steps

@octokit-next/* packages

• Create a new octokit-next.js repository with the minimal set of features needed to implement the extendable & composable types. I already started working on parts of it in https://github.com/gr2m/javascript-plugin-architecture-with-typescript-definitions/. octokit-next.js could build upon that to test the composability features of Base.defaults() and Base.plugins() and implement a .request() method which implements a few of GitHub's endpoints to test a setup that works against different environments (api.github.com, ghes-3.1, ghes-3.0)
  • extendable constructor options: feat: extend Base constructor options with plugins gr2m/javascript-plugin-architecture-with-typescript-definitions#56
  • inherit constructor options types when calling .defaults(): feat: derive constructor options from chainable Base.defaults() calls gr2m/javascript-plugin-architecture-with-typescript-definitions#57
  • Create octokit-next.js repository with an octokit.request method which utilizes endpoint types
  • Make the Octokit constructor derive types for octokit.request based on the version option passed to the constructor octokit-next.js#3
• Implement types for 3-5 endpoints that have differences across api.github.com, ghes-3.1, ghes-3.0. Make the types for api.github.com load by default.
• Add options.version parameter for the Octokit constructor and a options.request.version option when calling octokit.request(options). The type should be an extendable enum. By default it will only permit api.github.com. When importing types for ghes-3.0, it can be set to api.github.com and ghes-3.0 (and if it makes the implementation easier because of the cascade: ghes-3.1)
• Add the authentication strategy options (authStrategy, auth) and make the types for auth depend on what authStrategy is set to, either as part of the same options object or via .defaults() octokit-next.js#18
• Create types for ghes-3.1-compatible and ghes-3.0-compatible versions octokit-next.js#14
• Generate types for all endpoints of github.com, ghes-3.1, and ghes-3.0 octokit-next.js#15
• Generate the ghes-3.1-compatible and ghes-3.0-compatible versions octokit-next.js#16
• Create & generate types for ghae and ghae-compatible octokit-next.js#17
• implement the full functionality of @octokit/endpoint in @octokit-next/endpoint octokit-next.js#36
• implement the full functionality of @octokit/request in @octokit-next/request octokit-next.js#41
• implement the full functionality of @octokit/graphql in @octokit-next/graphql octokit-next.js#70
• implement the full functionality of @octokit/core in @octokit-next/core octokit-next.js#72
• Apply all core package changes since October '21 octokit-next.js#63
• Automate types for REST API endpoints octokit-next.js#88
• ESM-ify all official authentication strategies
• ESM-ify octokit
  • @octokit/oauth-methods
  • @octokit/oauth-app
  • @octokit/webhooks-methods
  • @octokit/webhooks
  • @octokit/app
  • @octokit/plugin-paginate-rest
  • @octokit/plugin-rest-endpoint-methods
  • @octokit/plugin-retry
  • @octokit/plugin-throttling

Implement changes in Octokit

• Setup octokit/types-rest-api.ts repository as a monorepo that publishes all the @octokit/types-rest-api-* packages with all the update-automation
• Migrate @octokit/types to the new version in the beta branch
• Migrate @octokit/endpoint (the lowest-level @octokit/* package that uses the endpoints types) in the beta branch
• Migrate @octokit/request in the beta branch
• Migrate @octokit/graphql in the beta branch
• Migrate @octokit/core in the beta branch
• Migrate the plugins loaded by octokit. Extend Octokit.Options as needed
• Migrate the @octokit/* authentication strategies
• Migrate octokit
• Migrate or rather re-create @octokit/plugin-enterprise-server to utilize the new types. Only generate methods for the latest GHES version that do not exist on github.com

Backlog

• remove the default GET / route types. Instead implement an override of request(route, parameters) that only applies if Octokit.ApiVersions["github.com"] is an empty object (if that is possible at all). Use that override's description to recommend installing one of the types packages

[gr2m]

This will unlock octokit/core.js#323

[gr2m]

For the curious ones, here's what this is all about

2021-08-16.16-07-49.mp4

[gr2m]

Another update, in 4 parts, because

1. There is more to show
2. I'm giving TypeScript a hard time

Part I: upcoming @octokit/core without any REST API types and with types for github.com

part-1.mp4

Part II: types for GitHub Enterprise Server (GHES)

part-2.mov

Part III: set API version on new Octokit() constructor

part-3.mp4

Part IV: types for GHES-compatible versions

part-4.mp4