TypeScript Definition Generation #14
Conversation
|
Naturally, this will need to be squashed prior to merging. I'm not doing that right now, since my modifications to dropbox-js-sdk reference a particular commit hash, and the TypeScript examples reference a particular commit hash of my SDK modifications (and I'm too lazy to go about changing it). |
|
I added some details on how the generator is used. |
|
+@qimingyuan: Want to check out this TS? |
|
I forgot to mention explicitly in this PR: You need TypeScript 2.0. It has a release candidate, but is not officially stable yet. The TS examples I linked to have a |
|
So the end user needs to construct a FileReference object when server accepts Resource and construct a File object when server accepts File? This seems a little confusing. Is it possible to let user able to construct same object no matter server accepts base or subtype? There are two possible options:
|
It is confusing, but that's how Stone's type system works (from what I understand). Anything different would be incorrect; I can't change how the Dropbox API works! :) Also, AFAIK there are no APIs that accept these references as an argument. The Dropbox server produces them and sends them as results. The developer should not have to construct them themselves. If they did, then it's the API designer's fault for making a confusing API.
Here's what I mean, concretely: // This is a nominal type.
function Dog() {
this.sound = "Bark!";
}
var spot = new Dog();
spot instanceof Dog; // true
// Object literal
var spot2 = {
sound: "Bark!"
};
spot2 instanceof Dog; // false, even though it has all of the same fieldsStone is designed so that developers use the |
I feel like this is more confusing. A user will probably write code like the following, which never even mentions the switch (resource['.tag']) {
case 'file':
// A file.FileReference is a supertype of file.File,
// so this is A-OK! :)
let fileResource = <file.File> resource;
} |
|
Oh I am surprised that you can explicit cast an object literal to a nominal typed object but instanceof check would still fail, which means developers should be instructed to not use instanceof check at all here. Not sure about how long the assumption will hold for input parameters, since the return type for get_metadata route is defined as MetadataReference|FileMetadataReference|FolderMetadataReference, I feel like the most intuitive action for developer would be casting to FileMetadataReference instead of FileMetadata. |
You'll be happy to hear that the TypeScript compiler will not let you perform an
I'm not sure what your concern is here. The TypeScript definition files that I've produced model Stone's data types accurately. If a company like Dropbox uses them in a confusing manner, then there's nothing I can do to simplify it. I agree that Stone's data types can get complicated, but my hands are tied.
Well, then they cast to |
|
Assuming this gets merged in - can we get some test cases (or at least a task to add some)? |
|
@wittekm I would've added some, but it looks like there are no tests for the JavaScript generator right now. My approach would've been to use this generator to typecheck tests for the JavaScript generator. |
|
I know it's been some time since there was activity here, but I'd like to see what I can do to get this merged. Aside from unit tests, @qimingyuan, have your concerns been addressed, or are there outstanding issues? |
|
Oh sorry for the delay and thanks for reviving the thread.I am fine with the proposal as it accurately describes the stone spec and we should get this in. |
|
@jvilk, can you squash with a consolidated comment, and I'll merge this? Thanks very much for the contribution, by the way, and my apologies for taking so long to get to it! |
jvilk
force-pushed
the
tsd-types-generator
branch
from
bc6ca24
to
fd6a3a7
Compare
|
@posita done! Let me know if you have any other particular requests. You may want to take a quick pass over the code to make sure I'm following all of your style conventions, since I don't think a basic style check has been done. |
|
Also, if you ever introduce JavaScript generator tests, let me know. We can re-use those to test the TypeScript typings generator. |
Adds a generator that produces TypeScript definition files for a JavaScript client. As with JavaScript, there are two generators: `tsd_types` and `tsd_client`.
The definition files require TypeScript 2.0, as they rely upon TypeScript tagged unions.
Overview: Mapping Stone Types to TypeScript
===========================================
Below, I will summarize how we map Stone types to TypeScript.
Basic Types
-----------
TypeScript's basic types match JSDoc, so there is no difference from the `js_types` generator.
Alias
-----
Aliases are emitted as `type`s:
``` typescript
type AliasName = ReferencedType;
```
Struct
------
Structs are emitted as `interface`s, which support inheritance. Thus, if a struct `A` extends struct `B`, it will be emitted as:
``` typescript
interface A extends B {
// fields go here
}
```
Nullable fields and fields with default values are emitted as _optional_ fields. In addition, the generator adds a field description with the default field value, if the field has one:
``` typescript
interface A {
// Defaults to False
recur?: boolean;
}
```
Unions
------
Unions are emitted as a `type` that is the disjunction of all possible union variants (including those from parent types!). Each variant is emitted as an individual `interface`.
```
union Shape
point
square Float64
"The value is the length of a side."
circle Float64
"The value is the radius."
```
``` typescript
interface ShapeCircle {
.tag: 'circle';
circle: number;
}
interface ShapeSquare {
.tag: 'square';
square: number;
}
interface ShapePoint {
.tag: 'point';
}
type Shape = ShapePoint | ShapeSquare | ShapeCircle;
```
TypeScript 2.0 supports [tagged union types](https://github.com/Microsoft/TypeScript/wiki/What%27s-new-in-TypeScript#tagged-union-types) like these ones, so the compiler should automatically infer the type of a shape when the developer writes code like so (and statically check that all cases are covered!):
``` typescript
var shape: Shape = getShape();
switch (shape['.tag']) {
case 'point':
console.log('point');
break;
case 'square':
// Compiler knows this is a ShapeSquare, so .square field is visible.
console.log('square ' + shape.square);
break;
// No 'circle' case! If developer enables the relevant compiler option, compilation will fail.
}
```
Unfortunately, [there is a bug that prevents this from happening](microsoft/TypeScript#10530) when you use bracket notation to access a field. It will be fixed in a future version of TypeScript. Until then, developers will need to cast:
``` typescript
var shape: Shape = getShape();
switch (shape['.tag']) {
case 'point':
console.log('point');
break;
case 'square':
console.log('square ' + (<ShapeSquare> shape).square);
break;
}
```
Struct Polymorphism
-------------------
When a struct explicitly enumerates its subtypes, direct references to the struct will have a `.tag` field to indicate which subtype it is. Direct references to the struct's subtypes will omit this field.
To capture this subtlety, the generator emits an interface that represents a direct reference to a struct with enumerated subtypes:
```
struct Resource
union
file File
folder Folder
path String
struct File extends Resource
...
struct Folder extends Resource
...
```
``` typescript
interface Resource {
path: string;
}
interface File extends Resource {
}
interface Folder extends Resource {
}
interface ResourceReference extends Resource {
'.tag': 'file' | 'folder';
}
interface FileReference extends File {
'.tag': 'file';
}
interface FolderReference extends Folder {
'.tag': 'folder';
}
```
Direct references to `Resource` will be typed as `FileReference | FolderReference | ResourceReference` if the union is open, or `FileReference | FolderReference` if the union is closed. A direct reference to `File` will be typed as `File`, since the `.tag` field will not be present.
TypeScript 2.0's tagged union support should work on these types once the previously-discussed bug is fixed.
Nullable Types
--------------
Nullable types are emitted as optional fields when referenced from structs.
Routes
------
Routes are emitted in the same manner as the JavaScript generators, **except** that TypeScript's type system is unable to type `Promise`-based errors. The generator adds text to the route's documentation that explicitly mentions the data type the developer should expect when an error occurs.
Example:
``` typescript
type DropboxError = DropboxTypes.Error;
db.filesListFolder({path: ''}).then((response) => {
// TypeScript knows the type of response, so no type annotation is needed.
}).catch(
// Add explicit annotation on err.
(err: DropboxError<DropboxTypes.files.ListFolderError>) => {
});
```
Import / Namespaces
-------------------
Stone namespaces are mapped directly to TypeScript namespaces:
```
namespace files;
import common;
struct Metadata
parent_shared_folder_id common.SharedFolderId?
```
``` typescript
namespace files {
interface Metadata {
parent_shared_folder_id?: common.SharedFolderId;
}
}
```
Using the Generator
===================
Both `tsd_types` and `tsd_client` consume a template file, which contains a skeleton around the types they omit. This skeleton is unavoidable, as SDKs may augment SDK classes (like `Dropbox` or `DropboxTeam`) with additional methods not described in stone.
The "templates" simply have a comment string that marks where the generator should insert code. For example, the following template has markers for route definitions and type definitions:
``` typescript
class Dropbox {
// This is an SDK-specific method which isn't described in stone.
getClientId(): string;
// All of the routes go here:
/*ROUTES*/
}
// All of the stone data types are defined here:
/*TYPES*/
```
In the above template, the developer would need to run the `tsd_types` generator to produce an output file, and then run the `tsd_client` generator on that output to insert the routes (or vice-versa).
The developer may also choose to have separate template files for types and routes:
``` typescript
// in types.d.ts
namespace DropboxTypes {
/*TYPES*/
}
```
``` typescript
/// <reference path="./types.d.ts" />
// ^ this will "import" the types from the other file.
// in dropbox.d.ts
namespace DropboxTypes {
class Dropbox {
/*ROUTES*/
}
}
```
Developers can customize the template string used for `tsd_client` with a command line parameter, in case they have multiple independent sets of routes:
``` typescript
namespace DropboxTypes {
class Dropbox {
/*ROUTES*/
}
class DropboxTeam {
/*TEAM_ROUTES*/
}
}
```
Generator Usage in Dropbox SDK
==============================
For Dropbox's JavaScript SDK, I've defined the following templates.
**dropbox.d.tstemplate**: Contains a template for the `Dropbox` class.
``` typescript
/// <reference path="./dropbox_types.d.ts" />
declare module DropboxTypes {
class Dropbox extends DropboxBase {
/**
* The Dropbox SDK class.
*/
constructor(options: DropboxOptions);
/*ROUTES*/
}
}
```
**dropbox_team.d.tstemplate**: Contains a template for the `DropboxTeam` class.
``` typescript
/// <reference path="./dropbox_types.d.ts" />
/// <reference path="./dropbox.d.ts" />
declare module DropboxTypes {
class DropboxTeam extends DropboxBase {
/**
* The DropboxTeam SDK class.
*/
constructor(options: DropboxOptions);
/**
* Returns an instance of Dropbox that can make calls to user api endpoints on
* behalf of the passed user id, using the team access token. Only relevant for
* team endpoints.
*/
actAsUser(userId: string): Dropbox;
/*ROUTES*/
}
}
```
**dropbox_types.d.ts**: Contains a template for the Stone data types, as well as the `DropboxBase` class (which is shared by both `Dropbox` and `DropboxTeam`).
``` typescript
declare module DropboxTypes {
interface DropboxOptions {
// An access token for making authenticated requests.
accessToken?: string;
// The client id for your app. Used to create authentication URL.
clientId?: string;
// Select user is only used by team endpoints. It specifies which user the team access token should be acting as.
selectUser?: string;
}
class DropboxBase {
/**
* Get the access token.
*/
getAccessToken(): string;
/**
* Get a URL that can be used to authenticate users for the Dropbox API.
* @param redirectUri A URL to redirect the user to after authenticating.
* This must be added to your app through the admin interface.
* @param state State that will be returned in the redirect URL to help
* prevent cross site scripting attacks.
*/
getAuthenticationUrl(redirectUri: string, state?: string): string;
/**
* Get the client id
*/
getClientId(): string;
/**
* Set the access token used to authenticate requests to the API.
* @param accessToken An access token.
*/
setAccessToken(accessToken: string): void;
/**
* Set the client id, which is used to help gain an access token.
* @param clientId Your app's client ID.
*/
setClientId(clientId: string): void;
}
/*TYPES*/
}
```
Then, I defined simple definition files for each of the ways you package up the SDK, which references these types. These can be readily distributed alongside your libraries.
`DropboxTeam-sdk.min.d.ts` (`DropboxTeam` class in a UMD module):
``` typescript
/// <reference path="./dropbox_team.d.ts" />
export = DropboxTypes.DropboxTeam;
export as namespace DropboxTeam;
```
`Dropbox-sdk.min.d.ts` (`Dropbox` class in a UMD module):
``` typescript
/// <reference path="./dropbox.d.ts" />
export = DropboxTypes.Dropbox;
export as namespace Dropbox;
```
`dropbox-sdk.js` (`Dropbox` class in a CommonJS module -- not sure why you distribute this when you have a UMD version!):
``` typescript
/// <reference path="./dropbox.d.ts" />
export = DropboxTypes.Dropbox;
```
Finally, for your Node module, there's `src/index.d.ts` which goes alongside `src/index.js` and defines all of your Node modules together. After adding a `typings` field to `package.json` that points to `src/index`, the TypeScript compiler _automatically_ picks up the definitions from the NPM module:
``` typescript
/// <reference path="../dist/dropbox.d.ts" />
/// <reference path="../dist/dropbox_team.d.ts" />
declare module "dropbox/team" {
export = DropboxTypes.DropboxTeam;
}
declare module "dropbox" {
export = DropboxTypes.Dropbox;
}
```
To properly bundle things, I added a `typescript-copy.js` script that NPM calls when you run `npm run build`. The script simply copies the TypeScript typings to the `dist` folder.
These are the files that must be maintained to provide complete TypeScript typings for all of your distribution methods. The files that are likely to change in the future are the templates, as you add/modify/remove SDK-specific interfaces.
jvilk
force-pushed
the
tsd-types-generator
branch
from
f30191a
to
459c59c
Compare
|
I have fixed the linting errors, so the travis build now succeeds. |
|
Awesome! Thank you! |
|
@posita should I open a PR on the Dropbox JS SDK that builds the TypeScript typings for that library? Or are you handling that? |
@braincore, any thoughts on this? |
|
@posita I already opened the PR. |
|
Ah! Brilliant! Thanks @jvilk! |
Adds a generator that produces TypeScript definition files for a JavaScript client. As with JavaScript, there are two generators:
tsd_typesandtsd_client.The definition files require TypeScript 2.0, as they rely upon TypeScript tagged unions.
Demo Generator Output
Install Visual Studio Code and view the Dropbox JS SDK examples in TypeScript. Try hovering over the various variables and function parameters, and writing new code. Pretty great, right?
Note that there are some compilation errors. The Dropbox SDK examples reference struct fields that are not contained in the Stone specification. Are these fields specific to the JS SDK? If so, we may want to modify
--extra-argsso that we can augment arbitrary stone types with extra fields.Dropbox JS SDK Changes
I forked the Dropbox JS SDK to generate TypeScript definitions. The SDK generates typings for:
src/index.d.ts), which TypeScript automatically pulls in when using Dropbox from NPM.dist.Overview: Mapping Stone Types to TypeScript
Below, I will summarize how we map Stone types to TypeScript.
Basic Types
TypeScript's basic types match JSDoc, so there is no difference from the
js_typesgenerator.Alias
Aliases are emitted as
types:Struct
Structs are emitted as
interfaces, which support inheritance. Thus, if a structAextends structB, it will be emitted as:Nullable fields and fields with default values are emitted as optional fields. In addition, the generator adds a field description with the default field value, if the field has one:
Unions
Unions are emitted as a
typethat is the disjunction of all possible union variants (including those from parent types!). Each variant is emitted as an individualinterface.TypeScript 2.0 supports tagged union types like these ones, so the compiler should automatically infer the type of a shape when the developer writes code like so (and statically check that all cases are covered!):
Unfortunately, there is a bug that prevents this from happening when you use bracket notation to access a field. It will be fixed in a future version of TypeScript. Until then, developers will need to cast:
Struct Polymorphism
When a struct explicitly enumerates its subtypes, direct references to the struct will have a
.tagfield to indicate which subtype it is. Direct references to the struct's subtypes will omit this field.To capture this subtlety, the generator emits an interface that represents a direct reference to a struct with enumerated subtypes:
Direct references to
Resourcewill be typed asFileReference | FolderReference | ResourceReferenceif the union is open, orFileReference | FolderReferenceif the union is closed. A direct reference toFilewill be typed asFile, since the.tagfield will not be present.TypeScript 2.0's tagged union support should work on these types once the previously-discussed bug is fixed.
Nullable Types
Nullable types are emitted as optional fields when referenced from structs.
Routes
Routes are emitted in the same manner as the JavaScript generators, except that TypeScript's type system is unable to type
Promise-based errors. The generator adds text to the route's documentation that explicitly mentions the data type the developer should expect when an error occurs.Example:
Import / Namespaces
Stone namespaces are mapped directly to TypeScript namespaces:
Using the Generator
Both
tsd_typesandtsd_clientconsume a template file, which contains a skeleton around the types they omit. This skeleton is unavoidable, as SDKs may augment SDK classes (likeDropboxorDropboxTeam) with additional methods not described in stone.The "templates" simply have a comment string that marks where the generator should insert code. For example, the following template has markers for route definitions and type definitions:
In the above template, the developer would need to run the
tsd_typesgenerator to produce an output file, and then run thetsd_clientgenerator on that output to insert the routes (or vice-versa).The developer may also choose to have separate template files for types and routes:
Developers can customize the template string used for
tsd_clientwith a command line parameter, in case they have multiple independent sets of routes:Generator Usage in Dropbox SDK
For Dropbox's JavaScript SDK, I've defined the following templates.
dropbox.d.tstemplate: Contains a template for the
Dropboxclass.dropbox_team.d.tstemplate: Contains a template for the
DropboxTeamclass.dropbox_types.d.ts: Contains a template for the Stone data types, as well as the
DropboxBaseclass (which is shared by bothDropboxandDropboxTeam).Then, I defined simple definition files for each of the ways you package up the SDK, which references these types. These can be readily distributed alongside your libraries.
DropboxTeam-sdk.min.d.ts(DropboxTeamclass in a UMD module):Dropbox-sdk.min.d.ts(Dropboxclass in a UMD module):dropbox-sdk.js(Dropboxclass in a CommonJS module -- not sure why you distribute this when you have a UMD version!):Finally, for your Node module, there's
src/index.d.tswhich goes alongsidesrc/index.jsand defines all of your Node modules together. After adding atypingsfield topackage.jsonthat points tosrc/index, the TypeScript compiler automatically picks up the definitions from the NPM module:To properly bundle things, I added a
typescript-copy.jsscript that NPM calls when you runnpm run build. The script simply copies the TypeScript typings to thedistfolder.These are the files that must be maintained to provide complete TypeScript typings for all of your distribution methods. The files that are likely to change in the future are the templates, as you add/modify/remove SDK-specific interfaces.