Provide standard types

[demurgos]

Hi,
This post is a suggestion to add general purpose types that others can rely on.
I would like to increase the number of lib-* types and other such type definitions that are not tied to a single library. Currently there are only two "lib" repositories: lib-json-schema and lib-http-status-codes. We could do more. There are many occasions when I need a type that is not tied to a library. I often end up writing my own ad-hoc definitions but having a standardized representations would help. I believe that this organization is the best place to host them.

Here are some concrete examples.

package.json, tsconfig.json

We could provide interfaces for popular JSON files. Typescript exposes definitions for parsed compiler options but not for "tsconfig.json" files. While writing build tools, being able to have a typed package.json would also help. We could also provide a generic Json type (more restrictive than any), types for web extension manifests, bower files, etc.
In addition to providing type information, it could also leverage doc comments to provide quick documentation.

Semantic types

This is mostly about providing aliases for primitive types to hint the usage, even if the compiler does not use it. Concretely, I often like to type my functions this way:

type PosixPath = string;
type Uint8 = number;

// ...

function readFirstByte(path: PosixPath): Uint8 {
  // ...
}

Typescript sees it as readFirstByte(path: string): number but as a developer, displaying the arguments or jumping to the declaration allows me to quickly understand that it's not any string but a POSIX path.
We could provide a library with commented aliases explaining the semantics of each alias to facilitate documentation purposes. Instead of repeating the description of an Uint8 I could generate documentation to link to the "canonical" description. Given the huge variety of string types, it could be nice to agree on some relation between a name and definition that can be used across the ecosystem.
Here are some example for strings: JsonString, Hex, Uuid, Host, MediaType, Cidr, CssColor, ClassName, Uri, SysPath, WinPath, PosixPath...

Node components

Node has a big API. If you need the types for only part of the API (for example streams or event emitters), you have to pull the whole definitions. It means that you create globals when in fact you may only want the event emitter types to expose a compatible implementation (for example for browsers). We could try to modularize the Node definitions, and eventually turn the Node definitions into a simple aggregation of smaller type definitions.

Utility types

I am not sure if those are still needed now that the standard lib comes bundled with many utility types. But we could have a package providing types of the same sort as Partial<T> or MapLike<T>. I no longer really rely on this sort of helper types so I am not sure if it's really needed but it's an idea.

---

What do you think about it? If such "general-purpose" types are created, how should they be published? They don't really belong to @types since they are not attached to a single library. They would fit nicely with typings but its deprecated. We could simply publish those as standalone npm packages using the old typed- prefix. (typed-package.json for example).

[blakeembrey]

I like it. We can also publish to @typings if we can figure out how to automate that effectively. I feel like we could probably go ahead and use this as the test run for DefinitelyTyped redirects though (e.g. instead of publishing to @types with DT, publish to @typings with lib-less types).

[blakeembrey]

I'm not 100% convinced on the value of the namespace though, so I'd love more input. Otherwise we can just establish a NPM package name pattern that we follow. E.g. json-schema-typings, typings-json-schema, etc. That has the advantage (and disadvantage) that anyone can publish these without infrastructure set up. However, I don't think the infrastructure needs to be extensive - even scraping DT regularly was ~200 LOC (https://github.com/typings/api/blob/master/src/scripts/sync/dt.ts) but for whatever we build we can just connect it as a webhook instead.

[unional]

👍 on publishing non-npm to @typings to simplify automation. Define every repository on @types as npm package (except things like _discussion and _generator..., and for this argument maybe it worth putting these to @typings also).

[demurgos]

Ok, thats a good idea. Since you would be expected to explicitly require the interfaces, there's no need to use typeroots for this. Importing from @typings would be a clear way to signal that it's only about types, not implementations.

import {PackageJson} from "@typings/package-json";

export function generatePackage(): PackageJson {
  // ...
}

(About names, I am not sure if we should add "schema" in the name, I feel that it sounds as if we also bundled a runtime function to test if an object matches an interface (schema validator), but I may be wrong)

[felixfbecker]

I don't really see the value in such type aliases.

• They don't increase type safety. You can still pass a Windows path just fine, so what you are really doing is documentation. Types are for type safety, docblocks are for documentation.
• The TypeScript language service will resolve aliases, so this "documentation" is actually less visible than real documentation:

vs

Going down the road of defining different type aliases for the possible values that are not actually the same type it's unclear where to draw the line. Do you want an alias for RelativePosixPath and AbsolutePosixPath too?

If you want real type safety, define a class that represents the path, like the WHATWG URI class does for URIs. A parameter typed as a URI is guaranteed to always only carry a 100% valid URI. Like the URI class, you could then store an underlying array of segments so methods that operate on it (e.g. dirname, relative, join) are faster because they don't need to reparse.

Re: @typings vs @types, I think that would cause a lot of confusion. I would rather publish an unscoped/unprefixed npm package that only contains types.

[unional]

The scope is about organization management. So that tools can be created to automate things.

Also, usage of it would be:

import '@typings/abc'

Abc.Xyz...

I agree that @typings could be confusing.
Maybe @global-types?

[demurgos]

It must depend on the editor. I am using Webstorm and it does not do full resolution. Here is what I get:

Completion:

Documentation:

I can even click on "PosixPath" to get a description:

If I remember well enough, Typedoc does not do full resolution but links to aliases (like Webstorm) so you would get the same benefits (explained type alias semantics).

About adding further distinctions (relative, absolute), I'd say that drawing this line is the role of the maintainer, to decide what's the reasonable point to stop. For a general-purpose library, I see the point for AbsolutePosixPath but having AbsolutePosixResolvedSymlinkPath is too much so I'd draw the line in between. Balancing usability with other guarantees is the reason why I think that public interfaces should provide these sort of aliases. Even if I use the URI class internally, there is some value to expose an overloaded public function that accepts either an URI or string, and then there is some value to state the expectations about the allowed strings.

Here are some real examples where type aliases (and quick doc) helped me.

• I have a package helping me to define some gulp tasks. As inputs, it accepts any path (windows/posix) and then normalizes it to posix to only manipulate posix paths internally. The generate gulp options only use absolute paths to be unambiguous. So inside my package, I have some functions that joins path segments to end with an absolute path. Being able to add type aliases currently serves me as some micro-documentation helping me to know if I expect a path to be resolved or not at this point in my function.
• Time units. When interfacing with other programs, you have to agree on time representation. If you don't use ISO date strings, then you often have a numeric timestamp. Now what: is it in seconds or milliseconds? Again, having the IDE displaying the function tooltips as I type helps me to double-check that there is nothing weird going on and that I have the correct unit (setFrameDuration(duration: TimeMs)).

I could use a Time class abstracting this and handling unit conversions, but at the end of the day I would still need to call this external function requiring a number, so do I use time.seconds() or time.ms()? Having the tooltip spares me a documentation lookup.
Overall, using dedicated classes offers you better abstractions but it adds a layer when using simpler types might be enough, and if you only use it occasionally it may add some overhead.

This is already a long message, but I still hear your point: the compiler does not help you there. I know, but there are some open issues to provide similar features: Tagged types, Units. Each compiler has its limits, I really love Typescript but sometimes you have to admit that you cannot express something and enforce it with the compiler, so type aliases are a best-effort idea to bridge the gap until tagged types are implemented. There also was some proposition to reintroduce small bits of nominal typings.

It may be a bit extreme but having nominally typed aliases would allow you to do rewrite the path.posix declarations like this:

// Current node.d.ts
interface posix {
  export function join(...paths: any[]): string;
  export function resolve(...pathSegments: any[]): string;
  // ...
}

// Using nominally typed aliases:
interface posix {
  export type Path<S extends string = string> = S;  // A `Path` is a subset of `string`
  export type AbsPath<P extends Path = Path> = P; // An `AbsPath` is a subset of `Path`
  export function join(path: AbsPath, ...paths: Path[]): AbsPath;
  export function join(...paths: Path[]): Path;
  export function resolve(...pathSegments: Path[]): AbsPath;
  // ...
}

// Then I would go as far as requiring compile-time checks
const foo: posix.Path = "test/tsconfig.json";
const bar: string = "package.json";

// Ok
const resolvedFoo: posix.AbsPath = posix.resolve(foo); // Ok

// Ok, but compilation error if `as posix.Path` is missing
const resolvedBar: posix.AbsPath = posix.resolve(bar as posix.Path); 

The last example may be a bit too disruptive for the ecosystem right now, but I'd like to tend in this direction were types can encode additional predicates.

Ok, so this was a long answer to explain my point of view on aliases. Should I open a separate issue?

---

About @typings vs @types, my original proposal was to use the typed-* prefix. If I understand well @unional prefers @typings because it grants you control over the namespace. I felt that teaching people that @types are for libraries and @typings are for pure types would be enough. Combining both may be too much (@typings/typed-*)? Also, as I write this I realize that people may already associate typed-* to library definitions due to the old typings repos so another prefix may be more suitable.

[unional]

It's not just about namespace. It's about tooling and avoid confusion. I would prefer the system to be generic and no special case handling.

The example I gave should really be /// <ref type=.../> instead of import '@typings/abc'

But the idea is the same. Those would be non-npm libraries and they are all global/script

[felixfbecker]

@demurgos than I assume Webstorm doesn't use the official TS language service, which is what we should optimise for.
You say you want the IDE to display you function tooltips, but why is a docblock like in my example not enough? In your example, as a newcomer I don't know that I am allowed to pass a string, I would assume I need to instantiate some PosixPath class.

@unional re: namespace, I prefer something like @global-types or @util-types over a generic @typings because the difference is not clear to @types. Also not sure what "tooling" exactly you imagine.

[unional]

@felixfbecker completely agree. I prefer @global-types

[unional]

For tooling, I mean microsoft/types-publisher#4 (comment)

[unional]

Just created https://github.com/global-types
Can I invite you guys?

[blakeembrey]

I'd prefer not to have these be global types though. The issue with globals is that they conflict and are abstract. Especially with NPM flattening, we can run into duplicate identifiers. I'd prefer to keep importing the types like import { PackageJson } from '@typings/package.json'.

Edit: The name itself doesn't matter too much. Like I mentioned, we can also just publish these as regular packages today like types-package.json or something.

[unional]

We can name it anything we want. I assume we are referring to non-npm library. In those cases, how do JS use them? My assumption is that they are global.

...oh, things like electron are used from require('electron')?

In the past, I have created organizations like bower-types. Something similar?

[demurgos]

@felixfbecker You may be right that optimizing for the official compiler is the right thing to do. I always used Webstorm so wasn't aware of the difference. Regarding the comparison with normal documentation, I guess that it's a matter of personal preference. It's not only about function arguments but also with local variables, since I type most of my variables, having an alias allows me to better express my assumptions about a variable. As stated in the original comment, it also allows me to provide a single detailed description of what an alias means, so I can focus my function documentation on the role of the parameters instead of their "shape".
I'd propose to set aside the idea of the aliases for the moment. I'll re-open an issue specifically to discuss it once we'll already have a few general purpose types.

@unional Sure, go ahead: you can invite me.

Regarding the imports, I also prefer the standard ES syntax. The Typescript references always seemed to me like an implementation detail introduced when TS did not follow the Node's resolution algorithm. Just to be sure, I assume that any of those types will be local and will have to be manually imported by any module relying on it. Different versions in different parts of the dependency tree should not clash. They are "global types" because they are useful for the whole ecosystem, not because they introduce globals.

[demurgos]

@unional Could you link to this "bower-types" organization? I couldn't find it, does it still exists?

Regarding the name, I fear that "global-types" sounds to much as if it extended the environment by adding only global types. Beyond the increased risk of conflict caused by globals, it's also harder to trace were the definitions comes from. Do you add /// <ref... only to the main module or to any module using it? If you add it to every module, what's the benefit compared to ES imports?

I like @felixfbecker's @util-types. We could also use a more abstract name (would it hurt discoverability/credibility?): I thought about @typewriter but it's not available on Github.

[unional]

@unional Could you link to this "bower-types" organization? I couldn't find it, does it still exists?

I have deleted it.

ES imports implies more as you are importing actual code instead of just types.

I am neutral on either way. Personally I also don't use /// <ref

[demurgos]

I'm closing this issue because it's stale. There's interest around having more types but it's better to work on them individually.

Types for JSON files would be better served by repositories with dedicated schemas, providing both TS types and various runtime check helpers.

What I called semantic types may be interesting but the only way to use them reliably for interop would be to have them as nominal/branded/tagged types inside tslib.

I still think that Node's typing should be broken down and refactored, but it requires a massive effort because they are so relied upon.

Utility types are already numerous in the standard lib. Additional types can just be exposed as individual npm packages.