Discussion: (Reflective) Type Model

[christyharagan]

Hi all,

We currently have a AST model for TypeScript which is useful for compilers, editors, and linters.

However, it would be extremely useful to have a type model, a la Java's reflective model. There are a huge number of use-cases supporting this, but I will list a few here:

• I have a REST interface and I define a data model for it as a TypeScript interface. I use this interface for the function definition of the REST implementation, but then use the type model to validate the input at runtime.
• I have a service model that abstracts an interface for some system (database, or application, etc.). I define this as a TypeScript interface (maybe with some decorators as meta-data), and have a generator library that extracts this information to either generate code - or create a runtime implementation - that actually implements the service.
• I have an injection (DI) system. I use injection decorators on class properties/constructor parameters, and use the type model to reflectively wire the correct instance.

A good example of what could be achieved with this, is something like the Spring platform for Java where our applications are built as decorated components and most of the boilerplate code is abstracted away by the platform thanks to reflection and decorators.

I have a "first stab" at implementing such a thing: typescript-schema. It's not feature complete and the design isn't finished either. But hopefully it gives a feel for what a reflective model could look like.

Is this something the TypeScript team would be interested in having as a first class citizen?

[danquirk]

You might be interested in #2577. Reflection type functionality has been an oft requested but controversial issue for us.

[mhegazy]

Another related topic would be #3136.

I use injection decorators on class properties/constructor parameters, and use the type model to reflectively wire the correct instance.

can you elaborate on how this is done.

[tobich]

@danquirk So when @jonathandturner said at http://blogs.msdn.com/b/typescript/archive/2015/03/05/angular-2-0-built-on-typescript.aspx

We've also added a way to retrieve type information at runtime. When enabled, this will enable developers to do a simple type introspection.

he meant metadata generated only for decorators, not full-scale reflection (for all public symbols, for example)?

[mhegazy]

he meant metadata generated only for decorators, not full-scale reflection (for all public symbols, for example)?

correct.

[sophiajt]

What he said :)

[remojansen]

I'm happy to see:

A few notes on metadata:

• Type metadata uses the metadata key "design:type".
• Parameter type metadata uses the metadata key "design:paramtypes".
• Return type metadata uses the metadata key "design:returntype".

From #2589

What other metadata are you planing to support in the future? There are a few things that I would like to see:

• Implemented interfaces uses the metadata key "design:implements".
• Parameter name metadata uses the metadata key "design:paramnames".

Of course "design:implements" would require the interface names to be serialized using the interfaces names not Object (or complex type serialization). The same any case in which an interface is serialized.

[mhegazy]

Parameter name metadata uses the metadata key "design:paramnames".

this sounds interesting. and should be doable

The same any case in which an interface is serialized.

Interfaces are not serialized today. only names with a value attached to them (i.e. classes) and built-ins are serialized.

[remojansen]

About:

Interfaces are not serialized today

I'm aware of a gist (by Ron Buckton) about complex type serialization, is that going ahead? Are you guys considering adding interface serialization support at some point?

[christyharagan]

As mentioned in my original post, I've got a project called typescript-schema that provides this kind of serialisation, as well as a reflective type model:

It has two types of model: A serialisable model, and a runtime reflective model.

It works by: AST -> Serialisable Model -> Runtime reflective model

It currently only supports 1.5, and only exported types. It's not 100% complete, but is able to process the entire TypeScript library itself, and the node type library too (as well as a number of other large libraries).

What I'd really like is to understand is:

a) if the TypeScript team would want any of this code, and if so
b) what would be required to make it ready for inclusion? I.e.: as it stands it's largely standalone (and probably doesn't follow all the code standards)

To give you an example of it at work, imagine this TypeScript code:

import {X} from 'my-module'
export interface MyInterface<T> {
  f(x: X, s: string):T
}

This will serialise to:

{
  "instanceType": {
    "typeKind": "COMPOSITE",
    "members": {
      "f": {
        "optional": false,
        "type": {
          "typeKind": "FUNCTION",
          "parameters": [{
            "name": "x",
            "type": {
              "module": "my-module"
              "name": "X"
            },
            {
              "name": "s",
              "type": {
                "typeKind": "PRIMITIVE",
                "primitiveTypeKind": "STRING"
              }
            }],
            "type": {
              "module": "@",
              "name": "T"
            }
          }
        }
      }
    }
  }
}

This can then be converted into a reflective model, which can be used like:

  let myInterfaceConstructor:InterfaceConstructor //Called an interface constructor because it has type parameters

  myInterfaceConstructor.parent // The parent module or namespace
  myInterfaceConstructor.name // "MyInterface"
  myInterfaceConstructor.typeParameters[0].name // "T"
  myInterfaceConstructor.instanceType.members['f'].type.name // "T"
  myInterfaceConstructor.instanceType.members['f'].parameters[0].name // "x"

You can then "close" that reflective interface by providing a type argument, and the output will be the corresponding interface with all type parameters correctly substituted:

  let myInterface = closeInterfaceConstructor(myInterfaceConstructor, [someType])
  myInterface.typeConstructor // myInterfaceConstructor
  myInterface.instanceType.members['f'].type // someType

To check it out, go to:

typescript-schema

and

typescript-package (which provides the code for converting a package of TypeScript into the serialisable form - including typings and nested packages via the node resolve algorithm)

[remojansen]

When I asked for a "design:paramnames" metadata key.

@mhegazy said:

this sounds interesting. and should be doable

So I have created a new issue to request it: #4905

[asvetliakov]

Interfaces are not serialized today. only names with a value attached to them (i.e. classes) and built-ins are serialized.

What if you guys will introduce new language semantic? Something like realtime interface Name {...} which will remain realtime (with empty method bodies for easing mocking in testing frameworks)?
I doubt the interface serialization will be implemented somewhen, since there is a much code written which assuming that interfaces are not realtime objects.

I forced to use classes as interfaces now to make DI work

[mikehaas763]

Is this issue getting at being able to do something like the following for using interface annotations as DI tokens?

interface IAlgorithm {
    execute(): void; 
}

class FancyAlgorithm implements IAlgorithm {
    execute(): void {
        let i = 123 * 456;
    }
}

class AlgorithmExecuter {
    constructor(private algorithm: IAlgorithm) { }

    execute(): void {
        this.algorithm.execute();
    }
}

class Program {
    main() {
        // DI API based on autofac DI for .NET
        // container could be populated any number of ways just imperative for illustration
        let builder: Container = new ContainerBuilder();
        builder.RegisterType(FancyAlgorithm).As(IAlgorithm);
        builder.RegisterType(AlgorithmExecuter).AsSelf();

        // now invoke the composition root
        let executer = builder.Resolve(AlgorithmExecuter);
        executer.execute();
    }
}

The specific reason I'm asking is because when using a DI system for TypeScript that uses type annotations as metadata (such as what exists for Angular 2) this is currently not possible because no runtime "Reflect" unique metadata is emitted for interfaces. In DI circles this is basically the way DI based apps are built. It's extremely powerful and makes for very flexible and easily testable apps.

If this issue doesn't involve this kind of metadata please let me know as I'd like to create a separate proposal issue. 😄

[mhegazy]

@mikehaas763 nothing stops you today from doing the same thing but using strings in place of types. i am assuming this would be sufficient. your DI system needs to have a key to store and locate these, if you use strings, then nothing in this issue should block you.

@export("IAlgorithm")
class FancyAlgorithm implements IAlgorithm {
    execute(): void {
        let i = 123 * 456;
    }
}

[mikehaas763]

@mhegazy I don't understand what you're trying to portray with that code snippet. What I would like to do is be able to register an interface type as a key in a DI container. Can you elaborate?

[mhegazy]

I don't understand what you're trying to portray with that code snippet. What I would like to do is be able to register an interface type as a key in a DI container.

I do not know much about autofac, so my comments are assuming you have a central registry that records mapping between a type and an exported entity, something a la MEF.

I am also assuming you need a key, and you want to use the name of the interface as key, all what i am saying is you can do that by making the key as a string (in this case using the name of the interface, but it should be something more unique).

the export decorator i am using refers to something like MEF ExportAttribute, in JS terms this would be something that takes a key (interface name) and a value (a class constructor), and keeps them in the list (along with arguments to the constructor possibly). that is basically will call your builder.RegisterType(FancyAlgorithm).As(IAlgorithm); except that it will be somethign like builder.RegisterType(FancyAlgorithm).As("IAlgorithm");

later on your call to builder.Resolve("IAlgorithm"); and that would return you all constructors with this key.

[mikehaas763]

UPDATE: I wrote this before you replied above. Reading your response now.

@mhegazy A problem with using strings in place of types is that you lose the automatic given uniqueness of that type. Two different libraries may have an IAlgorithm interface type and both may need to be registered with a single DI container. That's easy enough to do like so if the compiler generated some sort of unique metadata token to help guarantee uniqueness and a fake import could somehow still work at runtime. So something like the following:

import IAlgorithm as ILibAAlgorithm from './libA';
import IAlgorithm as ILibBAlgorithm from './libB';
import AImplementation from './a';
import BImplementation from './b';

builder.RegisterType(AImplementation).As(ILibAAlgorithm);
builder.RegisterType(BImplementation).As(ILibBAlgorithm);

The very nature of the two IAlgorithms being two different things is enforced by them being defined separately in separate modules. I understand that this is not possible today because there is no IAlgorithm to import at runtime (or do I have this wrong?). What I'm saying is that this would be a nice feature to have in TS. I'm speaking for myself now but also for the swaths of developers that I guarantee will reiterate wanting to see the same capability as TS becomes used more.

So either I have this completely wrong and it's already possible 😄 or if not it would be awesome if we could start talking about what the actual implementation would look like and make it a formal proposal for TS or there is just better ways to do this and I'm bringing over too much cruft to TS from Java/C# etc.

UPDATE: I finished reading your reply above. Yes you can assume there is a registry (the container in autofac). I get that it would be possible by registering a type against a string key in a container but do you see the concerns I have with that around uniqueness and robustness?

[mikehaas763]

I am also assuming you need a key, and you want to use the name of the interface as key

I don't want to use the string name of the interface as a key, I want to use the interface itself as a key. This is possible with concrete types because they exist at runtime. Interfaces don't exist at runtime and obviously for good reason but it would be nice to have this sort of metadata for interfaces at runtime.

One implementation that may work is to compile the interface to an empty lightweight concrete type so it can be used at runtime.

// for example
export default interface IFoo {
    foo(): void;
    bar(): void;
    lorem(): void;
    ipsum(): void;
}
// could be compiled to (depending on the target but in this case ES6)
export default class IFoo {}

That way, IFoo is a guaranteed unique key at runtime.

[BobbieBarker]

I think Mike's point addresses a real concern for serious application development which utilize frameworks and Dependency Injection systems. With out something like Mike's proposal in place DI systems typically work as you suggest on strings or tokenized versions of the strings. In a non TS world, that works for the most part. But in a TS/typed universe I think we should be including and utilizing types where possible in our DI systems.

[mikehaas763]

@mhegazy Do you think I should start a new issue to propose/track this?

I'd like to get more people's thoughts on this. After stepping away I think the most seamless way to do this would be like I mentioned above is to compile an interface type to a "class" type. This is already done with abstract class types.

Maybe a compiler option such as --runtimeInterfaces? I realize that this in a way is a huge change because the spec is explicit that interfaces do not exist at runtime. I know that abstract classes are now supported. Am I forced as a developer using DI to now use abstract classes wherever I would have previously used just an interface?

[mhegazy]

I mentioned above is to compile an interface type to a "class" type. This is already done with abstract class types.

I am not sure i understand what you mean by "compile" to "class type". interfaces do exist in a different space (type space), and emit a value for them would cause problems when it comes to merging (see declaration merging docs for more information).

I want to use the interface itself as a key. This is possible with concrete types because they exist at runtime.

one thing to note is TypeScript's type system is structural, it is not nominal (like other languages such as C# and Java with DI systems). so the interface name, or declaration is of no consequence here; consider an example with two different classes looking for interfaces with different names, but comparable structures, do you want your DI to find them or not?

interface Person {
     name: string;
     title?: string;
}

interface SomethingWithAName {
    name: string;
}

class C {
     constructor(c: SomethingWithAName) {}
}

var p: Person;
new C(p); // this is fine, the two interfaces are valid

then what would you do with things that do not have a name?

class C {
     constructor(a: {name: string}) {}
}

or type aliases?

type myType = string;
class C {
     constructor(a: myType) {}
}

or more complex type operators:

type myType = string;
class C {
     constructor(a: (string | { name: string }) & EventTarget) {}
}

Obviously a typical DI system from a language with nominal type system like C# would not fit here with no compromises. i would say you will have to limit the set of supported language constructs to allow your DI system to work, i.e. say only classes are supported by this system, interfaces, structural types, etc will not be allowed.

If you agree with this proposition, then it should just work with classes. classes have values, and can be used at rumtime as keys to your DI system. and that should be easy to model today using decorators, e.g.:

    // use an abstract class instead of an interface
    abstract class Algorithm {
        abstract execute(): void;
    }

    @exportAs(Algorithm)   // use the class constructor as a key for your type lookups
    class FancyAlgorithm implements Algorithm { // use implements here, so no runtime changes (i.e no calls to __extend)
        execute(): void {
            let i = 123 * 456;
        }
    }

    abstract class Class { }

    function exportAs<T extends Class>(typeObject: T) {
        return function (target: T): void {
            // wire the export
            builder.RegisterType(target).As(typeObject);
        }
    }

    // later at runtime you would do:
    let executer = builder.Resolve(Algorithm); // should find FancyAlgorithm

[asvetliakov]

Obviously a typical DI system from a language with nominal type system like C# would not fit here with no compromises. i would say you will have to limit the set of supported language constructs to allow your DI system to work, i.e. say only classes are supported by this system, interfaces, structural types, etc will not be allowed.
If you agree with this proposition, then it should just work with classes. classes have values, and can be used at rumtime as keys to your DI system. and that should be easy to model today using decorators, e.g.:

Using abstract classes as interfaces approach works not very well for DI. Any abstract methods will be omitted from compiled JS, so it's not possible to create tests stubs from abstracted classes (for ex. using sinon.createStubInstance()). Also the compiler doesn't emit any runtime checks to prevent creation abstract classes at runtime. If you forgot to bind implementation to abstract class acting as interface, it will create empty object of your interface instance, instead giving you error:

abstract class TestInterface { ... }

class TestController { 
   @Inject
   public myService: TestInterface;
}

// Forgot to bind any implementation to TestInterface
let controller = container.resolve(TestController); // Doesn't throw any error since abstract class could be created at runtime.

I forced to use now these ugly constructions:

class Algorithm {
    constructor() {throw new Error();}
    public execute(): void {}
    public calculate(): number {return}
}

class MyAlgorithm implements Algorithm { .... }

With this, runtime checking and mocking methods (like sinon.createStubInstance()) will work correctly.

[mikehaas763]

As an FYI, I was just suggesting compiling an interface to a type as one specific way to solve the problem of providing this sort of interface metadata. It could be handled other ways.

interfaces do exist in a different space (type space)

I'm aware. I suggested this purely as a means to be able to use interfaces in a meta way at runtime by just compiling it to a value type. I'm not saying it's the only way or even the proper way.

and emit a value for them would cause problems when it comes to merging

Why would this cause issues with merging? Merging does occur when an interface is defined in separate modules does it?

I'm aware that the type system is structural which I admit makes it harder for me to reason about this.

then what would you do with things that do not have a name?

If I was declaring the type annotation of something with a literal, than I wouldn't expect to be able to use that as a key in my DI system.

If you agree with this proposition, then it should just work with classes. classes have values, and can be used at rumtime as keys to your DI system

The problem with depending on classes is that it blatantly violates the dependency inversion principle: "Depend upon Abstractions. Do not depend upon concretions.". I can see people saying "see, just use an abstract class for that". Well, if I just end up using an abstract class just like an interface, what is the point of having interfaces in TS?

At the end of the day this conversation is just about possibilities to provide a means to an end and allowing something that makes development better and easier. That end being I want to be able to continue to program like the following snippet but also have dependencies injected as expected based on the type annotation that is already specified (IDependency).

class Foo {
    constructor(dependency: IDependency) {}
}

In the meantime I had planned on just using abstract classes as interfaces (like in the following snippet), but will have to look more closely at the problems that introduces that @asvetliakov mentioned above.

abstract class IAlgorithm {
    abstract execute(): void;
}

[asvetliakov]

Ideally it will be cool if you guys implement something like that:

interface realtime MyInterface {
    public method1(): void;
    public method2(): string;
}

Which will be compiled to:

function MyInterface() {
    throw new Error("Not possible to create MyInterface instance");
}

MyInterface.prototype.method1 = function () { throw new Error("Not possible to call interface method MyInterface::method1"); }
MyInterface.prototype.method2 = function() { throw new Error("Not possible to call interface method MyInterface::method2"); }

This will not interfere with existing interfaces in type scope and people will be able to use DI as it was intended.
Or at least provide runtime constructor checking & empty function stubs for abstract classes & methods

[mikehaas763]

@asvetliakov I like where you're going with that but at the same time IDK if I like that it has to be explicitly marked. If I'm implementing an interface that some 3rd party lib has provided, then I don't have the ability to mark that interface.

[asvetliakov]

@mikehaas763 Don't see any issues. 3rd party libs are usually declaring variable (with interface type) if exporting something, so there will be runtime information and you can use this variable as key for DI.

[mikehaas763]

@asvetliakov I mean them declaring an interface to use as in the concept of inversion of control. They define an interface that a consumer implements and passed back into their lib.

[mhegazy]

Why would this cause issues with merging? Merging does occur when an interface is defined in separate modules does it?

now you are talking about limiting it to only module scopes. that was not in the OP.

Well, if I just end up using an abstract class just like an interface, what is the point of having interfaces in TS?

interfaces in TypeScript are design-only constructs, just like the rest of the types. using these constructs in the value space is not a TS design goal.

[mhegazy]

talking with @rbuckton today, here is another option that works today and does not use classes:

   const IAlgorithm = new Symbol("IAlgorithm"); // unique identifier for the interface
   interface IAlgorithm {
       execute(): void;
    }

    @exportAs(IAlgorithm)   // the interface and the var are merged
    class FancyAlgorithm implements Algorithm { 
        execute(): void {
            let i = 123 * 456;
        }
    }

[rezonant]

@Hookyns Let's continue this discussion on the issue you opened to keep this thread clear for the primary discussion (which is typescript-rtti/typescript-rtti#4 for those interested)

[schmod]

I agree that decorators seem like a bad long-term solution for this problem. They're widely-disliked in other languages that use them (ie. Java), and they don't really solve the problem of having excessive/unchecked duplication between your types and validators.

However, lately I've started to become less and less convinced that TypeScript actually needs "first-class" reflection support to solve the real-world problems that most of us are trying to solve.

Instead of generating runtime/validation information from types, I've found that the opposite approach works quite well with TypeScript as it exists today. Libraries like io-ts and decoders do an excellent job of yielding good TS types via inference from the decoder/codec.

While I'm unaware of any popular implementations that exist today, I'd imagine that we might be able to achieve practical solutions for the other two big use-cases for reflection (ORMs and DI) using similar techniques.

---

Admittedly, I wouldn't be upset if TS provided better built-in tools (or official recommendations) for validation, but I think the language actually is far enough along that we may actually have good alternative solutions for nearly all common use-cases for reflective types.

Putting all of this aside, the underlying problem still remains: Runtime validation is an extremely common problem for most TS devs, and the language could do a lot more to help us with it. Absent a real reflection system, I'd be very happy making something like io-ts or decoders an actual part of the language and tslib. It's such a common use-case that developers struggle with today, and a little bit of language-level integration would go a long way, even in absence of a larger/more-generic reflection system.

[rezonant]

Instead of generating runtime/validation information from types, I've found that the opposite approach works quite well with TypeScript as it exists today. Libraries like io-ts and decoders do an excellent job of yielding good TS types via inference from the decoder/codec.

@schmod could you sum up the approach here for those (including myself) unfamiliar with io-ts/decoders?

Oh, I think this sums it up (from decoders):

//
// Incoming data at runtime
//
const externalData = {
    id: 123,
    name: 'Alison Roberts',
    createdAt: '1994-01-11T12:26:37.024Z',
    tags: ['foo', 'bar'],
};

//
// Write the decoder (= what you expect the data to look like)
//
const userDecoder = object({
    id: number,
    name: string,
    createdAt: optional(iso8601),
    tags: array(string),
});

[schmod]

import { DecoderType, object, string, boolean, guard } from 'decoders';

const userValidator = object({
  isActive: boolean,
  name: string
});

type User = DecoderType<typeof userValidator>;

// ... and in-use:
const unsafeObj: unknown = { isActive: true, name: 'me' };
const me = guard(userValidator)(unsafeUser); // <= validated `User` (throws an exception if invalid)
const alsoMe: User = me; // safe assignment because `me` is a `User`

In this scenario, instead of writing a TS type for User, I write my validator, and TS infers a type for User from that validator.

[rezonant]

One downside of this approach is that documentation does not seem to flow through-- though I think that is something that TS can do right?

(the intention would be for the docblock "Hello World" to make it into the intellisense)

Ahh no TS can't do this, its just an object literal. :-\

[rezonant]

Another downside to this approach is you cannot "go to definition" for any of these fields, ie in this context:

let u : User;

     u.createdAt
//     ^--- cannot navigate to definition

This would also preclude basically every automatic refactoring VS Code provides, which is losing a lot of value :-
(yeah, tested in StackBlitz, VS Code cannot provide refactoring, though it doesn't realize it cannot do it, it tries but fails to change anything, which is probably a bug)

[akutruff]

If you have an everyday schema that can be described with just arrays and primitives and objects, things like io-ts or tools to produce .d.ts from a JSON schema are super appropriate.

YES. Everyday schemas. That's the thing. Using a separate JSON schema or library's typing system is the problem, not the solution.

Here's a declaration from the io-ts docs:

type User = {
  userId: number;
  name: string;
}

My schema is right there in beautiful TypeScript. It is my source of truth. Anything else is a lie because this is the definition my code requires.

We use io-ts not because its the right solution, but because it's the best option we have. It is trying it's best to be like TypeScript. Their docs devotes a chunk of itself to explaining how to convert TypeScript declarations to io-ts declarations:

import * as t from 'io-ts'

const User = t.type({
  userId: t.number,
  name: t.string
})

Imagine if io-ts had a direct line to type User, then it can just focus on providing a good validation dsl.

A system can't take an arbitrary TypeScript type, flush it to an output of any sort, and then reason about whether or not a value matches that type without effectively being tsc).

I couldn't have said it better myself. We need a subset of tsc at run-time because we are reinventing the wheel and we're not as good at it as you all.

[schmod]

Another downside to this approach is you cannot "go to definition" for any of these fields, ie in this context:

I think it might be possible to fix those drawbacks without needing to build reflection (which, again, seems like an extremely heavy lift that the core team doesn't seem to be interested in undertaking).

Today, in my codebase, I often work around this by writing something like

interface User extends DecoderType<typeof userValidator> { /* prop types go here */ }

// or 

const userDecoder: Decoder<User> = object({ /* prop validators go here */ });

This adds some duplication, but preserves the "go to definition" stuff, and most importantly, does not break typesafety anywhere. If the decoder/interface are not kept in sync with each other, tsc will report it as an error.

[Hookyns]

@schmod, @akutruff Check this REPL out.

• Just a regular TypeScript,
• no decorators nor any other kind of hints,
• no problem with types from 3rd party packages.

Simplified version of that REPL:

export interface SomeInterface
{
	stringProp: string;
	numberProp: number;
	booleanProp: boolean;
	arrayProp: Array<string>;
}

export class SomeClass
{
	stringProp: string;
	anyProp: any;
	stringArrayProp: string[];
	optionalMethod?(this: SomeInterface, size: number): void { }
}

const someObject = {
	anyProp: true,
	stringProp: "",
	stringArrayProp: ["foo"],
	optionalMethod() { }
};

console.log("someObject is SomeInterface: ", isValid<SomeInterface>(someObject)); // > false
console.log("someObject is SomeClass: ", isValid<SomeClass>(someObject)); // > true

You don't have to write any code. You have just your application types (interfaces, classes,...), nothing more.

isValid() is function implemented in that REPL, it uses my tst-reflect.
tst-reflect has no extra features like validators, type-guards etc.. it is clear reflection emitting instances of class Type (containing all the information about type) which can be de facto standard. A lot of packages can be built on top of that, like this isValid() which can be turned into standalone package (with a little more work 😆, it is just a simple demo).

PS: Yeah, isValid<SomeClass>(someObject) should be false, because someObject is not an instance of the SomeClass. It's how I wrote that isValid function. It just checks members. There is no problem to check if it is an instance of class.
It's simple like:

	if (target.isClass())
	{
		return value instanceof target.ctor;
	}

[sinclairzx81]

@RyanCavanaugh Could a macro system based on conditional types be a possible solution to runtime reflection in TypeScript? Given that it's possible to compute new types from existing types (and that the language service can present the computed type in editor), having TSC emit the computed type as a JS value would let users define a variety of metadata derived from existing TypeScript definitions. For example.

Current: Conditional Types

The following generates a JSON schema type representation from a TypeScript static type.

type JsonSchema<T> =
    T extends object  ? { type: 'object', properties: {[K in keyof T] : JsonSchema<T[K]> } } :
    T extends string  ? { type: 'string' }  :
    T extends number  ? { type: 'number' }  :
    T extends boolean ? { type: 'boolean' } :
    never


type Vector = JsonSchema<{             // type Vector = {
    x: number,                         //   type: 'object';
    y: number,                         //   properties: {
    z: number                          //      x: {
}>                                     //        type: 'number';
                                       //      };
                                       //      y: {
                                       //        type: 'number';
                                       //      };
                                       //      z: {
                                       //        type: 'number';
                                       //      };
                                       //   };
                                       // }
									   

Future: Conditional Macros

Replace type for macro where the macro emits the result of the conditional type as a JS value or expression.

macro JsonSchema<T> =
    T extends object  ? { type: 'object', properties: {[K in keyof T] : JsonSchema<T[K]> } } :
    T extends string  ? { type: 'string' }  :
    T extends number  ? { type: 'number' }  :
    T extends boolean ? { type: 'boolean' } :
    never


const Vector = JsonSchema<{            // const Vector = {
    x: number,                         //   type: 'object';
    y: number,                         //   properties: {
    z: number                          //      x: {
}>                                     //        type: 'number';
                                       //      };
                                       //      y: {
                                       //        type: 'number';
                                       //      };
                                       //      z: {
                                       //        type: 'number';
                                       //      };
                                       //   };
                                       // }

// Reflect !
console.log(Vector.type)           
console.log(Object.keys(Vector.properties))  

And could be used for other things like.

macro NotImplemented<T> = T extends (...args: infer P) => infer U 
	? (...args: P) => U { throw Error('not implemented') } // permit any valid JavaScript expression
	: never

macro Implement<T extends object> = {
    [K in keyof T]: NotImplemented<T[K]>
}

interface Service {
    add(a: number, b: number): number,
    sub(a: number, b: number): number,
    mul(a: number, b: number): number,
    div(a: number, b: number): number,
}

const ImplementedService = Implement<Service> 
// const ImplementedService = {
//   add: (a: number, b: number): number => { throw Error('not implemented') },
//   sub: (a: number, b: number): number => { throw Error('not implemented') },
//   mul: (a: number, b: number): number => { throw Error('not implemented') },
//   div: (a: number, b: number): number => { throw Error('not implemented') },
// }


function test(service: Service) {
	service.add(1, 2)
	service.sub(1, 2)
	service.mul(1, 2)
	service.div(1, 2)
} 

test(ImplementedService)

There is an outstanding open issue for macro support here. Curious if conditional type mapping might serve as a good basis for some future TypeScript macro system.

[rezonant]

The v0.0.20 release of typescript-rtti supports reflecting on both classes (ie "by value") and interfaces ("by type") via reflect(MyClass) or reflect<MyInterface>(), respectively. I think it covers both use cases nicely. As a bonus it can validate object values against the emitted types-- while more extensive solutions (possibly built on top of these features) will always make sense, the basic validation cases are covered.

My thought is to take the reflection API (reflect() mainly) itself out of the transformer project so that it can objectively implement any/all of the relevant RTTI formats in one place which should help to guard against dependency on a single one while we collectively work out what the format itself should look like long term, and create opportunities for others to come up with other solutions if they want to. It already supports both emitDecoratorMetadata and typescript-rtti's format, it should be possible to map more to it.

I do think the format typescript-rtti uses right now has some merits, but gawking at the "weirdness" of it is not unexpected- to be compact and also retain some level of intentional awkwardness when trying to access the metadata directly (instead of via API), it is not something you'd ever want to interact with directly. That was indeed the point.

I'd considered symbols prior but thought it was pointless considering the code being generated itself needs access to the symbols. It also complicates the transformer because those imports need to be independently managed. On the other hand, at the very least it eliminates the runtime visibility of it (unless you're looking for it), and better matches the (for lack of a better term) WebIDL style, so moving to that might make sense to clean it up in preparation for wider usage.

Edit: On the other hand, any dependency on a symbol necessitates a reference to the source library to interpret it's metadata format, and I'm not sure that's reasonable.

[Hookyns]

@schmod, @akutruff, @capaj

Released version 0.7.0 of tst-reflect is now able to get type of runtime value. getType(someValue)

import { getType } from "tst-reflect";

class A
{
	constructor(public foo: string)
	{
	}
}

const someValue: unknown = new A("Lorem ipsum");
console.log(getType(someValue).is(getType<A>())); // > true

const someValue2: unknown = { foo: "dolor sit amet" };
console.log(getType(someValue2).is(getType<A>())); // > false
console.log(getType(someValue2).isAssignableTo(getType<A>())); // > true

The runtime value in memory can be class or some native type (object, array, string, etc..).

Then it is possible to call existing .isAssignableTo(), so I've rewritten REPL I've posted few comments above to this REPL which use just .isAssignableTo(). Object validation out of the box...

[rezonant]

Lot of progress on typescript-rtti lately including reflection on functions, shape matching, and more. We've also been battle testing it with complex codebases, both those dependent on emitDecoratorMetadata and not. You can try it out on this spiffy website I made that runs TS, the transformer and the reflect() API in the browser: https://typescript-rtti.org

[avin-kavish]

There are true technical limitations

I think we can easily say that some reflection is better than no reflection. It doesn't have to be a complete solution that sees all types. It can stop within a reasonable subset of types for which reflection will help meet geater use cases, which is inversion of control and dependency injection. And that mostly means reflection for simple classes and interfaces. When these types can't be reasonably expressed, typescript can emit a no-op, just like it emits Object now on emitDecoratorMetadata

A system can't take an arbitrary TypeScript type, flush it to an output of any sort, and then reason about whether or not a value matches that type without effectively being tsc. And at the point that you've reimplemented tsc.

You are assuming the reason for flushing a type is to see whether it fits structurally. But let's say for a common use case of reflection like model building. a la entity framework, the only question that the consumer needs to ask a type is what are your inherited public properties. I mean how hard it is to ask a question like that from the type checker? It's hard. So that's what we want, we want a "Reflection API" on top of the type information that is already available in typescript.

And that's why I use tst-reflect, you can ask sensible questions from it at runtime instead of having to deal with the compilation oriented typechecker api.

So basically what you are doing is looking at the entire problem and saying that it is not solvable. But a subset of the problem is solvable. And we know from prior experience that solving a subset of the problem (emitDecoratorMetadata) helps meet a subset of use cases. (Angular and TypeORM). So yeah, this can absolutely be done to a reasonably useful level.

[mindplay-dk]

DeepKit is doing full-on reflection, and they've built a complete framework making use of these reflection facilities: ORM, dependency injection, configuration, routing, RPC, validation, etc. - what's interesting about this is they've not only built it, they're using it, and it has lots of interesting applications.

One major down side to this though, is the fact it's only going to work with the official TypeScript compiler - whereas probably most projects these days are actually built with Babel, ESBuild, SWC, etc... expecting all of these projects to implement full scale reflection is probably asking a lot. It might put the official compiler back in the game though?

[akutruff]

@DanielRosenwasser Belated 8th anniversary of this issue, and an update just as an FYI - I just received a flurry of PR's for https://github.com/akutruff/typescript-needs-types. The list of projects grows and grows. Is there really no way that the TS team could do something?

[Hookyns]

I've already resigned waiting for this. 👎

I've already started building new robust version of my tst-reflect which will support all build tools (typescript itself with all TS based bundlers as well as Vite, esbuild and SWC).

[akutruff]

What's extra frustrating is that the language also prevents us from building our own in userspace properly. Recursive types are almost impossible to implement without the hackery Zod has to resort to. The other one is being able to conditionally add property modifiers in mapped types. #44261. The lack of first class union-detection as a utility type leads to a very deep rabbit hole as well.

[Hookyns]

I agree. I personally miss the Ability to decorate abstract members/classes/interfaces; it would be great feature for developers that use the reflection.

On the other hand I have no problem with recursive types. It wasn't difficult to handle it so that generated metadata would work as expected.

[mindplay-dk]

I personally miss the Ability to decorate abstract members/classes/interfaces; it would be great feature for developers that use the reflection.

Yes, it feels like the language is inconsistent on this point. It would be natural for new developers to ask "why does this feature only work on certain types of members", and of course, if you're already a JavaScript expert, you would know why - but I'm sure an increasing number of new developers are jumping head-first into TypeScript, and this would be surprising if, for example, they've been taught Java or C# in school.

Arguably, TS is a superset of JS, so not really intended to stand alone - but still, as a language in and of itself, I think it's also arguable that TS is inheriting more limitations from JS than if the same language had been designed from the ground up.

Regardless of static typing being largely a solved problem with TS, the lack of good DI support on the server-side remains a sore point, and the main reason I still can't confidently recommend TS as a server-side language for anything large. Great for the client-side, but not so great on the server-side, in practice, at scale. More type checking improvements can't fix that.

Side note, for those who don't know, @Hookyns is building a reflection library for TS with ergonomics similar to C# - I'm personally pretty excited about this! 🤩

[DanielRosenwasser]

Hey all,

While I understand the appeal and potential use cases for purely-reflective or intentionally runtime-validated types, I have to say be explicit and state that this is not the direction of the language.

First and foremost, TypeScript is primarily designed as a static type checker. Its main goal is to catch type errors during development, providing early feedback and improving code quality. Adding runtime-reified types (even just for reflective purposes) would introduce a significant overhead, both in terms of compiler performance and code size. The TypeScript team has made a conscious decision to prioritize static typing over other approaches.

Second, over time we have intentionally been making room for alternative compilers. These alternative compilers have varying levels of cross-file analysis, ranging from full analysis across implementation files and declaration files, to none at all. Realistically, the only compiler that has type information available to it is the TypeScript compiler itself; but over time, our focus on emit has only waned. We believe most of the power of TypeScript comes from type-checking and tooling, and developers benefit from erasable emit that works in alternative (often faster!) compilers than ours.

While part of the discussion in this issue is in regards to a reflective type representation, many here are discussing runtime-validated types. Some discussion here is a mix, with the notion that some runtime type validation could be performed through the use of a reflective model. The high-level idea is that metadata is injected so that libraries can perform runtime checks based on the types themselves. As some point out, there is a repo listing all the runtime type-checking libraries as a sort of "proof" that this is necessary; however, the fact that each library varies in needs and scope means that some theoretically emitted metadata needs to be maximally inclusive of whatever tool might ask for it. This is a nightmare for bundle size and might be a fool's errand to implement. As @RyanCavanaugh mentioned here, maybe tools are better off using the TypeScript API as part of this since these tools know precisely what they need.

As I've mentioned in the same thread, libraries have taken 2 approaches - either use our API to generate runtime validation code out of a set of types, or leverage TypeScript's type system to derive static types from the construction of the runtime validator itself! We think that these directions are a lot more promising than any sort of philosophical change to our emit.

We know that there's been a lot of feedback on this issue, and as such believe that there is not much new ground left to cover with further comments on the topic. To prevent a flood of notifications on everyone's inbox, we're temporarily locking this issue for 2 weeks for some pre-emptive cooldown, but further discussion can pick up at that time if needed.