Proposal - Bridging decorator annotations with this proposal

[matthew-dean]

Hi, I was sent this way from tc39/proposal-decorators#489 because the decorators proposal has an extensions page that discusses annotations that I believe could be more suitable for metadata.

I don't have any objections to this (repo's) proposal, per se, but I think one of the thing that the extensions annotations proposal and my subsequent proposal based on that does well is make the runtime effects of annotations very minimal.

This makes those proposals more suitable for type inference / type-checking systems, because no functions are (usually) executed at runtime, making them ideal for static analysis, whereas this proposal has a larger runtime cost. So while this current proposal-decorator-metadata is not suitable as a static typing system for TypeScript / Flow, what I'm proposing would absolutely be suitable for static type-checking systems.

Also, the annotations proposal has a much tighter syntax than this metadata proposal, making the developer ergonomics more attractive for replacement of type-checking systems.

Please take a look at let me know if there's an opportunity to merge these proposals. -- https://github.com/matthew-dean/proposal-annotations

Example

Just as an example, this repo's proposal adds metadata like:

const METADATA = new WeakMap();

function meta(value) {
  return (_, context) => {
    METADATA.set(context.metadataKey, value);
  };
}

@meta('a')
class C {
  @meta('b')
  m() {}
}

METADATA.get(C[Symbol.metadata]); // 'a'
METADATA.get(C.m[Symbol.metadata]); // 'b'

My proposal takes the much more concise extensions on the decorators proposal and turns the above example into this form:

@'a'
class C {
  @'b'
  m() {}
}

C[Symbol.metadata]; // ['a']
C.m[Symbol.metadata]; // ['b']

// (note in the above that metadata is an array to allow easy "tagging"
// of multiple annotations, just like the extensions proposal)

That's a reduction of ~250 characters to ~80 characters.

Where this syntax really shines is as a replacement for the controversial type-annotations proposal. Instead of adding a bunch of "ignorable" syntax to JS, it adds a single annotation syntax (@'', @[], @{}), with versatility to use strings, arrays, and objects to mean whatever a system might want it to mean.

Some examples from the proposal.

let @"string" x;
x = "hello";

let @{
  name: 'string',
  age: 'number'
} Person

Person[Symbol.metadata]; // { name: 'string', age: 'number' }

@'boolean'
function equals(@'number' x, @'number' y) {
    return x === y;
}

@'<T>'
function foo(@'T' x) {
    return x;
}

This would supplant the need to add type blocks, interface blocks, generics, type assignment, type assertions, non-nullable assertions etc etc etc to the JavaScript language as "comments".

Instead, this metadata approach, because it has very little runtime effect (it does not need to execute a function to decorate with simple primitives), could supply all TypeScript / Flow requirements within JavaScript, as well as preserve those type (or whatever kind) of annotations at runtime, making writing custom runtime helpers trivial.

Thanks for considering this!

[pzuraq]

Thanks for opening the issue over here @matthew-dean, appreciate it!

As noted on the other issue, I think that this style of annotation could possibly be useful for the type annotations that typescript is looking to emit (cc @rbuckton) but its usage of Symbol.metadata would conflict with the other metadata system's usage of that symbol. There are many more use cases for dynamic metadata than static metadata, so if we have to choose between the two I would choose dynamic (e.g. the one outlined by in this repo). If there is a way to expose this information without the conflict though I could see it being useful.

[matthew-dean]

Dynamic metadata

There are many more use cases for dynamic metadata than static metadata, so if we have to choose between the two I would choose dynamic (e.g. the one outlined by in this repo). If there is a way to expose this information without the conflict though I could see it being useful.

I was just about to address this!

I would say a few things:

1. I don't think this style of static annotation actually prevents dynamically setting metadata. I see no reason why you couldn't write a decorator that does this:

function dynamic(annotation) {
  return (value, context) => {
    context.addInitializer(function() {
      context[Symbol.metadata] = [annotation];
    });
  }
}

@dynamic('a')
function fn() { }

In other words, with decorators themselves, if they were simply expanded to functions, variables, and values, could be used to dynamically assign values to the Symbol.metadata key. Or, perhaps (or in addition to this), this proposal could add very simple sugar to decorators.

function meta(annotation) {
  return (value, { addMeta }) => {
    context.addMeta(annotation);
  }
}

@meta('a')
function fn() { }

Once you begin to view the metadata info as a simple object key on values, the concept becomes very powerful.

2. The proposal I have also allows dynamic data by allowing you to pass in an array at definition time. That means you can also do this:

@[dynamic('a')]
function fn() { }

But again, you could also do this with regular decorators, as long as decorators are just expanded to other primitive types and you allow setting the metadata key, because decorators also have this format for evaluating an expression that's done as a function call:

@(dynamic('a')) // already in the Stage 3 proposal

3. If you define Symbol.metadata as a convention of the prototype (basically just a built-in symbol), you could ALSO theoretically do this.

const { metadata } = Symbol

function fn(x) {
  // alter metadata at call time
  fn[metadata] = [x]
}

None of having statically-analyzable metadata prevents a huge number of dynamic options for metadata. So optimizing for static first doesn't at all prevent dynamic assignment (in JavaScript, it's often just objects all the way down), but optimizing for dynamic first, which this repo's proposal currently does does prevent (IMO) adoption by static-analysis engines like type-checkers, because the runtime effects (a huge increase in runtime function calls) would be considered an unacceptable trade-off.

[pzuraq]

To clarify, the current idea for dynamic metadata is to pass in an object on context.metadata. That object is then assigned to the Symbol.metadata on the class itself.

function dec(_, context) {
  context.metadata.foo = 'bar';
}

@dec class C {}

C[Symbol.metadata].foo; // 'bar';

Annotations would need to work with this system, e.g. by assigning themselves to a predetermined name on the metadata object. Some questions that raises:

1. What is the predefined name of a private element?
2. What is the predefined name of the class itself?

Alternatively annotations could expose metadata in a different way, so there isn't a conflict with Symbol.metadata

[matthew-dean]

@pzuraq Oh interesting. I guess the fundamental question is: Is this metadata proposal and my proposal two different proposals? Or is there space for metadata / annotations to work together (or be combined into one proposal)?

[pzuraq]

I would say it's moreso that this metadata proposal is currently at stage 2 and has had a lot of thought behind it and consensus being built so far. There are also a lot of use cases for it, so it seems likely that it'll move forward.

Annotations could in theory be added to this proposal, but that is likely to slow it down and cause it be held up in committee. Previous feedback has always been to make the proposals smaller, that's why this was broken out into its own proposal in the first place. So, more likely, annotations would need to build on top of metadata as a separate proposal.

That said, we can design metadata such that it would be possible to add annotations on top relatively seamlessly, that's how a lot of these proposals work. So we can continue the conversation, and try to figure out what that would look like.

The starting point for that is taking the current design of metadata and trying to build annotations on top of it. If that's not possible, maybe we change the design of metadata - but we're going to need some compelling reasons to do so. The choices behind the metadata design are intentional and were made to solve various constraints, so we need to A. make sure it's possible to solve those same constraints with an alternative design and B. make sure that the alternate design is worthwhile.

[matthew-dean]

There are also a lot of use cases for it, so it seems likely that it'll move forward.

I guess part of what I'm trying to demonstrate, is that with the right minimal design, annotations could possibly jump in demand to one of the highest-demand proposals, if it essentially jump-starts compile-less type-checking systems. But, that said, I also get what you're saying and don't want to slow anything down.

The starting point for that is taking the current design of metadata and trying to build annotations on top of it.

Hmm. I think if there's a hard requirement to include the @meta keyword, I'm not sure there's compatibility there. What are the constraints?

[pzuraq]

@meta is not a keyword, it's an example of a decorator that uses the metadata APIs, specifically context.metadataKey. Any decorator can do this, which is why decorator metadata is more expressive overall than annotations - a decorator could replace a field and add metadata, which some do in some cases.

This iteration of the proposal came to be because the committee rejected the more complex version where a metadata object was created which had a predetermined shape. The complexity was too opinionated and seemed overengineered. My worry is that annotations would have the same problem, which is why I asked these questions:

1. What is the predefined name of a private element?
2. What is the predefined name of the class itself?

Let's consider an annotated class:

@('foo')
class C {
  @('bar')
  @('type:string')
  publicField;

  @('baz')
  @('type:number')
  #privateField;

  @('foobar') publicMethod() {}
  @('barbaz') #privateMethod() {}
}

How do we expose all of these annotations to the user? It's not as simple as just adding them to Symbol.metadata on the annotated value, because class fields do not have a value that can be annotated. So all of these values need to end up on C[Symbol.metadata], or be split up in a non-obvious way (e.g. annotations on methods go on the function, but annotations on fields go on the class). Splitting up has been rejected in previous discussions, so it all goes on the class.

We can't just add the annotations in an array on the class, because it matters what was being annotated. So there are two options:

1. We push a descriptor of sorts to an array which describes what is being annotated:

C[Symbol.metadata] = [
  {
    type: 'class',
    annotations: ['foo'],
  },
  {
    type: 'field',
    public: true,
    name: 'publicField',
    annotations: ['bar', 'type:string'],
  },
  {
    type: 'field',
    public: false,
    name: 'privateField',
    annotations: ['baz', 'type:number'],
  },
  // ...
];

One thing to note here is that the private field annotations are effectively useless, we can't actually do anything with them because we don't have access to the private field. We can fix this by adding a getter/setter for that.

C[Symbol.metadata] = [
  // ...
  {
    type: 'field',
    public: false,
    name: 'privateField',
    access: { get, set },
    annotations: ['baz', 'type:number'],
  },
  // ...
];

The other alternative is to build up a mapping datastructure of sorts. This is what the previous proposal for decorators did:

C[Symbol.metadata] = {
  own: ['foo'],
  public: {
    publicField: ['bar', 'type:string'],
    // ...
  },
  private: [
    {
      name: 'privateField',
      access: { get, set },
      annotations: ['baz', 'type:number'],
    },
    // ...
  ],
};

Note that private is still an array because unlike public elements, private element names really mean nothing. If we wanted to find the super metadata, it wouldn't make sense to shadow it based on the name, it can't be overridden. But maybe it would be fine for private metadata to be its own mapping.

Either way, we end up with a fairly complex data structure that has to be generated just to encode all of the relevant information to the user. This is the exact same place the previous metadata proposal ran into a wall, as the committee preferred something much less complex and opinionated. The current proposal doesn't have this issue because how metadata is stored/represented is left up to the decorator author.

I recognize that there would be some advantages to a more "static" version of decorator metadata, but I think these issues would make it very hard to get through committee, plus as noted before it's considerably less expressive than the current metadata proposal. So between those two things, I'm inclined to stick with the current proposal, and annotations can be explored as a potential future addition.

[trusktr]

TLDR: https://github.com/tc39/proposal-type-annotations seems better to me. But I also don't like them being ignorable. Can we fork that proposal and make them non-ignorable. Actual type checking would be awesome (but would have a cost). Or, at least, it would be possible for type annotations to expose metadata like decorators do (and we'd also need to have decorators everywhere for it to align well).

---

I feel like the annotations would be a really ugly language feature.

This,

import {int, string} from 'some-runtime-type-lib'

function foo(@string name, @int num) {
  // ...
}

is much cleaner than this,

import {check} from 'some-runtime-type-lib'

function foo(@'string' name, @'int' num) {
  check(foo, name, num)
  // ...
}

or this,

import {check} from 'some-runtime-type-lib'

function foo(@{type: 'string'} name, @{type: 'int'} num) {
  check(foo, name, num)
  // ...
}

.

Decorators are already annotations, and they already accept data if they are decorator factory functions. Having all this extra noise, f.e.

@{foo: "one"} @{bar: "two"} class C {
  @{lorem: 123} method() { }
  @{ipsum: 456} field;
}

is really really noisy and to me is very ugly.

It seems to me the main motivation for this feature so far is type checking. Why introduce such ugly type syntax? Why don't we work on the type annotation proposal instead of making a new syntax that TypeScript and Flow both do not support at all yet, that would be completely redundant, and that would be inconvenient to type on keyboard compared to what we already have?

[rbuckton]

I'm not personally swayed by a @("foo") syntax for Metadata. Most systems that want to employ Metadata will want it in a specific format. Arbitrary Metadata as applied by something like @("foo") is impossible to type-check or validate at runtime. Compare that to exposing a decorator that attaches metadata. Such a decorator can validate its inputs and format and attach Metadata consistently.

If you want arbitrary Metadata, you could still do this instead:

@((_, c) => c.metadata.foo = "bar")
class C {}

Which could just as easily be a reusable @meta decorator.