Rework of the Shape type system

[sam-goodwin]

This is nearing completion and I'm super excited about it!

Classes instead of Values

Closes: #87

It used to be that a shape was just a value:

const Type = struct({
  key: string
});

But this made it awkward to create and use values, as the type of a value was:

const value: Value.Of<typeof Type> = {
  key: 'a key'
};

Now, developers dynamically create classes by extending the result of a function call:

class Type extends Record({
  key: string
}) {}

This kills two birds with one stone:

1. Gives the developer a nice and pretty class name to use at runtime instead of the ugly mapped type:

const value = new Type({
  // type-safe constructor
  key: 'a key'
});

2. Enables both compile-time and run-time reflection capabilities.

Type.members; { key: Member<StringShape, key, {}>; }
Type.members.key.Shape; // StringShape

const instance = new Type({ .. });
instance[ClassShape.Members]; // same as Type.members, but available on an instance

This effectively emulates java's runtime reflection (Type.class or instance.getClass()) while also supporting awesome TypeScript type-magic for DSLs/ORMs.

Traits and Decorators

TypeScript decorators can be put on a class as normal:

@decorator
class Type extends Record({ .. }) {}

But, decorators must be put a redundant declaration of a member:

class Type extends Record({
  key: string
}) {
  @decorator
  public readonly key: string; // re-declaration
}

This is a bummer, but at least they're supported now.

To remove this redundancy, we also introduce the concept of "Traits" - type-safe "decorators" that can be applied to shapes. Not only do they support adding metadata to members in a record, but they also support augmenting the type-signature of that member - adding type-level properties that can influence the type-mappings used throughout shape DSLs.

An example:

class Type extends Record({
  key: string
    .apply(MaxLength(10))
    .apply(Pattern('.*'))
}) {}

This is analogous with an ordinary class definition:

class Type {
  @MaxLength(10)
  @Pattern('.*')
  public readonly key: string;
}

Except the type-signature of the string shape is augmented, adding metadata from the traits to it (available at compile-time):

Type.members.key;
// is of type
StringShape & {
  [Decorated.Data]: {
    maxLength: 10,
    pattern: '.*'
  }
}

This information is then utilized at runtime to perform the validation, but also enables some interesting type-level machinery. For example, those literal values are preserved when mapping a Shape to its corresponding JSON schema:

const schema = JsonSchema.of(Type);
// is of type
interface TypeJsonSchema {
  type: 'object',
  properties: {
    key: {
      type: 'string',
      // literal types are preserved
      maxLength: 10,
      pattern: '.*'
    }
  }
}

The type-signature is the exact same as the value (literals and all) - I think that is pretty cool!

expect(schema).toEqual({
  type: 'object',
  properties: {
    key: {
      type: 'string',
      // literal types are preserved
      maxLength: 10,
      pattern: '.*'
    }
  }
})

This machinery will enable a bunch of customizability features for DSLs derived from Shapes.

Supporting an ecosystem of ORMs and DSLs

Closes: #12
Emulates missing behavior in TypeScript critical for ORMs: microsoft/TypeScript#7169

This change also extracts Shapes into its own package, @punchcard/shape, and fractures individual DSL/ORM implementations into their own package:

• @punchcard/shape-dynamodb - DynamoDB ser/de and Condition/Filter/Query Expression DSL
• @punchcard/shape-json - JSON ser/de
• @punchcard/shape-jsonschema - Create a JSON schema from a Shape
• @punchcard/shape-jsonpath - Type-Safe DSL for generating JSON path expressions over a Shape

By fracturing the implementations, it is now possible to implement third-party DSLs for various domains using Shapes - hopefully this will enable an ecosystem to grow!

It is achieved with module augmentation and ad-hoc polymorphism - @punchcard/shape provides the primitive shape implementations which libraries augment to add their own mappings.

Example:

// create a named symbol to identify your DSL
const MyDslTag = Symbol.for('my-dsl');

// augment each of the types, using the tag to map a specific type to a type in your DSL's domain
declare module '@punchcard/shape/lib/primitive' {
  interface StringShape {
    [MyDslTag]: MySpecialStringMapping;
  }
  // repeat for the other types
}

Take a look at how DynamoDB is built for more detail:
https://github.com/punchcard/punchcard/blob/c9341abcec06105e70c1645ba007b7a526a77894/packages/%40punchcard/shape-dynamodb/lib/collection.ts#L6-L26

Developers then implement a ShapeVisitor to map from the Shape Abstract Syntax Tree (AST) to a new AST that represents their domain. Again, here is how that is achieved in DynamoDB:

https://github.com/punchcard/punchcard/blob/c9341abcec06105e70c1645ba007b7a526a77894/packages/%40punchcard/shape-dynamodb/lib/dsl.ts#L38-L98

TODO

• Add READMEs for each package describing how to use it.
• Update README and Docs with new way of using Shapes
• Rename things and move them around to better support discovery and auto-import with IDE
• Bunch of code documentation
• Another pass to make sure tests are comprehensive
• Aesthetic refactors for code legibility - cleanup any artifacts lying around.

Example usages

Check out the tests to see the new DSLs in action. Or as always, check out the Stream Processing example:

https://github.com/punchcard/punchcard/blob/c9341abcec06105e70c1645ba007b7a526a77894/examples/lib/stream-processing.ts#L22-L176

Derive a JSON schema from a Record type:

const schema = JsonSchema.of(Type);
// note how the type of the JSON schema preserves information from traits, e.g. Minimum(0)

typeof schema // is of type:
interface SchemaType {
  type: 'object',
  properties: {
    key: {
      type: 'string',
    },
    count: {
      type: 'number',
      minimum: 0, // literal type of zero is known thanks to the type-safe Minimum trait
    },
    list: {
      type: 'array',
      items: {
        type: 'string'
      }
    }, //etc.
  }
}

DynamoDB Attribute Types also have a fully-preserved type-mapping

AttributeValue.of(Type); // is of type
interface TypeAttributeValue {
  M: {
    key: {
      S: string;
    },
    count: {
      N: string;
    },
    list: {
      L: [{
        S: string;
      }]
    }
  }
}

DynamoDB DSL:

const hashKey =  new DynamoDBClient(Type, key /* just a string for hash-keys */, {
  tableName: 'my-table-name'
});
const hashAndSortKey = new DynamoDBClient(Type, ['key', 'count'] */ tuple for hash and sort key pair */, {
  tableName: 'my-table-name'
});

DynamoDB Put If:

// now you pass an instance of the Record class - much nicer than some ugly mapped typer!
await table.putIf(new Type({
    key: 'key',
    count: 1,
    list: ['a', 'b'],
    dict: {
      key: 'value'
    },
    dynamic: 'dynamic-value'
  }), _ =>
    _.count.equals(1).and(
    // can now index lists with an array accessor
    _.list[0].lessThanOrEqual(0)).and(
    // same for maps ...
    _.dict.a.equals('value')));

DynamoDB Update:

await table.update(['key', 1], item => [
    item.list.push('item'),
    item.dynamic.as(string).set('dynamic-value'),
    item.count.set(item.count.plus(1)),
    item.count.increment()
  ]);

DynamoDB Query and Filter:

await sortedTable.query(['id', count => count.greaterThan(1)], {
  filter: item => item.array.length.equals(item.count)
});

JSON Path has a similar DSL:
https://github.com/punchcard/punchcard/blob/c9341abcec06105e70c1645ba007b7a526a77894/packages/%40punchcard/shape-jsonpath/test/json-path.test.ts#L17-L65

[sam-goodwin]

This should only copy values for this class's properties and also validate them for optional/required/constraints etc.