Open discriminated unions to deal with frontend/backend schema version skew?

[tibbe]

A common problem with schemas is that the frontend and backend might not be using the same version. For example, we might update the backend to add a new member to a discriminated union but the frontend won't know about it until updated. Updating them in unison isn't possible.

Catching the parsing error isn't workable. The discriminated union might be part of a large structure that would otherwise parse fine. Think of an input that consists of an array of discriminated union values. Most of the values could be of a type the frontend knows about and we want to parse those (and e.g. ignore the rest).

Other parsing systems, like Google's protocol buffers, have solutions for cases like this.

In TypeScript/zod we need the discriminated union to be defined in such a way that parsing will succeed even for unknown variants of the discriminator. At the same time we want to discriminate the type for know values of the discriminator (i.e. we want to be able to use the discriminator in a switch or if statement to infer which member of the discriminated union we have.)

Here's an example discriminated union:

import { z } from "zod";

const Circle = z.object({
  kind: z.literal('circle'),
  radius: z.number()
});

const Rectangle = z.object({
  kind: z.literal('rectangle'),
  width: z.number(),
  height: z.number()
});

const Shape = z.discriminatedUnion("kind", [Circle, Rectangle]);

const Shapes = z.array(Shape);

We have two constraints:

• We'd like to use Shapes on an unknown input e.g. {kind: 'square', side: 1.0}.
• We'd like to have a switch statement on kind.

The latter would look something like this:

const shapes = Shapes.parse([
  {kind: 'circle', radius: 1.0},
  {kind: 'rectangle', width: 1.0, height: 2.0},
  {kind: 'square', side: 1.0}
]);
shapes.forEach(shape => {
  switch (shape.kind) {
    case 'circle':
      console.log('circle:', shape);
      break;
    case 'rectangle':
      console.log('rectangle:', shape);
      break;
    default:  // `kind: never`
      console.log('unknown:', shape);
      break;
  }
});

The above Shapes.parse doesn't work due to the unknown discriminator value.

I think I want something like

const Shapes = z.array(Shape.passthrough_unknown_discriminant());

(passthrough_unknown_discriminant would be a new method on discriminated unions.)

The type would be unchanged but the runtime behavior would be to just not fail parsing on an unknown discriminated and e.g. passthrough its values. Known discriminates would still parse against their schema.

An alternative that I haven't explored would be to run a preprocess step where every unknown discriminant would be mapped to a known discriminant (e.g. "unknown") that then would be part of the discriminated union. Something like this:

type Circle = {
  kind: 'circle';
  radius: number;
};

type Rectangle = {
  kind: 'rectangle';
  width: number;
  height: number;
};

type UnknownShape = {
  kind: 'unknown';
  // Maybe [k: string]: any
};

type Shape = Circle | Rectangle | UnknownShape;

Thoughts on a solution?

[tibbe]

The preprocess approach seems to work:

const UnknownShape = z.object({
  kind: z.literal('unknown'),
});

const Shape = z.discriminatedUnion("kind", [Circle, Rectangle, UnknownShape]);

const PreprocessedShape = z.preprocess(v => {
  switch (v.kind) {
    case 'circle':
    case 'rectangle':
      return v;
    default:
      return {kind: 'unknown'};
  }
}, Shape);

const Shapes = z.array(PreprocessedShape);

I still need to properly type v above but the runtime behavior and the generated parse return type are what I want.

[tibbe]

Here's a fully typed PreprocessedShape. Quite verbose:

const PreprocessedShape = z.preprocess(v => {
  function hasKind(arg: any): arg is {kind: string} {
    if (typeof arg !== 'object') {
      throw {
        code: 'invalid_type',
        expected: 'object',
        received: String(arg),
        path: [],
        message: 'Required',
      };
    }
    if (arg === null) {
      throw {
        code: 'invalid_type',
        expected: 'object',
        received: String(arg),
        path: [],
        message: 'Expected object, received null',
      };
    }
    if (!('kind' in arg)) {
      throw {
        code: 'invalid_type',
        expected: 'string',
        received: 'undefined',
        path: ['kind'],
        message: 'Required',
      };
    }
    if (typeof arg.kind !== 'string') {
      throw {
        code: 'invalid_type',
        expected: 'string',
        received: String(arg.kind),
        path: ['kind'],
        message: `Expected string, received ${String(arg.kind)}`,
      };
    }
    return true;
  }

  if (!hasKind(v)) {
    throw Error('Invalid shape');
  }
  switch (v.kind) {
    case 'circle':
    case 'rectangle':
      return v;
    default:
      return {kind: 'unknown'};
  }
}, Shape);

[tibbe]

An alternative is to piggy-back on a missing kind field leading to a parse error down the line:

const PreprocessedShape = z.preprocess(v => {
  function hasKind(arg: any): arg is {kind: string} {
    return (
      typeof arg === 'object' &&
      arg !== null &&
      'kind' in arg &&
      typeof arg.kind === 'string'
    );
  }

  if (!hasKind(v)) {
    return v;
  }
  switch (v.kind) {
    case 'circle':
    case 'rectangle':
      return v;
    default:
      return {kind: 'unknown'};
  }
}, Shape);

[tibbe]

Even simpler, just try to parse the discriminator as z.string in preprocess:

const PreprocessedShape = z.preprocess(v => {
  const parsed = z.object({kind: z.string()}).parse(v);
  switch (parsed.kind) {
    case 'circle':
    case 'rectangle':
      return v;
    default:
      return {kind: 'unknown'};
  }
}, Shape);