RFC: refactor property signature APIs (e.g. optional)

[gcanti]

This RFC addresses the APIs related to the concept of "optional," specifically the optional API in the current version.

Main Issues:

• The optional API deviates from the standard (API pipeable) as it returns a PropertySignature interface with two methods, namely toOption and withDefault.
• It doesn't cover all use cases requested over time by users. While toOption and withDefault address two common use cases, they do not allow the description of other equally common and important use cases.

For example, the current implementation cannot express the following transformation:

decoding:
<missing value> -> Option.none()
`undefined` value -> Option.none() // This is the part that is currently inexpressible
value -> Option.some(value)

encoding:
Option.none() -> <missing value>
Option.some(value) -> value

Proposed Solution:

• Rename the existing optional API to optionalExact (reminiscent of exactOptionalPropertyTypes in tsconfig.json).
• Introduce a new primitive, optionalExactToRequired, which allows transforming an optional field (i.e. marked with the ? modifier) into a required one by specifying transformation modes in both directions (decode and encode).

Of particular importance is the new primitive optionalExactToRequired, which would enable deriving a series of other combinators, as shown in the example snippet below:

import * as S from "@effect/schema/Schema"
import * as Option from "effect/Option"
import * as Predicate from "effect/Predicate"
import { identity } from "effect/Function"

// ------------------------------
// primitives
// ------------------------------

export declare const propertySignature: <I, A>(
  schema: S.Schema<I, A>,
  options: S.DocAnnotations<A>
) => S.PropertySignature<I, false, A, false>

// NEW this is the old `optional` API renamed
export declare const optionalExact: <I, A>(
  schema: S.Schema<I, A>,
  options?: S.DocAnnotations<A>
) => S.PropertySignature<I, true, A, true>

// NEW
export declare const optionalExactToRequired: <I, A, B, C>(
  from: S.Schema<I, A>,
  to: S.Schema<B, C>,
  decode: (o: Option.Option<A>) => B, // `none` here means: the value is missing in the input
  encode: (b: B) => Option.Option<A>, // `none` here means: the value will be missing in the output
  options?: S.DocAnnotations<A>
) => S.PropertySignature<I, true, C, false>

// ------------------------------
// derivations
// ------------------------------

// NEW
const optional = <I, A>(
  schema: S.Schema<I, A>,
  options?: S.DocAnnotations<A | undefined>
): S.PropertySignature<I | undefined, true, A | undefined, true> => optionalExact(S.union(S.undefined, schema), options)

// NEW this is the old optional().toOption()
const optionalExactToOption = <I, A>(
  schema: S.Schema<I, A>,
  options?: S.DocAnnotations<A>
): S.PropertySignature<I, true, Option.Option<A>, false> =>
  optionalExactToRequired(
    schema,
    S.optionFromSelf(S.to(schema)),
    Option.filter(Predicate.isNotUndefined),
    identity,
    options
  )

// NEW
const optionalToOption = <I, A>(
  schema: S.Schema<I, A>,
  options?: S.DocAnnotations<A | undefined>
): S.PropertySignature<I | undefined, true, Option.Option<A>, false> =>
  optionalExactToRequired(
    S.union(S.undefined, schema),
    S.optionFromSelf(S.to(schema)),
    Option.filter(Predicate.isNotUndefined),
    identity,
    options
  )

// NEW this is the old optional().withDefault()
const optionalExactWithDefault = <I, A>(
  schema: S.Schema<I, A>,
  value: () => A,
  options?: S.DocAnnotations<A>
): S.PropertySignature<I, true, A, false> =>
  optionalExactToRequired(schema, S.to(schema), Option.match({ onNone: value, onSome: identity }), Option.some, options)

// NEW
const optionalWithDefault = <I, A>(
  schema: S.Schema<I, A>,
  value: () => A,
  options?: S.DocAnnotations<A | undefined>
): S.PropertySignature<I | undefined, true, A, false> =>
  optionalExactToRequired(
    S.union(S.undefined, schema),
    S.to(schema),
    Option.match({ onNone: value, onSome: (a) => (a === undefined ? value() : a) }),
    Option.some,
    options
  )

/*
const schema: S.Schema<{
    readonly a: string;
    readonly b?: string;
    readonly c?: string | undefined;
    readonly d?: string;
    readonly e?: string | undefined;
    readonly f?: string;
    readonly g?: string | undefined;
}, {
    readonly a: string;
    readonly b?: string;
    readonly c?: string;
    readonly d: Option.Option<string>;
    readonly e: Option.Option<string>;
    readonly f: string;
    readonly g: string;
}>
*/
const schema = S.struct({
  a: propertySignature(S.string, {}),
  b: optionalExact(S.string),
  c: optional(S.string),
  d: optionalExactToOption(S.string),
  e: optionalToOption(S.string),
  f: optionalExactWithDefault(S.string, () => ""),
  g: optionalWithDefault(S.string, () => "")
})

[datner]

TL;DR but LGTM

[gcanti]

I simplified it a bit by merging all APIs into optional (via an optional argument):

/*
const schema: S.Schema<{
    readonly a: string;
    readonly b?: string;
    readonly c?: string | undefined;
    readonly d?: string;
    readonly e?: string | undefined;
    readonly f?: string;
    readonly g?: string | undefined;
}, {
    readonly a: string;
    readonly b?: string;
    readonly c?: string;
    readonly d: Option.Option<string>;
    readonly e: Option.Option<string>;
    readonly f: string;
    readonly g: string;
}>
*/
const schema = S.struct({
  a: S.string.pipe(S.propertySignatureAnnotations({ description: "my string" })),
  b: S.optional(S.string, { exact: true }),
  c: S.optional(S.string),
  d: S.optional(S.string, { exact: true, toOption: true }),
  e: S.optional(S.string, { toOption: true }),
  f: S.optional(S.string, { exact: true, default: () => "" }),
  g: S.optional(S.string, { default: () => "" })
})

[datner]

Really glad we're doing breaking api changes with a random voting LGTM 😄