Pipe/flow/chain type support

[andykais]

Search Terms

recursive, flow, pipe, chain

Suggestion

I do not have a specific implementation in mind to support this. There is a decent existing solution by a stackoverflow user @jcalz, which does its best given what typescript is able to accomplish now. I have created this issue to more to specifically point out the current limitation of typescript in regard to this feature, and the current use cases. This feature request falls under the larger umbrella of recursive type inference.

At its most broad, I would like to suggest that typescript include some form of an official Chain type that developers can use.

Unanswered Questions

• should this type have an official implementation with a max recursion value? pipe: Chain<In, Out, number>
• will this design pattern cease to be relevant if the tc39 pipeline proposal is integrated into javascript?

Use Cases

There are a number of implementations for javascript pipes in common libraries.

• lodash pipe
• rxjs pipe
• ramda pipe

Related Issues

#27102
#28505

Examples

a typical implementation looks like this:

function pipe<T, A>(arg1: (in: T) => A): A
function pipe<T, A, B>(arg1: (in: T) => A, arg2: (in: A) => B): (in: T) => B
function pipe(...args: any[]): (in: any) => any


const strToNumber = (str: string) => parseInt(str)
const add = (x: number) => (y: number) => x + y
const safeAdder = pipe(strToNumber, add(5))
const out: number = safeAdder('6') // returns 11

The limitation of the current implementation is that if you use a typed pipe function with more arguments than the written overloads, you run lose type inference.

Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• This feature would agree with the rest of TypeScript's Design Goals.

[steve-taylor]

As this needs a proposal, here’s my $0.02, using the example of a hypothetical observable library heavily inspired by RxJS.

class Observable<T> {
    // ...

    pipe2<...A>(operator: Operator<T, A[0]>, ...restOperators: Operator<A[0], A[1]>[]): Observable<A[-1]>

    pipe<A>(operator: Operator<T, unknown>, ...restOperators: Operator<unknown, unknown>[]): Observable<A> {
        return restOperators.reduce(
            (observable, operator) => operator(observable),
            this as Observable<unknown>
        ) as Observable<A>
    }
}

In a regular parameter, A is indexed using an absolute index.

In a rest parameter, A is indexed using an index relative to the index of the actual parameter.

In the return type, A[-1] represents the last type parameter. (A[-2] would be the 2nd last type parameter, and so on.)

Note: I can’t think of a way to reference these types in the method body. It might be impossible. For that reason, my proposal is limited to method/function declaration only. It wouldn’t be value syntax for method/function implementations.

The implementation of functions and methods would be no worse than the current situation, but at least usages of these functions and methods would now be type checked to any depth without having to pump out a dozen overloads.

[andykais]

I suppose if were spitballing, Ill take a stab as well. I do think thats a decent idea, but its a little hairy because the generic param ...A has to read the function argument indexes to figure out what its minimum length is and ...restOperators: Operator<A[0], A[1]>[] doesnt actually specify that the pattern is repeating. I believe what we need is some sort of recursive structure so heres another thought:

type Operation<In,Out> = (x: In) => Out

pipe2<T, U, V, X, Y, ...A extends [Operation<T,U>, Operation<U,V>, ...A],  (...ops: A[], opLast<X,Y>): Y {

}

but even this doesnt really describe that the 2nd and 3rd arguments should follow the same pattern as the 1st and 2nd.

[MobiusHorizons]

My vote would be to describe such types using something similar to the mathematical notation where n is used to describe a series, which is then defined by relationships between members of that series.

function pipe<T[n]>(...operators: Array<OperatorFunction<T[n], T[n+1]>>): Observable<T[n]> {
...
}

[mdbetancourt]

export type TransformFn<I = unknown, O = unknown> = (input: I) => O;

export type TransformPipe<F, T extends TransformFn[]> = T extends [
  TransformFn<F, infer O1>,
  ...infer Rest
]
  ? Rest extends TransformFn[] ? [TransformFn<F, O1>, ...TransformPipe<O1, Rest>] : [TransformFn<F, O1>]
  : T

function pipe<V, H extends TransformFn[]>(v: V, ...t: TransformPipe<V, H>): TransformPipe<V, H> {}

i manage to get it to "work" but looks like a typescript error

[profound7]

I managed to implement a pipe function type. Instead of overloading, it uses tuple rest args and recursion.

There are 2 flavors of the pipe type. One returns a function (Pipe), and the other has a value as the first argument and immediately executes (PipeValue).

// A is argument type, R is return type
type Fn<A = any, R = any> = (a: A) => R;

type PipeResult<A, T extends Fn[]> = T extends [Fn<any, infer R>, ...infer Rest]
    ? Rest extends [Fn, ...any]
        ? PipeResult<A, Rest>
        : Fn<A, R>
    : never;

type PipeArgs<A, T extends Fn[]> = T extends [Fn<any, infer R>, ...infer Rest]
    ? Rest extends [Fn, ...any]
        ? [Fn<A, R>, ...PipeArgs<R, Rest>]
        : [Fn<A, R>]
    : T;

// usage: pipe(fn0, fn1, fn2...)(x)
type Pipe = <A, T extends Fn[]>(...fns: PipeArgs<A, T>) => PipeResult<A, T>;

// usage: pipe(x, fn0, fn1, fn2...)
type PipeValue = <A, T extends Fn[]>(a: A, ...fns: PipeArgs<A, T>) => ReturnType<PipeResult<A, T>>;

// example implementation (I apologize for my liberal use of any)
const pipe: Pipe = ((...fns: any[]) => (x: any) => fns.reduce((v, f) => f(v), x)) as any;
const pipeValue: PipeValue = ((x: any, ...fns: any[]) => fns.reduce((v, f) => f(v), x)) as any;

It correctly identifies the first incorrect argument.
The return type of the pipe function is correctly inferred even if there are incorrect arguments.

For compose, just switch around the inferring of A and R

type ComposeResult<R, T extends Fn[]> = T extends [Fn<infer A, any>, ...infer Rest]
    ? Rest extends [Fn, ...any]
        ? ComposeResult<R, Rest>
        : Fn<A, R>
    : never;

type ComposeArgs<R, T extends Fn[]> = T extends [Fn<infer A, any>, ...infer Rest]
    ? Rest extends [Fn, ...any]
        ? [Fn<A, R>, ...ComposeArgs<A, Rest>]
        : [Fn<A, R>]
    : T;

type Compose = <R, T extends Fn[]>(...fns: ComposeArgs<R, T>) => ComposeResult<R, T>;

const compose: Compose = ((...fns: any[]) => (x: any) => fns.reduceRight((v, f) => f(v), x)) as any;

[smolijar]

Works great @profound7. I wonder, is it also possible to optionally infer the value of the first function from the composed function argument?

// Works great, 2 is checked to be a number
const x = pipe((a: number) => a.toString(), x => +x)(2)

// Could we infer `a` in the first function to be the type inferred from 2 (Number | '2')?
const y = pipe(a => a.toString(), x => +x)(2)

[Azarattum]

@profound7, unfortunately this still performs worse that overloads implementations, since it can not infer types from a previous function in a chain:

pipe(
    (n: number) => n * 2,
    (n) => {}, // n should be `number`
//   ^? (parameter) n: any
);

This looks to be a Typescript bug, since the playground even suggests that n should be a number when you start typing the second function. But when you hover over it, typescript says n is any.

Is there any progress to fully support chain types in Typescript? The issue is here since 2019...

[Azarattum]

This is the best (shortest & fullest) solution I've made so far:

• It correctly infers up to 7 piped function (using overloads)
• It type checks any number of functions (using @profound7's recursive method)
• It always infers the right return type
• It supports an arbitrary number of arguments for the first function
• It is written as short as possible while keeping readability

Try it on the playground!

type Fn<A = any[], Z = any> = (...args: A extends any[] ? A : [A]) => Z;

type Arg1<A, Z> = [Fn<A, Z>];
type Arg2<A, B, Z> = [Fn<A, B>, Fn<[B], Z>];
type Arg3<A, B, C, Z> = [...Arg2<A, B, C>, Fn<[C], Z>];
type Arg4<A, B, C, D, Z> = [...Arg3<A, B, C, D>, Fn<[D], Z>];
type Arg5<A, B, C, D, E, Z> = [...Arg4<A, B, C, D, E>, Fn<[E], Z>];
type Arg6<A, B, C, D, E, F, Z> = [...Arg5<A, B, C, D, E, F>, Fn<[F], Z>];
type Arg7<A, B, C, D, E, F, G, Z> = [...Arg6<A, B, C, D, E, F, G>, Fn<[G], Z>];
type ArgAny<A, B, C, D, E, F, G, H, Z extends Fn[]> = [
  ...Arg7<A, B, C, D, E, F, G, H>,
  ...Args<H, Z>
];

type Args<A, Z extends Fn[]> = Z extends [Fn<any, infer R>, ...infer Rest]
  ? Rest extends [Fn, ...any]
    ? [Fn<A, R>, ...Args<R, Rest>]
    : [Fn<A, R>]
  : Z;

type Result<A, Z extends Fn[]> = Z extends [Fn<any, infer R>, ...infer Rest]
  ? Rest extends [Fn, ...any]
    ? Result<A, Rest>
    : Fn<A, R>
  : never;

type Pipe = {
  <A>(): Fn<A, A extends any[] ? A[0] : A>;
  <A, Z>(...fn: Arg1<A, Z>): Fn<A, Z>;
  <A, B, Z>(...fn: Arg2<A, B, Z>): Fn<A, Z>;
  <A, B, C, Z>(...fn: Arg3<A, B, C, Z>): Fn<A, Z>;
  <A, B, C, D, Z>(...fn: Arg4<A, B, C, D, Z>): Fn<A, Z>;
  <A, B, C, D, E, Z>(...fn: Arg5<A, B, C, D, E, Z>): Fn<A, Z>;
  <A, B, C, D, E, F, Z>(...fn: Arg6<A, B, C, D, E, F, Z>): Fn<A, Z>;
  <A, B, C, D, E, F, G, Z>(...fn: Arg7<A, B, C, D, E, F, G, Z>): Fn<A, Z>;
  <A, B, C, D, E, F, G, H, Z extends Fn[]>(
    ...fn: ArgAny<A, B, C, D, E, F, G, H, Z>
  ): Result<A, Z>;
};

If you can think of a better/shorter solution, please post it in this thread and tag me. Thanks.

UPDATE: an improved version

[Bessonov]

@Azarattum great, thanks for sharing! It's me or x resolves to any instead of number in the line 53?

[Azarattum]

@Bessonov this is by design. I did this intentionally to demonstrate that type inference works only for the first 7 functions. Since they are implemented manually with overloads. The rest is just type checked but not inferred. So, if you write (x: string) => 123, on the line 53, you'll get an error. This is a typescript limitation see #50468.

[Azarattum]

I've made an updated version which:

• Supports a pipeline pattern (with automatic pipe inference)
• Supports inference when used as a callback
• Supports spread arguments (with the best inference possible)

Playground

Code

type Fn<A = any[], Z = any> = (...args: A extends any[] ? A : [A]) => Z;

type Arg1<A, Z> = [Fn<A, Z>];
type Arg2<A, B, Z> = [Fn<A, B>, Fn<[B], Z>];
type Arg3<A, B, C, Z> = [...Arg2<A, B, C>, Fn<[C], Z>];
type Arg4<A, B, C, D, Z> = [...Arg3<A, B, C, D>, Fn<[D], Z>];
type Arg5<A, B, C, D, E, Z> = [...Arg4<A, B, C, D, E>, Fn<[E], Z>];
type Arg6<A, B, C, D, E, F, Z> = [...Arg5<A, B, C, D, E, F>, Fn<[F], Z>];
type Arg7<A, B, C, D, E, F, G, Z> = [...Arg6<A, B, C, D, E, F, G>, Fn<[G], Z>];
type ArgAny<A, B, C, D, E, F, G, H, Z extends Fn[]> = [
  ...Arg7<A, B, C, D, E, F, G, H>,
  ...Args<H, Z>
];

type Args<A, Z extends Fn[]> = Z extends [Fn<any, infer R>, ...infer Rest]
  ? Rest extends [Fn, ...any]
    ? [Fn<A, R>, ...Args<R, Rest>]
    : [Fn<A, R>]
  : Z;

type Result<Z> = Z extends [...any, Fn<any[], infer R>]
  ? R
  : Z extends Fn<any[], infer R>[]
  ? R
  : Z;

type To<A, Z, Y> = Y extends false
  ? Fn<A extends [infer T] ? T : A, Result<Z>>
  : Result<Z>;

type Pipe<X = unknown, Y = false> = {
  <A = X>(): To<A, A extends any[] ? A[0] : A, Y>;
  <Z, A = X>(..._: Arg1<A, Z>): To<A, Z, Y>;
  <B, Z, A = X>(..._: Arg2<A, B, Z>): To<A, Z, Y>;
  <B, C, Z, A = X>(..._: Arg3<A, B, C, Z>): To<A, Z, Y>;
  <B, C, D, Z, A = X>(..._: Arg4<A, B, C, D, Z>): To<A, Z, Y>;
  <B, C, D, E, Z, A = X>(..._: Arg5<A, B, C, D, E, Z>): To<A, Z, Y>;
  <B, C, D, E, F, Z, A = X>(..._: Arg6<A, B, C, D, E, F, Z>): To<A, Z, Y>;
  <B, C, D, E, F, G, Z, A = X>(..._: Arg7<A, B, C, D, E, F, G, Z>): To<A, Z, Y>;
  <B, C, D, E, F, G, H, Z extends Fn[], A = X>(
    ..._: ArgAny<A, B, C, D, E, F, G, H, Z>
  ): To<A, Z, Y>;
  <Z extends Fn[], A = X>(..._: Args<A, Z>): To<A, Z, Y>;
};

type Pipeline = <T extends any[]>(...data: T) => Pipe<T, true>;

Examples

// Pipe simple example
const data = pipe(
  (x: boolean, s: string) => 123,
  (x) => [],
  (x) => ({}),
  (x) => 123,
  (x) => "123",
  (x) => false,
  (x) => "123",
  // Arguments below are not inferred automatically
  (x: string) => 123,
  (x: number) => 123,
  (x) => 123,
  (x: number) => ["123"],
);
const result1 = data(true, "123");

// Pipeline verbose example
const process = pipeline(123, 321);
const result2 = process(
  (x, y) => x * y,
// ^? (parameter) x: number
  (x) => x.toString(), 
  (x) => x.toUpperCase()
)

// Pipeline simple example
const result3 = pipeline("123")(
  (x) => x.toUpperCase(),
  (x) => ({ data: x })
)

// Function callback example
function test(callback: (x: string, y: number) => number) {}

test(pipe(
  (x, y) => x.toUpperCase() + y.toPrecision(),
// ^? (parameter) x: string
  (x) => +x
))

// Spread arguments example
const result4 = pipe(
//    ^? const result4: string | number
// This became an array, not a tuple. Therefore there is no "last" function.
//   We make our best guess which is a union of all returns.
  ...[
    (s:any) => 123,
    (s:any) => "123"
  ]
)(42);

[ghost]

@profound7
any chance you could modify your solution to enable async functions?

[Azarattum]

@olmo-hake you can check out my full pipe/pipeline implementation that I’m using for my libfun library here. It supports promises and other promise-like monadic structures.

[janekeilts]

I made this work with async its not clean but works well:

type IsPromise<T> = T extends Promise<any> ? true : false;

type SomePromise<T extends any[]> = T extends [infer First, ...infer Rest]
  ? IsPromise<First> extends true
    ? true
    : SomePromise<Rest>
  : false;

type Fn<A extends unknown[], R> = (...args: A) => R;

type Last<T extends any[]> = T extends [...any[], infer L] ? L : never;

type Result<R extends unknown[]> = SomePromise<R> extends true
  ? (...args: [R[0]]) => Promise<Last<Awaited<R>>>
  : (...args: [R[0]]) => Last<Awaited<R>>;

type ChainOverload = {
  <A extends unknown[], R0>(fn0: Fn<A, R0>): (...args: A) => Result<[R0]>;

  <A extends unknown[], R0, R1>(fn0: Fn<A, R1>, fn1: Fn<[Awaited<R0>], R1>): Result<
    [R0, R1]
  >;
  <A extends unknown[], R0, R1, R2>(
    fn0: Fn<A, R0>,
    fn1: Fn<[Awaited<R0>], R1>,
    fn2: Fn<[Awaited<R1>], R2>
  ): Result<[R0, R1, R2]>;

  <A extends unknown[], R0, R1, R2, R3>(
    fn0: Fn<A, R0>,
    fn1: Fn<[Awaited<R0>], R1>,
    fn2: Fn<[Awaited<R1>], R2>,
    fn3: Fn<[Awaited<R2>], R3>
  ): Result<[R0, R1, R2, R3]>;

  <A extends unknown[], R0, R1, R2, R3, R4>(
    fn0: Fn<A, R0>,
    fn1: Fn<[Awaited<R0>], R1>,
    fn2: Fn<[Awaited<R1>], R2>,
    fn3: Fn<[Awaited<R2>], R3>,
    fn4: Fn<[Awaited<R3>], R4>
  ): Result<[R0, R1, R2, R3, R4]>;

  <A extends unknown[], R0, R1, R2, R3, R4, R5>(
    fn0: Fn<A, R0>,
    fn1: Fn<[Awaited<R0>], R1>,
    fn2: Fn<[Awaited<R1>], R2>,
    fn3: Fn<[Awaited<R2>], R3>,
    fn4: Fn<[Awaited<R3>], R4>,
    fn5: Fn<[Awaited<R4>], R5>
  ): Result<[R0, R1, R2, R3, R5, R5]>;

  // todo: add more here ...
};

const chain: ChainOverload = (() => {}) as any;

const final = chain(
  (one: string) => one,
  async (two) => Number(two),
  (three) => three
);

export const result: number = await final('hey');

[janekeilts]

Also I guess:

import { Is } from '@janekeilts/is';
import { Pipe } from '@Pipe';

type SingleArgFunction = (arg: any) => any;

export const pipe: Pipe = (...fns: SingleArgFunction[]) => {
  const somePromise: boolean = fns.some(Is.Function.async);

  return somePromise
    ? async (arg: any) => {
        let result = arg;

        for (const fn of fns) result = await fn(result);

        return result;
      }
    : (arg: any) => {
        let result = arg;

        for (const fn of fns) result = fn(arg);

        return result;
      };
};