Type inference problem when using inline vs declared generator function

[patrickshipe]

🔎 Search Terms

async generator inference pipeline anonymous

🕗 Version & Regression Information

• This is the behavior in every version I tried, and I reviewed the FAQ for entries about declared vs anonymous function inference

⏯ Playground Link

https://www.typescriptlang.org/play/?target=99&ts=5.6.3#code/JYWwDg9gTgLgBAbzmYYCmAbYA7NcC+cAZlBCHAOTYQAmaAXAM4xRoCGIA9GKSMI2kYUA3AChRbRgE9sAY2IBXOTGARscEGxwAKAJSJRcOLLXM4OMAvgBeOAG0AjABo4AJhcBmFwBYAumMM4NgB3LXgUdCxcbUCjCysnWKDpOUVlVWwAKjhtRggFKFk0fQQkoyJoINDgeG0TbDNZAAslAGs4CCI4PIKikrKjOClgTBpjFux27NcxQcH8JPxEuckZeSIlWRU1bNz8wuKDOfLKkLCc+saJ9s7u-b6j46Nh0fG2uABqOAdZ44W5pZJVapDbpNQ5HoHfpPCpQKrnOqmeDNd63SEPUpPIyXCAYNAAOgwEAA5toKAAxHBsDBwbAKEAAIzQUCELhRk10vwBi2WcE5ogWEhS60223UmhwABVghA9I9LvB4jZ7M43J4fP5xNikUEaDQAPK4OC2YEisFZHJJdEMOAAQWFAEkYMy2Ay8QAeOmM5kAPl5unoduFAHE0LgoGwYNBPfSmVAXAA3CDAGguNjYKQ+x4nOFnGoXHXsm5da3Q44vDBjIufb5cgiBfABIx58KoTA4NAxOZK3nN4VpLYZXal7ODWHw-OIhrI64dEv3Q6YrEVquz6Z1oz-ea93UG3A700DsUQhdlubjlsF6dvSZzu69RcDbUNXEEomkilUmleuOsm+tfkni3Td-TEQUJWwPQxAg6VZX5IA

💻 Code

import { pipeline } from 'node:stream/promises';

async function main() {
  const input = [1, 2, 3, 4];

  await pipeline(
    input,
    async function* (source) {
      for await (const chunk of source) {
        yield chunk * 2;
      }
    },
    async function* (source) {
      for await (const chunk of source) {
        yield chunk + 1;
      }
    },
    async function (source) {
      for await (const chunk of source) {
        console.log('Final numbers', chunk);
      }
    },
  );
}

async function mainTwo() {
  const input = [1, 2, 3, 4];

  const addOne = async function* (
    source: AsyncIterable<number>,
  ): AsyncGenerator<number, void, any> {
    for await (const chunk of source) {
      yield chunk + 1;
    }
  };

  await pipeline(
    input,
    async function* (source) {
      for await (const chunk of source) {
        yield chunk * 2;
      }
    },
    addOne,
    async function (source) {
      for await (const chunk of source) {
        console.log('Final numbers', chunk);
      }
    },
  );
}

main();
mainTwo();

🙁 Actual behavior

When inlining the addOne function (in main), the type of the rest of the pipeline is inferred.

Type of function:

Type of argument in the next function:

When extracting addOne (in mainTwo), even matching the exact same types, the compiler is unable to infer the type of the next async iterable and you get an error for implicit any.

Type of function (exact same as before 🤔 )

Type of argument in the next function not inferred:

🙂 Expected behavior

The second example should behave exactly as the first, and source should be inferred.

Additional information about the issue

I was hesitant to file this issue given the possibility something is wrong with @types/node, but this seems to be beyond the typings because the typings shouldn't know/care whether or not I declare the function inline or reference it.

[Andarist]

With @types/node inlined: TS playground, and slimmed down version: TS playground

[Andarist]

This one is tricky and falls under #47599 umbrella.

The problem is that inference is performed in 2 passes when context-sensitive functions are involved. The first one replaces such functions with "any function type" and blocks inference from it. The problem is that the signature inferred from the first pass has to pass getSignatureApplicabilityError (or well, fail it - as in, it can't result in an applicability error) for the inference to move to the second pass. I verified locally that if we'd move to the second pass this would succeed but that also had some undesired effects on other existing test cases.

So what happens here after this second pass that the third param type gets computed as PipelineTransform<PipelineTransform<number[], any>, any> which is basically equivalent to ReadWriteStream | ((source: ReadWriteStream | AsyncIterable<any>) => AsyncIterable<any>). And your function doesn't satisfy this requirement because it can only accept an async iterable (but not a stream).

The reason why we get this param type is that it's derived from constraints (in absence of legit inference candidates and at this point - as I mentioned - your context-senstivie functions were ignored). The reason why it works when it's inlined is that in that scenario it's also context-sensitive so it's also excluded. So the inference is OK with the signature inferred from the first pass and moves to the second.

The very minimal (even if nonsensical 😉 ) repro:

interface ReadableStream {
  readable: boolean;
  read(size?: number): string;
}

type PipelineTransform<S, U> = ReadableStream | ((source: S) => U);

declare function pipeline<
  A extends Iterable<any>,
  T1 extends PipelineTransform<A, any>,
  T2 extends PipelineTransform<T1, any>,
>(source: A, transform1: T1, transform2: T2): void;

export async function main() {
  const input = [1, 2, 3, 4];

  pipeline(
    input,
    (source) => source,
    (source) => {},
  );
}

export async function mainTwo() {
  const input = [1, 2, 3, 4];

  const identity = (source: number[]) => source;

  pipeline(
    input,
    (source) => source,
    identity, // error 😢
  );
}

[patrickshipe]

Wow, thank you for the explanation! What's even more fascinating is that if I tweak the addOne function in my example to use generics, everything seems to work:

  const addOne = async function* <T extends number>(
    source: AsyncIterable<T>,
  ): AsyncGenerator<number, void, any> {
    for await (const chunk of source) {
      yield chunk + 1;
    }
  };

[Andarist]

Yee, IIRC the generic functions are also skipped by the first pass - although perhaps that's done conditionally.

[patrickshipe]

The inference seems weak though, e.g. the output type T seemes to be the minimum type governed by the extends, not based on what I actually passed in. At least it's a workaround to getting typing to work with the preceding function in the pipeline, since if it returns an incompatible type TS will still catch it.

[typescript-bot]

This issue has been marked as "Design Limitation" and has seen no recent activity. It has been automatically closed for house-keeping purposes.