RFC: TypeScript SDK Unit Testing

[jpwilliams]

Note

@inngest/test is now available that follows parts of this RFC.

Please direct any issues or PRs at that package! 🙂

---

The Inngest Dev Server lets you test your functions alongside your usual code, but we need ways to programmatically assert the functionality of functions and steps.

We'll start by using unit testing tools like Jest, Mocha, Vitest, or similar to thoroughly test functions and steps. Later, we'll introduce integration testing helpers that boot up and communicate with an Inngest Dev Server, which is exactly how our SDKs run integration tests now.

Feedback

We'd love your feedback on these tools - whether they meet your needs for testing functions and any improvements or changes you suggest.

Early usage

We're assessing the tooling and techniques based on this thread in:

• Add experimental @inngest/test package inngest-js#688
• Initial @inngest/test package inngest-js#704
• Add inngest.experimental.mocked inngest-py#152

Unit Testing

We'll always import your function as normal. This also allows you to use common tooling like Jest to mock any other libraries and imports you may need to before you import the function itself, meaning Inngest's tools can be unopinionated about how you do that.

We also import TestThing which we'll use as a thin wrapper over the usual functionality to execute your function.

import { myFn } from "./myFn";
import { TestThing } from "inngest/test";

Creating a test function that we can execute is really easy:

const testFn = new TestThing({ function: MyFn });

We now have a pre-mocked Inngest function that we can execute and assert the outputs of!

First, let's set some defaults so that our tests themselves stay nice and clean. We'll just add the events that will trigger the function.

const testFn = new TestThing({
  function: MyFn,
  events: [{ name: "demo/event.sent" }],
});

// or we could keep the original instance for re-use it across many test cases

const demoTestFn = testFn.withDefaults({
  events: [{ name: "demo/event.sent" }],
});

Now let's execute the function. This one should resolve immediately.

const { result } = await testFn.execute();

expect(result).toMatchObject({
  type: "function-resolved",
  data: "Hello, World!",
});

result is what that individual execution returned with. It could be one of:

• function-resolved - the run has finished successfully
• function-rejected - the run has finished with an error
• step-ran - a step.run() call was executed
• steps-found - we found some steps that were queued

Typing will show you which values you can assert for each of these cases.

We can also, then, use this method to mock step outputs and ensure that future steps are run.

const { result } = await testThing.execute({
  steps: [{ id: "foo", data: "foo result!" }],
});

expect(result).toMatchObject({
  type: "step-ran",
  id: "bar-after-foo",
  data: "bar result!",
});

While we're asserting the output, the TestThing is also mocking step methods such that we can assess that they've been run correctly. This is great for testing complex logic where certain chains of steps are run conditionally.

const { ctx } = await testThing.execute({
  steps: [{ id: "foo", data: "foo result!" }],
});

expect(ctx.step.run).toHaveBeenCalledTimes(2);
expect(ctx.step.run).toHaveBeenCalledWith("foo");
expect(ctx.step.run).toHaveBeenCalledWith("bar-after-foo");

Alternatively, we can access the state of the function, which provides us with a map of the tooling used so we can make assertions about both their presence and return values.

const { state } = await testThing.execute({
  steps: [{ id: "foo", data: "foo result!" }],
});

expect(state.foo).toHaveBeenCalled();
expect(state.foo).toHaveReturnedWith("foo result!");

expect(state["bar-after-foo"]).toHaveBeenCalled();
expect(state["bar-after-foo"]).toHaveReturnedWith("bar result!");

The exposure of the input ctx for the function means we can also now access details such as runId. This can be crucial for asserting that steps correctly use those values. For example, here we test that an event is correctly sent using the run ID passed to the function:

const { ctx } = await testThing.execute();

expect(ctx.step.sendEvent).toHaveBeenCalledWith("send-run-event", {
  name: "run-event",
  data: expect.objectContaining({
    runId: `${ctx.runId}-plus-some-data`,
  }),
});

Testing individual executions can only get us so far, though, and still requires some understanding of how Inngest is executing the function. For more holistic unit testing, we can use the run output to be able to continue executing the function or wait for particular events to occur.

For example, here we'll execute a function with some data and wait for the function to be fully resolved.

const { run } = await testThing.execute();

const { result } = await run.waitFor({
  type: "function-resolved",
});

expect(result).toMatchObject({
  data: "Hello, World! foo result! bar result!",
});

We can also assert that steps ran during that time, just like with a regular execution.

const { run } = await testThing.execute();

const { result, state } = await run.waitFor({
  type: "function-resolved",
});

expect(state.foo).toHaveReturnedWith("foo result!");
expect(state["bar-after-foo"]).toHaveReturnedWith("bar result!");

expect(result).toMatchObject({
  data: `Hello, World! ${state.foo.result} ${state["bar-after-foo"].result}`,
});

Note that state here is from the run.waitFor() call, meaning it's accumulated throughout all of the executions required to get to that point. This means we can make assertions about the function's behaviour overall, even if not all steps are run in every execution due to non-determinism.

Fine-grained control is also available (which is what run.waitFor() uses internally), but is likely not how folks would want to test due to then needing to understand how Inngest is stepping over the function.

let { run } = await testThing.execute();
({ run } = await run.next());

A good example of waitFor() is to test the logic of a particular step. Given a particular event input, I want my function to run organically and ensure that step foo registers the correct waitForEvent:

const userId = "123";

const { run } = await testThing.execute({
  events: [{ name: "demo/event.sent", data: { userId } }],
});

const { result } = await run.waitFor({
  type: "steps-found",
  steps: expect.arrayContaining([
    expect.objectContaining({
      id: "foo",
      op: StepOpCode.WaitForEvent,
      opts: {
        if: `event.data.userId === "${userId}-12345"`,
      },
    }),
  ]),
});

[michealroberts]

Why not Vitest rather than Jest?

[jpwilliams]

Vitest should work too. 🙂 Wanting to ensure mock/assertion compatibility with the major testing frameworks. I'll add these to the list.

[antonio-bluco]

Thank you for this, will love to try this out!

Where can we download the 'inngest/test' package? Or are these not available to use yet?

[jpwilliams]

Not available yet; we're gathering feedback on these APIs before we build to make sure they'd suit everyone's needs.

Does this look like you'd be able to achieve everything needed?

[danielpowell4]

I'm stoked that this is something you all are working on.

We've been doing a mix of things to date to get this sort of functionality

Homegrown solutions

Create inngest caller, test the function

Currently our best approach is making an inngest caller that we can pass an inngest function to a la:

import type { InngestFunction } from 'inngest';
import { uniqueId } from 'lodash';

import type { InngestEvents } from '@/server/types/inngest';
// type InngestEvents = { 'some-events-for-inngest-schema': { 'expected': { 'payload': object  }} }

export const createInngestCaller = (
  inngestFunction: InngestFunction<any, any, any>, // eslint-disable-line @typescript-eslint/no-explicit-any
) => {
  const caller = async (opts?: {
    data?: any; // eslint-disable-line @typescript-eslint/no-explicit-any
    eventName?: keyof InngestEvents;
    user?: any; // eslint-disable-line @typescript-eslint/no-explicit-any
  }) => {
    const { data = {}, eventName = 'inngest/function-invoked', user } = opts ?? {};
    const execution = inngestFunction['createExecution']({
      partialOptions: {
        data: {
          event: {
            data,
            name: eventName,
            user,
          },
        } as never,
        headers: {},
        reqArgs: [],
        runId: uniqueId(),
        stepCompletionOrder: [],
        stepState: {},
      },
      version: 1,
    });

    const res = await execution.start();
    switch (res.type) {
      case 'function-resolved':
        return res.data;
      case 'function-rejected':
        throw new Error((res.error as { message: string })?.message);
      case 'step-not-found':
        throw new Error(`Step not found: ${res.step.displayName}`);
      case 'step-ran':
      case 'steps-found':
        return res;
    }
  };

  return caller;
};

We then use it in jest as follows

const myFunction = inngest.createFunction(
  { id: 'some-id', middleware: [prismaMiddleware], retries: 5 },
  {. event: 'some-event' },
  async ({ event: { data }, prisma }) => {
    /* biz logic here */
  }
)

describe('my function', () => {
  const inngestCaller = createInngestCaller(myFunction);
  
  it('should operate nicely', () => {
    const result = await inngestCaller({
        data: { foo: 'bar' },
      });

      expect(result).toBe('things worked great');
  })
})

Sending pieces of 'step' that are easier to mock

In other situations, we've been sending things from 'step' to sub-utilities so they are more easily unit testable a la:

// handler has unit tests
export const someEventHandler = async ({
  event: {
    data: { uuid },
  },
  prisma,
  step,
}: FunctionOptions) => {
  await anotherUtilThatHasTests({
    event: { uuid, trigger: 'biz-logic' },
    prisma,
    sendEvent: step.sendEvent,  // <-- NOTICE SENDING 'sendEvent' off of 'step' to keep composable + small (also easier to mock in jest)
  });
};

// actual inngest function
export const invoiceSyncedWorkflow = inngest.createFunction(
  { id: 'my-id-here', middleware: [prismaMiddleware] },
  [{ event: 'my-event-here' }],
  someEventHandler,
);

Feedback / Questions

1. I'm curious if '.execute()' is the only planned API available new TestThing a la

const testFn = new TestThing({
  function: MyFn,
  events: [{ name: "demo/event.sent" }],
});

// or we could keep the original instance for re-use it across many test cases

const demoTestFn = testFn.withDefaults({
  events: [{ name: "demo/event.sent" }],
});

// Now let's execute the function. This one should resolve immediately.

const { result } = await testFn.execute(); // <-- can I call other things?

2. Will there be a way to bypass a waitFor condition in a function?

Example: I don't want to wait 1 day in my test to send a drip email campaign, but I do want to test the outcome would happen

Perhaps I misread how the 'state' will accumulate with waitFor's 🙃

[jpwilliams]

Thanks for the write-up, @danielpowell4! 🙌 Awesome to see that we're all thinking along the same lines.

execute() is indeed the main API planned, but run.waitFor() will also use this under the hood (e.g. in InngestTestRun.ts). I think it's much more likely that you'll want to wait for a particular event to occur (like the run ending) before making any assertions.

For example:

import { InngestTestEngine } from "inngest/test";
import { helloWorld } from "./helloWorld";

describe("helloWorld", () => {
  const t = new InngestTestEngine({
    function: helloWorld,
    events: [{ name: "demo/event.sent" }],
  });

  test("returns a greeting based on the event", async () => {
    const { run } = await t.execute();

    const {
      result,
      ctx: {
        event: { name: eventName },
      },
    } = await run.waitFor("function-resolved");

    expect(result).toMatchObject({
      type: "function-resolved",
      data: { message: `Hello ${eventName}!` },
    });
  });
});

Testing individual executions shouldn't be too common as it's starting to test Inngest's specific execution model which isn't something that any of your code (tests or otherwise) should be considering.

Is there other tooling that you'd like to see complement this?

Regarding bypassing steps that "pause" execution such as step.sleep() or step.waitForEvent(), the idea is that you'd use the ability to mock the step outputs for this. For example:

// Mock the `my-wait-id` step to simulate it finding a similar event
const { run } = await t.execute({
  steps: [{ id: "my-wait-id", data: null }],
});

As a further improvement, we may opt to use a generic handler instead, allowing you to mock data or errors based on the function input too.

const { run } = await t.execute({
  steps: [
    {
      id: "my-wait-id",
      handler: (ctx) => {
        return {
          name: `demo/event.waitfor.${ctx.event.name}`,
          data: { some: "data" },
        } satisfies EventPayload;
      },
    },
  ],
});