core[patch]: Adding raiseError Field to Callbacks

[CahidArda]

Hi there!

We want to add a callback to manage rate limiting based on the number of requests or the number of tokens. For this to work, callback needs to be able to stop the execution of the chain. This is not possible in LangChain Js, while it is possible in LangChain Python (see raise_error field). In LangChain Js, the error is simply logged but the chain continues.

To make this possible, we update the BaseRunManager and its subclasses by throwing the error if the new raiseError field of the handler is true.

To give you an idea about the rate limiting callback we want to add to LangChain under community, here is a code sample:

Ratelimiting Callback

import * as uuid from "uuid";
import {
  BaseCallbackHandler, type BaseCallbackHandlerInput
} from "@langchain/core/src/callbacks/base.js";
import {
  CallbackManager
} from "@langchain/core/src/callbacks/manager.js";
import { Serialized } from "@langchain/core/src/load/serializable.js";
import { ChainValues } from "@langchain/core/src/utils/types/index.js";

// mock rate limit request
const checkRateLimit = async (identifier: string) => {
  return false;
};

export class RateLimitError extends Error {};

class RateLimitHandler extends BaseCallbackHandler {
  name = "rate-limit-handler"
  raiseError = true; // will make the callback manager throw the error
  checked = false;   // to call rate limit only once in a pipeline
  identifier: string;

  constructor(identifier: string,  inputs?: BaseCallbackHandlerInput) {
    super(inputs);
    this.identifier = identifier;
  }

  async handleChainStart(_chain: Serialized, _inputs: ChainValues): Promise<void> {
    if (!this.checked) {
      const success = await checkRateLimit(this.identifier);
      if (success) {
        this.checked = true;
      } else {
        throw new RateLimitError("Limit reached!")
      } 
    }
  }
}

const serialized: Serialized = {
  lc: 1,
  type: "constructor",
  id: ["test"],
  kwargs: {},
};

const handler = new RateLimitHandler("ip-address");
const manager = new CallbackManager(undefined);
manager.addHandler(handler);

try {
  await manager.handleChainStart(serialized, ["test"]);
} catch (err) {
  if (err instanceof RateLimitError) {
    // handle rate limit error
    console.log("handling error!")
  }
  // handle other error
}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchainjs-api-refs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 14, 2024 11:48pm
langchainjs-docs	✅ Ready (Inspect)	Visit Preview		May 14, 2024 11:48pm

[jacoblee93]

These errors will be uncatchable if backgrounding is enabled - I think it'd be reasonable to make a callback awaited if this condition is true.

[jacoblee93]

@baskaryan can you take a quick look?

[jacoblee93]

@CahidArda will aim to ship this next week as part of 0.2 - if you have a burning need for it sooner, I can cherry pick and cut a 0.1.x core release