Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose the LLM provider response in AISDKError #3779

Open
williamlmao opened this issue Nov 19, 2024 · 0 comments
Open

Expose the LLM provider response in AISDKError #3779

williamlmao opened this issue Nov 19, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@williamlmao
Copy link

Feature Description

I have two other feature requests (#3451, #3208) that I've opened, both of which I will be closing in favor of this one as I believe this suggestion is the best way to handle this.

Currently when you use generateObject or generateText with a function call and the LLM responds with data that does not match your provided schema, the behavior is that the vercel AI SDK will throw a InvalidToolArgumentsError or AISDKError, which just returns the zod parsing error.

The issue here is that now the entire generateText or generateObject call is throwing a zod error, and the response from the provider is no longer accessible at all, even though that was technically a successful response. This is a problem for a few reasons

  1. We lose access to what the LLM actually generated. Sometimes the response is recoverable / fixable, but we do not have an opportunity to handle those error cases.
  2. Usage data is now lost (I know we could still save it with fetch intercepts, but this means we have to work with the data outside of the calling function scope)

I think of the vercel AI SDK as an extension of calling these providers, so it feels weird to me that a successful response from the LLM provider is completely inaccessible due to a zod parse error. I do agree that a generateText/generateObject should throw this error, but it should include the full LLM provider response.

In the case of #3208 when Groq throws an error directly from their API, the error that groq throws should be passed through the AISDKError.

If the error was thrown like this, that would be a huge, huge help.

try {
  const {result} = await generateText({
    ...
  });
} catch (e: InvalidToolArgumentsError) {
  e.response
  // Do something with the e.response
 // Log usage data still
}

Use Cases

No response

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant