Fix multiple parallel tool call support in streamUI

[mfclarke-cnx]

Bug Description

streamUI doesn't gracefully handle multiple parallel tool calls. Currently when the model makes multiple tool calls, the tool handler gets called multiple times, but the first call to resolve closes the UI steam and the rest throw:

⨯ node_modules/.pnpm/ai@3.1.26_openai@4.48.1_react@18.3.1_solid-js@1.8.17_svelte@4.2.17_vue@3.4.27_zod@3.23.8/node_modules/ai/rsc/dist/rsc-server.mjs (1231:12) @ assertStream
 ⨯ unhandledRejection: Error: .update(): UI stream is already closed.

[camwest]

It looks like this hardcoded "true" is where the error is happening.

ai/packages/core/rsc/stream-ui/stream-ui.tsx

Line 303 in f7002ad

true,

It's a bad assumption. The tool-call event doesn't immediately mean it's the last render call because there may be multiple tool-call events.

[camwest]

A potential fix would be to add a 'all-tool-calls-done' event which gets pushed to stream after the loop right around here:

ai/packages/openai/src/openai-chat-language-model.ts

Line 343 in f7002ad

Then we handle that in stream-ui and finalize the stream there.

The challenge is we'd have to update anything that implements LanguageModelV1 to trigger this event, or version to V2.

[camwest]

Another option is to close the stream on finish:

ai/packages/openai/src/openai-chat-language-model.ts

Lines 382 to 388 in f7002ad

	flush(controller) {
	controller.enqueue({
	type: 'finish',
	finishReason,
	logprobs,
	usage,
	});

Currently not implemented in streamUI:

ai/packages/core/rsc/stream-ui/stream-ui.tsx

Line 314 in f7002ad

// Nothing to do here.

[bneigher]

ah this is the culprit of my issue! been doing this since v3.1.25

[camwest]

@mfclarke-cnx can you change this to a bug instead of enhancement?

[mfclarke-cnx]

@camwest unfortunately I don't have permissions to change the labels. @lgrammel would you be able to update labels please?

[miguelvictor]

I also have this issue but the weird thing about mine is the same tool is being called twice.

[gclark-eightfold]

As tagged by @bneigher, I have an approach I'm working on in #1904, but it's still a WIP.

Main issue I have right now is that because we don't know how many tool calls will be made ahead of time, the parent "streamableUI" doesn't call .done until all tool calls complete.

I think my expectation is that the UI will be added to as the final tool calls stream in.

[camwest]

Main issue I have right now is that because we don't know how many tool calls will be made ahead of time, the parent "streamableUI" doesn't call .done until all tool calls complete.

I think you can get this information at the provider level.

For example the OpenAI provider passes an array of tool_calls so you know when you're finished iterating that it's 'done'

[HECHONG999]

"Parallel support, how to resolve the 'done' error bug now."

[dutsik-p]

Maybe anybody knows how to tune the prompt so the model doesnot put number of tools to call at the same time?

[grrowl]

As a kind of "workaround" you can set parallelToolCalls to false (as introduced in #1937) — but lack of parallel tool calling support really should be mentioned on https://sdk.vercel.ai/docs/troubleshooting/common-issues

Frustrating that this common benefit of AI tool calling is broken on Vercel AI UI :(