Skip to content

Error When Using OfArrayOfContentParts, ToolCall & PDF document #423

Open
@backkem

Description

@backkem

When using the openai-go SDK to create a chat completion with a ChatCompletionAssistantMessageParamContentUnion that includes OfArrayOfContentParts in combination with tool calling & a PDF docucment, the API returns a 400 Bad Request error with the message:
Invalid parameter: messages with role 'tool' must be a response to a preceding message with 'tool_calls'.

This error seems incorrect as there is no such issue in this request. I assume it's masking some other error. In addition, replacing OfArrayOfContentParts with OfString in the same message structure resolves the issue, and the request succeeds. Replacing the PDF document with a text message also doesn't reproduce the issue.

Steps to Reproduce

Use the following code snippet to create a chat completion request:

package main

import (
    "context"
    "fmt"

    "github.com/openai/openai-go"
)

func main() {
	client := openai.NewClient()
	ctx := context.Background()
	params := openai.ChatCompletionNewParams{
		Model: openai.ChatModelGPT4o,
		Messages: []openai.ChatCompletionMessageParamUnion{
			openai.UserMessage("what's the weather in nyc and describe the document contents."),
			{
				OfAssistant: &openai.ChatCompletionAssistantMessageParam{
					Content: openai.ChatCompletionAssistantMessageParamContentUnion{
						OfArrayOfContentParts: []openai.ChatCompletionAssistantMessageParamContentArrayOfContentPartUnion{},
						// OfString: openai.String(""), // Empty string instead works.
					},
					ToolCalls: []openai.ChatCompletionMessageToolCallParam{
						{
							ID:   "call_123",
							Type: "function",
							Function: openai.ChatCompletionMessageToolCallFunctionParam{
								Name:      "get_weather",
								Arguments: `{"location":"NYC","unit":"f"}`,
							},
						},
					},
				},
			},
			{
				OfTool: &openai.ChatCompletionToolMessageParam{
					Content: openai.ChatCompletionToolMessageParamContentUnion{
						OfString: openai.String("40f"),
					},
					ToolCallID: "call_123",
				},
			},
			// openai.UserMessage("Document content: I'm a PDF document."), // No document instead works
			 openai.UserMessage(
			 	[]openai.ChatCompletionContentPartUnionParam{
			 		{
			 			OfFile: &openai.ChatCompletionContentPartFileParam{
			 				File: openai.ChatCompletionContentPartFileFileParam{
			 					Filename: openai.String("test.pdf"),
			 					FileData: openai.String(`data:application/pdf;base64,JVBERi0xLjQKMSAwIG9iago8PC9UeXBlIC9DYXRhbG9nCi9QYWdlcyAyIDAgUgo+PgplbmRvYmoKMiAwIG9iago8PC9UeXBlIC9QYWdlcwovS2lkcyBbMyAwIFJdCi9Db3VudCAxCj4+CmVuZG9iagozIDAgb2JqCjw8L1R5cGUgL1BhZ2UKL1BhcmVudCAyIDAgUgovTWVkaWFCb3ggWzAgMCA1OTUgODQyXQovQ29udGVudHMgNSAwIFIKL1Jlc291cmNlcyA8PC9Qcm9jU2V0IFsvUERGIC9UZXh0XQovRm9udCA8PC9GMSA0IDAgUj4+Cj4+Cj4+CmVuZG9iago0IDAgb2JqCjw8L1R5cGUgL0ZvbnQKL1N1YnR5cGUgL1R5cGUxCi9OYW1lIC9GMQovQmFzZUZvbnQgL0hlbHZldGljYQovRW5jb2RpbmcgL01hY1JvbWFuRW5jb2RpbmcKPj4KZW5kb2JqCjUgMCBvYmoKPDwvTGVuZ3RoIDUzCj4+CnN0cmVhbQpCVAovRjEgMjAgVGYKMjIwIDQwMCBUZAooRHVtbXkgUERGKSBUagpFVAplbmRzdHJlYW0KZW5kb2JqCnhyZWYKMCA2CjAwMDAwMDAwMDAgNjU1MzUgZgowMDAwMDAwMDA5IDAwMDAwIG4KMDAwMDAwMDA2MyAwMDAwMCBuCjAwMDAwMDAxMjQgMDAwMDAgbgowMDAwMDAwMjc3IDAwMDAwIG4KMDAwMDAwMDM5MiAwMDAwMCBuCnRyYWlsZXIKPDwvU2l6ZSA2Ci9Sb290IDEgMCBSCj4+CnN0YXJ0eHJlZgo0OTUKJSVFT0YK`),
			 				},
			 			},
			 		},
			 	},
			 ),
		},
	}

	_, err = client.Chat.Completions.New(ctx, params)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
	} else {
		fmt.Println("Success")
	}
}

output:

Error: POST "https://api.openai.com/v1/chat/completions": 400 Bad Request {
    "message": "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.",
    "type": "invalid_request_error",
    "param": "messages.[2].role",
    "code": null
  }

Expected Behavior

Seems like this should just work.

sysinfo

SDK Version: openai-go v1.5.0
Go Version: go1.24.1 windows/amd64
Model: gpt-4o

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions