Function calling - FunctionCallingOptions doesn't work with OpenAiChatClient/AzureOpenAiChatClient

[tha2015]

Bug description
Function calling doesn't happen when using FunctionCallingOptions with OpenAiChatClient/AzureOpenAiChatClient

Environment
spring-ai-core-1.0.0-20240420.063719-118.jar
spring-ai-openai-1.0.0-20240420.063719-101.jar
spring-ai-azure-openai-1.0.0-20240420.063719-112.jar

Steps to reproduce

    // kotlin code
    val functionOptions = FunctionCallingOptions.builder()
        .withFunction("weatherFunction")
        .build()

    val response: ChatResponse = chatClient.call(
        Prompt(
            listOf(UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?")),
            functionOptions
        )
    )

    println(response.result.output.content)

OpenAiChatClient:

Sorry, as an AI, I don't have real-time capabilities to provide current weather updates. Please check a reliable weather forecasting website or app for the most accurate and up-to-date information.

AzureOpenAiChatClient:

Sorry, as an AI, I don't have real-time capabilities to provide current weather updates. Please check a reliable weather forecasting website or app for this information.

Expected behavior
Function calling should happen and response should contain function call result.

Analysis
AzureOpenAiChatClient.toAzureChatCompletionsOptions and OpenAiChatClient.createRequest don't use FunctionCallingOptions.getFunctions() for building the request, so function call didn't happen.

[tzolov]

@tha2015 where is your weatherFunction defined? Not clear from your snippets above?

As the documentation suggests you can [register your functions] as plain Function Bean or FunctionCallback wrapper.

Here is a working integration tests for FunctionCallback wrapper: AzureOpenAiChatClientFunctionCallIT.

And you can find more working ITs here: https://github.com/spring-projects/spring-ai/tree/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/azure/tool

Let me know if this helps

[tzolov]

Also mind that the property spring.ai.azure.openai.chat.options.model has been renamed to spring.ai.azure.openai.chat.options.deployment-name.

[tzolov]

Also, I just updated to 1.0.0.beta8: 7252ba1

[tha2015]

This is how the weatherFunction is defined in my code:

    @Bean
    fun weatherFunction(objectMapper: ObjectMapper): FunctionCallbackWrapper<Request, Response> {
        val result =
            FunctionCallbackWrapper.builder(MockWeatherService())
                .withName("weatherFunction")
                .withDescription(
                    "Get the weather in location (e.g. San Francisco or Tokyo or Paris)")
                .withObjectMapper(objectMapper)
                .build()
        return result
    }

I will try with the newer build and update the result.

[edumaxsantos]

I've been having the same issue with OpenAI. Just to test something, I copied OpenAiChatClient class and changed only one thing.

I actually didn't have the time to understand what is the purpose of IS_RUNTIME_CALL, although this tweak helped me make it work.

if (this.defaultOptions != null) {
            // I removed '!' that was modifying IS_RUNTIME_CALL
            Set<String> defaultEnabledFunctions = this.handleFunctionCallbackConfigurations(this.defaultOptions,
                    IS_RUNTIME_CALL);

            functionsForThisRequest.addAll(defaultEnabledFunctions);

            request = ModelOptionsUtils.merge(request, this.defaultOptions, OpenAiApi.ChatCompletionRequest.class);
        }

[tha2015]

I could not setup the spring-ai project to run the tests, and when I copying the test to my project it failed. I'm attaching the screenshot of the code where it failed for your reference. My understanding is FunctionCallingOptions was handled like a normal ChatOptions (without tools property) and so the tools property was not used.

[tha2015]

@tzolov I found out why the FunctionCallingOptions IT test passed. It is because that test case functionCallWithPortableFunctionCallingOptions doesn't have assertions like other test cases. I believe that if you add assertThat(response.getResult().getOutput().getContent()).contains("30", "10", "15"); , the test will fail.