Log requests and responses to Open AI

[iAMSagar44]

Has anyone managed to find a way to log the requests and responses to/from Open AI using Spring AI.
I have tried various logging settings in the application.properties but no luck.
With Langchain4J, there is a specific property for logging the requests and responses to/from the AI models, but couldn't find anything similar with Spring AI.

[iamroberthanson]

I was looking to do this too, and did a little digging.

RestClient uses a default RestClient.Builder bean, but you can override it with your own. Here I took the default implementation and added a requestInterceptor() to the builder. The code is in Groovy, so you will need to add some semi-colons and stuff.

// Groovy code   

/**
 * This overrides the default factory bean for the REST client, and allows us to customize it.
 * In this case we are adding an interceptor to log the request and response.
 * The rest of this code comes from the default implementation in
 *   org.springframework.boot.autoconfigure.web.client.RestClientAutoConfiguration
 */
@Bean
@Scope("prototype")
RestClient.Builder restClientBuilder (RestClientBuilderConfigurer restClientBuilderConfigurer)
{
    RestClient.Builder builder = RestClient.builder()
        .requestFactory(ClientHttpRequestFactories.get(ClientHttpRequestFactorySettings.DEFAULTS))
        .requestInterceptor(new RestClientInterceptor())
    return restClientBuilderConfigurer.configure(builder);
}

Then your interceptor would look like this. This is just some working test code right now, so it just prints to the console.

import org.apache.commons.io.IOUtils
import org.springframework.http.HttpRequest
import org.springframework.http.client.ClientHttpRequestExecution
import org.springframework.http.client.ClientHttpRequestInterceptor
import org.springframework.http.client.ClientHttpResponse
import org.springframework.util.StreamUtils

import java.nio.charset.Charset

class RestClientInterceptor implements ClientHttpRequestInterceptor
{

    @Override
    ClientHttpResponse intercept (HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException
    {
        // log the request
        logRequest(request, body)
        
        // execute the request
        ClientHttpResponse response = execution.execute(request, body)
        
        // wrap the response that buffers the body, so that we can read from it multiple times
        BufferingClientHttpResponseWrapper responseWrapper = new BufferingClientHttpResponseWrapper(response)

        // log the response
        logResponse(responseWrapper)

        // return the wrapped response
        return responseWrapper
    }

    static void logRequest (HttpRequest request, byte[] bytes)
    {
        println "Request: ${request.method} ${request.URI}"
        println "Headers: ${request.headers}"
        println "Body: ${new String(bytes)}"
    }

    static void logResponse (ClientHttpResponse response)
    {
        println "Response: ${response.statusCode} ${response.statusText}"
        println "Headers: ${response.headers}"
        println "Body: ${IOUtils.toString(response.body, Charset.forName('UTF-8')).length()}"
    }

    // A Groovy-ier version of:
    // https://github.com/spring-projects/spring-framework/blob/main/spring-web/src/main/java/org/springframework/http/client/BufferingClientHttpResponseWrapper.java
    static class BufferingClientHttpResponseWrapper implements ClientHttpResponse
    {
        @Delegate ClientHttpResponse response;
        byte[] body;

        BufferingClientHttpResponseWrapper (ClientHttpResponse response)
        {
            this.response = response;
        }

        @Override
        InputStream getBody() throws IOException
        {
            if (this.body == null) {
                this.body = StreamUtils.copyToByteArray(this.response.getBody())
            }
            return new ByteArrayInputStream(this.body)
        }
    }
}

Because you can only read the body of the response once, you need to wrap the response in a wrapper class that buffers the body. If you are using Java, you can use the link in the comment for the wrapper class to get the Java-ish source.

[apappascs]

The Spring AI team added the SimpleLoggerAdvisor that logs the request and response data of the ChatClient.
https://docs.spring.io/spring-ai/reference/api/chatclient.html#_logging

For custom implementation, https://docs.spring.io/spring-ai/reference/api/advisors.html#_logging_advisor

[shivangshah]

It's not the same. SimplerLoggerAdvisor seems to be only logging the request and the final response from the chat model. It does not intercept the internal tool calls and definitely does not log them.

[apappascs]

I see. thank you @shivangshah for mentioning that.

[rwankar]

I posted a Java version of the Rest logging here that worked with 1.0.0-SNAPSHOT and Spring Boot 3.4.3. (#883 (comment))

[qsLI]

Need something like what langchain ConsoleCallbackHandler do: