Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUERY] About chat context management tools in Azure OpenAI SDK #43359

Open
kissycn opened this issue Dec 12, 2024 · 0 comments
Open

[QUERY] About chat context management tools in Azure OpenAI SDK #43359

kissycn opened this issue Dec 12, 2024 · 0 comments
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-triage Workflow: This issue needs the team to triage. OpenAI question The issue doesn't require a change to the product in order to be resolved. Most issues start as that

Comments

@kissycn
Copy link

kissycn commented Dec 12, 2024

Query/Question
Is there a built-in tool or utility in the Azure OpenAI SDK for managing chat context (history and message truncation)?
Currently, I'm manually managing the chat messages list and context, like this:

List<ChatRequestMessage> chatMessages = new ArrayList<>();
chatMessages.add(new ChatRequestSystemMessage("..."));
chatMessages.add(new ChatRequestUserMessage("..."));
chatMessages.add(new ChatRequestAssistantMessage("..."));
chatMessages.add(new ChatRequestUserMessage("..."));


IterableStream<ChatCompletions> chatCompletionsStream = client.getChatCompletionsStream(deploymentOrModelId,
        new ChatCompletionsOptions(chatMessages));

I'm looking for functionality that could help with:

Managing conversation history
Automatic message truncation when context length is exceeded
Smart context window management

If such tools don't exist in the SDK, are there any recommended patterns or best practices for implementing this?

@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-triage Workflow: This issue needs the team to triage. OpenAI question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-triage Workflow: This issue needs the team to triage. OpenAI question The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Projects
None yet
Development

No branches or pull requests

1 participant