Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement conversation context and streaming with OllamaSharp #310

Merged
merged 18 commits into from
Dec 3, 2024

Conversation

kborowinski
Copy link
Contributor

@kborowinski kborowinski commented Nov 21, 2024

PR Summary

This PR refactors the AIShell.Ollama.Agent to use the OllamaSharp library, enhancing functionality and maintainability. Key updates include support for conversation context, streaming, improved settings management, and better error handling.

Important:
If you choose this PR, then close Implement settings file for Ollama agent

PR Context

This update improves the AIShell.Ollama.Agent by:

  1. Switching to OllamaSharp:

    • Simplifies API calls with OllamaApiClient.
    • Supports both streaming and non-streaming responses.
  2. Introducing Context Support:

    • Enables the agent to remember previous responses, improving conversational flow.
  3. Enhancing Configuration Management:

    • Manages settings via ollama.config.json with real-time updates using FileSystemWatcher.
    • Provides a sample configuration file to simplify setup.
  4. Improving Chat Interaction:

    • Adds robust error handling, especially when prerequisites (like the Ollama server) are not met.
    • Validates settings (e.g., Model and Endpoint) to ensure proper functionality.
    • Supports local and remote Ollama endpoints

This PR also aligns with prerequisites outlined in issue #155 and improves the user experience with clearer feedback and enhanced features.

@kborowinski kborowinski changed the title Implement settings file for Ollama agent Implement conversation context and streaming with OllamaSharp Nov 21, 2024
Copy link
Member

@daxian-dbw daxian-dbw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kborowinski Thanks for making improvements to the ollama agent!

if (Process.GetProcessesByName("ollama").Length is 0)
{
host.RenderFullResponse("Please be sure the Ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");
host.MarkupWarningLine($"[[{Name}]]: Please be sure the Ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
host.MarkupWarningLine($"[[{Name}]]: Please be sure the Ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");
host.WriteErrorLine($"[{Name}]: Please be sure the Ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, given that the prompt will be @ollama when the agent is in use, maybe we don't need to include the [{Name}] part in the error message.

Copy link
Contributor Author

@kborowinski kborowinski Nov 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I think we should check if ollama is running only when the Endpoint is set to localhost. This will enable support for remote ollama endpoints.

if (IsLocalHost().IsMatch(_client.Uri.Host) && Process.GetProcessesByName("ollama").Length is 0)
{
    host.WriteErrorLine("Please be sure the Ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");
    return false;
}

and

/// <summary>
/// Defines a generated regular expression to match localhost addresses
/// "localhost", "127.0.0.1" and "[::1]" with case-insensitivity.
/// </summary>
[GeneratedRegex("^(localhost|127\\.0\\.0\\.1|\\[::1\\])$", RegexOptions.IgnoreCase)]
internal partial Regex IsLocalHost();

<configuration>
<packageSources>
<clear />
<add key="PowerShell_PublicPackages" value="https://pkgs.dev.azure.com/powershell/PowerShell/_packaging/powershell/nuget/v3/index.json" />
<add key="NuGet" value="https://api.nuget.org/v3/index.json" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have saved the OllamaSharp package to the MS feed (we're required to use it only), so you can revert the changes in this file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OllamaSharp was just updated to 4.0.8 and you do not have this version on the MS feed yet, so I will keep it for the moment. I guess in the future you will pull it automatically to the MS feed.

shell/agents/AIShell.Ollama.Agent/OllamaAgent.cs Outdated Show resolved Hide resolved
shell/agents/AIShell.Ollama.Agent/OllamaAgent.cs Outdated Show resolved Hide resolved
shell/agents/AIShell.Ollama.Agent/OllamaAgent.cs Outdated Show resolved Hide resolved
shell/agents/AIShell.Ollama.Agent/Settings.cs Outdated Show resolved Hide resolved
shell/agents/AIShell.Ollama.Agent/OllamaAgent.cs Outdated Show resolved Hide resolved
shell/agents/AIShell.Ollama.Agent/OllamaAgent.cs Outdated Show resolved Hide resolved
@kborowinski
Copy link
Contributor Author

@daxian-dbw Thank you very much for your time and the review. I did implement the changes you required. If you have any additional suggestions, please let me know.

@kborowinski
Copy link
Contributor Author

@daxian-dbw Apologies for the repeated pings. Please take a look whenever you have a chance.

@daxian-dbw
Copy link
Member

daxian-dbw commented Nov 30, 2024 via email

@StevenBucher98
Copy link
Collaborator

This is AWESOME @kborowinski ! Thanks for the contributions! Will definitely make the Ollama agent better 😄 I will work to modify the agent creation instructions to either comply with these changes or keep the old code since it may serve better as example, thanks again!

@daxian-dbw
Copy link
Member

daxian-dbw commented Dec 3, 2024

All look good! I made a couple minor changes to the style, nothing big. Thanks again @kborowinski!

@daxian-dbw daxian-dbw merged commit 943b701 into PowerShell:main Dec 3, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants