Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ERI server assistant #231

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
8802657
Added EDI to the dictionary
SommerEngineering Dec 4, 2024
42fb6cb
Added the EDI assistant
SommerEngineering Dec 4, 2024
ae2209e
Update
SommerEngineering Dec 4, 2024
d9bc74b
Marked the EDI assistant as a preview feature
SommerEngineering Dec 4, 2024
e40d95f
Added programming languages
SommerEngineering Dec 4, 2024
2aa912f
Added data sources
SommerEngineering Dec 4, 2024
4027e4f
Fixed layout
SommerEngineering Dec 4, 2024
4781fc6
Added auth methods
SommerEngineering Dec 4, 2024
898d1a0
Added generate method to trigger validation
SommerEngineering Dec 4, 2024
48d0941
Improved spelling
SommerEngineering Dec 4, 2024
e1c62e8
Fixed layout issues when selecting `other` items
SommerEngineering Dec 8, 2024
c6f78fc
Implemented remaining fields for EDI assistant
SommerEngineering Dec 8, 2024
65feb13
Added the EDI specification
SommerEngineering Dec 9, 2024
9bc6d04
Added server name & description
SommerEngineering Dec 9, 2024
3f5ebc5
Renamed EDI to ERI
SommerEngineering Dec 14, 2024
d06c913
Added headers to sections
SommerEngineering Dec 18, 2024
370de3a
Added allowed LLM provider setting (data protection)
SommerEngineering Dec 18, 2024
7ed32f4
Added header for provider selection
SommerEngineering Dec 18, 2024
bbc906f
Implemented a form change event
SommerEngineering Dec 19, 2024
720c530
Added an auto-save option
SommerEngineering Dec 19, 2024
07df50b
Renamed embedding dialog for embedding providers
SommerEngineering Dec 25, 2024
75b11ff
Added embedding information
SommerEngineering Dec 25, 2024
75705eb
Improved data source port handling
SommerEngineering Dec 27, 2024
b4eef64
Renamed embedding infos
SommerEngineering Dec 27, 2024
08ea4a9
Removed unnecessary retrieval description
SommerEngineering Dec 27, 2024
75e12c7
Improved layout
SommerEngineering Dec 27, 2024
6c0da03
Spelling
SommerEngineering Dec 27, 2024
bc49273
Added explanations
SommerEngineering Dec 27, 2024
81790b9
Spelling
SommerEngineering Dec 27, 2024
169e1cb
Limit the embedding name and type length
SommerEngineering Dec 27, 2024
5f54229
Improved link null handling
SommerEngineering Dec 27, 2024
67ab9c6
Implemented the retrieval process configuration
SommerEngineering Dec 27, 2024
36c6584
Ensure unique names for embedding methods and retrieval processes
SommerEngineering Dec 27, 2024
c7b207e
Fixed deleting a used embedding method
SommerEngineering Dec 27, 2024
dff83c3
Fixed retrieval process editing
SommerEngineering Dec 27, 2024
3134781
Fixed unique embedding method and retrieval process name rules when e…
SommerEngineering Dec 27, 2024
b6f4307
Formatting
SommerEngineering Dec 27, 2024
3362051
Disable auto-save when preselection is disabled
SommerEngineering Dec 27, 2024
0eda014
Added ERI server settings
SommerEngineering Dec 27, 2024
ff8eb07
Fixed spelling
SommerEngineering Dec 28, 2024
ece0788
Allow to manage multiple ERI server configurations
SommerEngineering Dec 28, 2024
ece4c12
Updated RAG task list
SommerEngineering Dec 28, 2024
5f9cebb
Updated RAG task list
SommerEngineering Dec 28, 2024
358566b
Fixed spelling
SommerEngineering Dec 28, 2024
6684a31
Updated changelog
SommerEngineering Dec 28, 2024
e998e97
Allow adding issues to the issue list
SommerEngineering Dec 29, 2024
e932c21
Added manual validation steps
SommerEngineering Dec 29, 2024
45ca0cd
Implemented rate limit handling for providers
SommerEngineering Dec 31, 2024
3828e2a
Added possibility to show the entire chat thread
SommerEngineering Dec 31, 2024
d9a9acc
Added the ability to hide some AI responses
SommerEngineering Dec 31, 2024
9b2a375
Added prompt translation for auth values
SommerEngineering Dec 31, 2024
aefd8ad
Rename
SommerEngineering Dec 31, 2024
e654408
Improved ERI specification handling
SommerEngineering Dec 31, 2024
36487b0
Rename
SommerEngineering Dec 31, 2024
f8245c4
Implemented prompting
SommerEngineering Dec 31, 2024
06bd412
Fixed auto-save of the selected provider
SommerEngineering Dec 31, 2024
f40222e
Fixed a bias of the day workspace bug in case the workspace component…
SommerEngineering Dec 31, 2024
0783206
Ensure that the ERI workspace is present & create bias workspace only…
SommerEngineering Dec 31, 2024
e9f4d5f
Added the possibility that a user selects a directory
SommerEngineering Jan 1, 2025
3f22380
Added component to let the user choose a directory
SommerEngineering Jan 1, 2025
170cda4
Added configuration for writing code into the file system
SommerEngineering Jan 1, 2025
cbb0a36
Implemented writing generated code to disk
SommerEngineering Jan 1, 2025
afcc8a6
Merge branch 'main' into add-edi-assistant
SommerEngineering Jan 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Things we are currently working on:
- [x] ~~App: Metadata for providers (which provider offers embeddings?) (PR [#205](https://github.com/MindWorkAI/AI-Studio/pull/205))~~
- [x] ~~App: Add an option to show preview features (PR [#222](https://github.com/MindWorkAI/AI-Studio/pull/222))~~
- [x] ~~App: Configure embedding providers (PR [#224](https://github.com/MindWorkAI/AI-Studio/pull/224))~~
- [x] ~~App: Implement an [ERI](https://github.com/MindWorkAI/ERI) server coding assistant (PR [#231](https://github.com/MindWorkAI/AI-Studio/pull/231))~~
- [ ] App: Management of data sources (local & external data via [ERI](https://github.com/MindWorkAI/ERI))
- [ ] Runtime: Extract data from txt / md / pdf / docx / xlsx files
- [ ] (*Optional*) Runtime: Implement internal embedding provider through [fastembed-rs](https://github.com/Anush008/fastembed-rs)
Expand Down
2 changes: 2 additions & 0 deletions app/MindWork AI Studio.sln.DotSettings
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<wpf:ResourceDictionary xml:space="preserve" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:s="clr-namespace:System;assembly=mscorlib" xmlns:ss="urn:shemas-jetbrains-com:settings-storage-xaml" xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=AI/@EntryIndexedValue">AI</s:String>
<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=EDI/@EntryIndexedValue">EDI</s:String>
<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=ERI/@EntryIndexedValue">ERI</s:String>
<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=LLM/@EntryIndexedValue">LLM</s:String>
<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=LM/@EntryIndexedValue">LM</s:String>
<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=MSG/@EntryIndexedValue">MSG</s:String>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ the logistical challenges that come with an increasing number of participants.
SystemPrompt = SystemPrompts.DEFAULT,
};

protected override void ResetFrom()
protected override void ResetForm()
{
this.inputContent = string.Empty;
this.contentLines.Clear();
Expand Down
15 changes: 13 additions & 2 deletions app/MindWork AI Studio/Assistants/AssistantBase.razor
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<InnerScrolling HeaderHeight="6em">
<ChildContent>
<MudForm @ref="@(this.form)" @bind-IsValid="@(this.inputIsValid)" @bind-Errors="@(this.inputIssues)" Class="pr-2">
<MudForm @ref="@(this.form)" @bind-IsValid="@(this.inputIsValid)" @bind-Errors="@(this.inputIssues)" FieldChanged="@this.TriggerFormChange" Class="pr-2">
<MudText Typo="Typo.body1" Align="Align.Justify" Class="mb-6">
@this.Description
</MudText>
Expand Down Expand Up @@ -35,11 +35,22 @@
<div id="@BEFORE_RESULT_DIV_ID" class="mt-3">
</div>

@if (this.ShowResult && this.resultingContentBlock is not null)
@if (this.ShowResult && !this.ShowEntireChatThread && this.resultingContentBlock is not null)
{
<ContentBlockComponent Role="@(this.resultingContentBlock.Role)" Type="@(this.resultingContentBlock.ContentType)" Time="@(this.resultingContentBlock.Time)" Content="@(this.resultingContentBlock.Content)"/>
}

@if(this.ShowResult && this.ShowEntireChatThread && this.chatThread is not null)
{
foreach (var block in this.chatThread.Blocks.OrderBy(n => n.Time))
{
@if (!block.HideFromUser)
{
<ContentBlockComponent Role="@block.Role" Type="@block.ContentType" Time="@block.Time" Content="@block.Content"/>
}
}
}

<div id="@AFTER_RESULT_DIV_ID" class="mt-3">
</div>
</ChildContent>
Expand Down
52 changes: 48 additions & 4 deletions app/MindWork AI Studio/Assistants/AssistantBase.razor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@

using Microsoft.AspNetCore.Components;

using MudBlazor.Utilities;

using RustService = AIStudio.Tools.RustService;
using Timer = System.Timers.Timer;

namespace AIStudio.Assistants;

Expand Down Expand Up @@ -55,7 +58,7 @@ public abstract partial class AssistantBase : ComponentBase, IMessageBusReceiver
_ => string.Empty,
};

protected abstract void ResetFrom();
protected abstract void ResetForm();

protected abstract bool MightPreselectValues();

Expand All @@ -68,6 +71,8 @@ public abstract partial class AssistantBase : ComponentBase, IMessageBusReceiver
private protected virtual RenderFragment? Body => null;

protected virtual bool ShowResult => true;

protected virtual bool ShowEntireChatThread => false;

protected virtual bool AllowProfiles => true;

Expand All @@ -91,8 +96,10 @@ public abstract partial class AssistantBase : ComponentBase, IMessageBusReceiver
protected MudForm? form;
protected bool inputIsValid;
protected Profile currentProfile = Profile.NO_PROFILE;

protected ChatThread? chatThread;

private readonly Timer formChangeTimer = new(TimeSpan.FromSeconds(1.6));

private ContentBlock? resultingContentBlock;
private string[] inputIssues = [];
private bool isProcessing;
Expand All @@ -101,6 +108,13 @@ public abstract partial class AssistantBase : ComponentBase, IMessageBusReceiver

protected override async Task OnInitializedAsync()
{
this.formChangeTimer.AutoReset = false;
this.formChangeTimer.Elapsed += async (_, _) =>
{
this.formChangeTimer.Stop();
await this.OnFormChange();
};

this.MightPreselectValues();
this.providerSettings = this.SettingsManager.GetPreselectedProvider(this.Component);
this.currentProfile = this.SettingsManager.GetPreselectedProfile(this.Component);
Expand Down Expand Up @@ -161,7 +175,35 @@ public Task ProcessMessage<T>(ComponentBase? sendingComponent, Event triggeredEv

return null;
}

private void TriggerFormChange(FormFieldChangedEventArgs _)
{
this.formChangeTimer.Stop();
this.formChangeTimer.Start();
}

/// <summary>
/// This method is called after any form field has changed.
/// </summary>
/// <remarks>
/// This method is called after a delay of 1.6 seconds. This is to prevent
/// the method from being called too often. This method is called after
/// the user has stopped typing or selecting options.
/// </remarks>
protected virtual Task OnFormChange() => Task.CompletedTask;

/// <summary>
/// Add an issue to the UI.
/// </summary>
/// <param name="issue">The issue to add.</param>
protected void AddInputIssue(string issue)
{
Array.Resize(ref this.inputIssues, this.inputIssues.Length + 1);
this.inputIssues[^1] = issue;
this.inputIsValid = false;
this.StateHasChanged();
}

protected void CreateChatThread()
{
this.chatThread = new()
Expand Down Expand Up @@ -221,7 +263,7 @@ protected DateTimeOffset AddUserRequest(string request, bool hideContentFromUser
return time;
}

protected async Task<string> AddAIResponseAsync(DateTimeOffset time)
protected async Task<string> AddAIResponseAsync(DateTimeOffset time, bool hideContentFromUser = false)
{
var aiText = new ContentText
{
Expand All @@ -236,6 +278,7 @@ protected async Task<string> AddAIResponseAsync(DateTimeOffset time)
ContentType = ContentType.TEXT,
Role = ChatRole.AI,
Content = aiText,
HideFromUser = hideContentFromUser,
};

if (this.chatThread is not null)
Expand Down Expand Up @@ -313,7 +356,7 @@ private async Task InnerResetForm()
await this.JsRuntime.ClearDiv(RESULT_DIV_ID);
await this.JsRuntime.ClearDiv(AFTER_RESULT_DIV_ID);

this.ResetFrom();
this.ResetForm();
this.providerSettings = this.SettingsManager.GetPreselectedProvider(this.Component);

this.inputIsValid = false;
Expand Down Expand Up @@ -341,6 +384,7 @@ private async Task InnerResetForm()
public void Dispose()
{
this.MessageBus.Unregister(this);
this.formChangeTimer.Dispose();
}

#endregion
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
using System.Text;

using AIStudio.Components;
using AIStudio.Chat;
using AIStudio.Settings.DataModel;

namespace AIStudio.Assistants.BiasDay;
Expand Down Expand Up @@ -50,7 +50,7 @@ this bias.

protected override bool ShowReset => false;

protected override void ResetFrom()
protected override void ResetForm()
{
if (!this.MightPreselectValues())
{
Expand Down Expand Up @@ -124,7 +124,7 @@ private async Task TellBias()
{
var biasChat = new LoadChat
{
WorkspaceId = Workspaces.WORKSPACE_ID_BIAS,
WorkspaceId = KnownWorkspaces.BIAS_WORKSPACE_ID,
ChatId = this.SettingsManager.ConfigurationData.BiasOfTheDay.BiasOfTheDayChatId,
};

Expand All @@ -147,7 +147,7 @@ private async Task TellBias()
BiasCatalog.ALL_BIAS[this.SettingsManager.ConfigurationData.BiasOfTheDay.BiasOfTheDayId] :
BiasCatalog.GetRandomBias(this.SettingsManager.ConfigurationData.BiasOfTheDay.UsedBias);

var chatId = this.CreateChatThread(Workspaces.WORKSPACE_ID_BIAS, this.biasOfTheDay.Name);
var chatId = this.CreateChatThread(KnownWorkspaces.BIAS_WORKSPACE_ID, this.biasOfTheDay.Name);
this.SettingsManager.ConfigurationData.BiasOfTheDay.BiasOfTheDayId = this.biasOfTheDay.Id;
this.SettingsManager.ConfigurationData.BiasOfTheDay.BiasOfTheDayChatId = chatId;
this.SettingsManager.ConfigurationData.BiasOfTheDay.DateLastBiasDrawn = DateOnly.FromDateTime(DateTime.Now);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ messages. You can also help with code refactoring and optimization.

protected override Func<Task> SubmitAction => this.GetSupport;

protected override void ResetFrom()
protected override void ResetForm()
{
this.codingContexts.Clear();
this.compilerMessages = string.Empty;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<MudTextField T="string" @bind-Text="@this.CodingContext.Id" AdornmentIcon="@Icons.Material.Filled.Numbers" Adornment="Adornment.Start" Label="(Optional) Identifier" Variant="Variant.Outlined" Margin="Margin.Dense" UserAttributes="@USER_INPUT_ATTRIBUTES" Class="mb-3"/>
<MudStack Row="@true" AlignItems="AlignItems.Center" Class="mb-3">
<MudStack Row="@true" Class="mb-3">
<MudSelect T="CommonCodingLanguages" @bind-Value="@this.CodingContext.Language" AdornmentIcon="@Icons.Material.Filled.Code" Adornment="Adornment.Start" Label="Language" Variant="Variant.Outlined" Margin="Margin.Dense">
@foreach (var language in Enum.GetValues<CommonCodingLanguages>())
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public partial class AssistantEMail : AssistantBaseCore
SystemPrompt = SystemPrompts.DEFAULT,
};

protected override void ResetFrom()
protected override void ResetForm()
{
this.inputBulletPoints = string.Empty;
this.bulletPointsLines.Clear();
Expand Down
9 changes: 9 additions & 0 deletions app/MindWork AI Studio/Assistants/ERI/AllowedLLMProviders.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
namespace AIStudio.Assistants.ERI;

public enum AllowedLLMProviders
{
NONE,

ANY,
SELF_HOSTED,
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
namespace AIStudio.Assistants.ERI;

public static class AllowedLLMProvidersExtensions
{
public static string Description(this AllowedLLMProviders provider)
{
return provider switch
{
AllowedLLMProviders.NONE => "Please select what kind of LLM provider are allowed for this data source",
AllowedLLMProviders.ANY => "Any LLM provider is allowed: users might choose a cloud-based or a self-hosted provider",
AllowedLLMProviders.SELF_HOSTED => "Self-hosted LLM providers are allowed: users cannot choose any cloud-based provider",

_ => "Unknown option was selected"
};
}
}
Loading