Skip to content

Commit

Permalink
feat(GPT/Anthropic): Continue Regenerating & Generation Buttons (dann…
Browse files Browse the repository at this point in the history 
…y-avila#808)

* feat(useMessageHandler.js/ts): Refactor and add features to handle user messages, support multiple endpoints/models, generate placeholder responses, regeneration, and stopGeneration function

fix(conversation.ts, buildTree.ts): Import TMessage type, handle null parentMessageId

feat(schemas.ts): Update and add schemas for various AI services, add default values, optional fields, and endpoint-to-schema mapping, create parseConvo function

chore(useMessageHandler.js, schemas.ts): Remove unused imports, variables, and chatGPT enum

* wip: add generation buttons

* refactor(cleanupPreset.ts): simplify cleanupPreset function
refactor(getDefaultConversation.js): remove unused code and simplify getDefaultConversation function

feat(utils): add getDefaultConversation function

This commit adds a new utility function called `getDefaultConversation` to the `client/src/utils/getDefaultConversation.ts` file. This function is responsible for generating a default conversation object based on the provided parameters.

The `getDefaultConversation` function takes in an object with the following properties:
- `conversation`: The conversation object to be used as a base.
- `endpointsConfig`: The configuration object containing information about the available endpoints.
- `preset`: An optional preset object that can be used to override the default behavior.

The function first tries to determine the target endpoint based on the preset object. If a valid endpoint is found, it is used as the target endpoint. If not, the function tries to retrieve the last conversation setup from the local storage and uses its endpoint if it is valid. If neither the preset nor the local storage contains a valid endpoint, the function falls back to a default endpoint.

Once the target endpoint is determined,

* fix(utils): remove console.error statement in buildDefaultConversation function
fix(schemas): add default values for catch blocks in openAISchema, googleSchema, bingAISchema, anthropicSchema, chatGPTBrowserSchema, and gptPluginsSchema

* fix: endpoint not changing on change of preset from other endpoint, wip: refactor

* refactor: preset items to TSX

* refactor: convert resetConvo to TS

* refactor(getDefaultConversation.ts): move defaultEndpoints array to the top of the file for better readability
refactor(getDefaultConversation.ts): extract getDefaultEndpoint function for better code organization and reusability

* feat(svg): add ContinueIcon component
feat(svg): add RegenerateIcon component
feat(svg): add ContinueIcon and RegenerateIcon components to index.ts

* feat(Button.tsx): add onClick and className props to Button component
feat(GenerationButtons.tsx): add logic to display Regenerate or StopGenerating button based on isSubmitting and messages
feat(Regenerate.tsx): create Regenerate component with RegenerateIcon and handleRegenerate function
feat(StopGenerating.tsx): create StopGenerating component with StopGeneratingIcon and handleStopGenerating function

* fix(TextChat.jsx): reorder imports and variables for better readability
fix(TextChat.jsx): fix typo in condition for isNotAppendable variable
fix(TextChat.jsx): remove unused handleStopGenerating function
fix(ContinueIcon.tsx): remove unnecessary closing tags for polygon elements
fix(useMessageHandler.ts): add missing type annotations for handleStopGenerating and handleRegenerate functions
fix(useMessageHandler.ts): remove unused variables in return statement

* fix(getDefaultConversation.ts): refactor code to use getLocalStorageItems function
feat(getLocalStorageItems.ts): add utility function to retrieve items from local storage

* fix(OpenAIClient.js): add support for streaming result in sendCompletion method
feat(OpenAIClient.js): add finish_reason metadata to opts in sendCompletion method
feat(Message.js): add finish_reason field to Message model
feat(messageSchema.js): add finish_reason field to messageSchema
feat(openAI.js): parse chatGptLabel and promptPrefix from req.body and pass rest of the modelOptions to endpointOption
feat(openAI.js): add addMetadata function to store metadata in ask function
feat(openAI.js): add metadata to response if available
feat(schemas.ts): add finish_reason field to tMessageSchema

* feat(types.ts): add TOnClick and TGenButtonProps types for button components
feat(Continue.tsx): create Continue component for generating button
feat(GenerationButtons.tsx): update GenerationButtons component to use Continue component
feat(Regenerate.tsx): create Regenerate component for regenerating button
feat(Stop.tsx): create Stop component for stop generating button

* feat(MessageHandler.jsx): add MessageHandler component to handle messages and conversations
fix(Root.jsx): fix import paths for Nav and MessageHandler components

* feat(useMessageHandler.ts): add support for generation parameter in ask function
feat(useMessageHandler.ts): add support for isEdited parameter in ask function
feat(useMessageHandler.ts): add support for continueGeneration function
fix(createPayload.ts): replace endpoint URL when isEdited parameter is true

* chore(client): set skipLibCheck to true in tsconfig.json

* fix(useMessageHandler.ts): remove unused clientId variable
fix(schemas.ts): make clientId field in tMessageSchema nullable and optional

* wip: edit route for continue generation

* refactor(api): move handlers to root of routes dir

* fix(useMessageHandler.ts): initialize currentMessages to an empty array if messages is null
fix(useMessageHandler.ts): update initialResponse text to use responseText variable
fix(useMessageHandler.ts): update setMessages logic for isRegenerate case
fix(MessageHandler.jsx): update setMessages logic for cancelHandler, createdHandler, and finalHandler

* fix(schemas.ts): make createdAt and updatedAt fields optional and set default values using new Date().toISOString()
fix(schemas.ts): change type annotation of TMessage from infer to input

* refactor(useMessageHandler.ts): rename AskProps type to TAskProps
refactor(useMessageHandler.ts): remove generation property from ask function arguments
refactor(useMessageHandler.ts): use nullish coalescing operator (??) instead of logical OR (||)
refactor(useMessageHandler.ts): pass the responseMessageId to message prop of submission

* fix(BaseClient.js): use nullish coalescing operator (??) instead of logical OR (||) for default values

* fix(BaseClient.js): fix responseMessageId assignment in handleStartMethods method
feat(BaseClient.js): add support for isEdited flag in sendMessage method
feat(BaseClient.js): add generation to responseMessage text in sendMessage method

* fix(openAI.js): remove unused imports and commented out code
feat(openAI.js): add support for generation parameter in request body
fix(openAI.js): remove console.log statement
fix(openAI.js): remove unused variables and parameters
fix(openAI.js): update response text in case of error
fix(openAI.js): handle error and abort message in case of error
fix(handlers.js): add generation parameter to createOnProgress function
fix(useMessageHandler.ts): update responseText variable to use generation parameter

* refactor(api/middleware): move inside server dir

* refactor: add endpoint specific, modular functions to build options and initialize clients, create server/utils, move middleware, separate utils into api general utils and server specific utils

* fix(abortMiddleware.js): import getConvo and getConvoTitle functions from models
feat(abortMiddleware.js): add abortAsk function to abortController to handle aborting of requests
fix(openAI.js): import buildOptions and initializeClient functions from endpoints/openAI
refactor(openAI.js): use getAbortData function to get data for abortAsk function

* refactor: move endpoint specific logic to an endpoints dir

* refactor(PluginService.js): fix import path for encrypt and decrypt functions in PluginService.js

* feat(openAI): add new endpoint for adding a title to a conversation

- Added a new file `addTitle.js` in the `api/server/routes/endpoints/openAI` directory.
- The `addTitle.js` file exports a function `addTitle` that takes in request parameters and performs the following actions:
  - If the `parentMessageId` is `'00000000-0000-0000-0000-000000000000'` and `newConvo` is true, it proceeds with the following steps:
    - Calls the `titleConvo` function from the `titleConvo` module, passing in the necessary parameters.
    - Calls the `saveConvo` function from the `saveConvo` module, passing in the user ID and conversation details.
- Updated the `index.js` file in the `api/server/routes/endpoints/openAI` directory to export the `addTitle` function.
- This change adds

* fix(abortMiddleware.js): remove console.log statement
refactor(gptPlugins.js): update imports and function parameters
feat(gptPlugins.js): add support for abortController and getAbortData
refactor(openAI.js): update imports and function parameters
feat(openAI.js): add support for abortController and getAbortData

fix(openAI.js): refactor code to use modularized functions and middleware
fix(buildOptions.js): refactor code to use destructuring and update variable names

* refactor(askChatGPTBrowser.js, bingAI.js, google.js): remove duplicate code for setting response headers
feat(askChatGPTBrowser.js, bingAI.js, google.js): add setHeaders middleware to set response headers

* feat(middleware): validateEndpoint, refactor buildOption to only be concerned of endpointOption

* fix(abortMiddleware.js): add 'finish_reason' property with value 'incomplete' to responseMessage object
fix(abortMessage.js): remove console.log statement for aborted message
fix(handlers.js): modify tokens assignment to handle empty generation string and trailing space

* fix(BaseClient.js): import addSpaceIfNeeded function from server/utils
fix(BaseClient.js): add space before generation in text property
fix(index.js): remove getCitations and citeText exports
feat(buildEndpointOption.js): add buildEndpointOption middleware
fix(index.js): import buildEndpointOption middleware
fix(anthropic.js): remove buildOptions function and use endpointOption from req.body
fix(gptPlugins.js): remove buildOptions function and use endpointOption from req.body
fix(openAI.js): remove buildOptions function and use endpointOption from req.body

feat(utils): add citations.js and handleText.js modules
fix(utils): fix import statements in index.js module

* refactor(gptPlugins.js): use getResponseSender function from librechat-data-provider

* feat(gptPlugins): complete 'continue generating'

* wip: anthropic continue regen

* feat(middleware): add validateRegistration middleware

A new middleware function called `validateRegistration` has been added to the list of exported middleware functions in `index.js`. This middleware is responsible for validating registration data before allowing the registration process to proceed.

* feat(Anthropic): complete continue regen

* chore: add librechat-data-provider to api/package.json

* fix(ci): backend-review will mock meilisearch, also installs data-provider as now needed

* chore(ci): remove unneeded SEARCH env var

* style(GenerationButtons): make text shorter for sake of space economy, even though this diverges from chat.openai.com

* style(GenerationButtons/ScrollToBottom): adjust visibility/position based on screen size

* chore(client): 'Editting' typo

* feat(GenerationButtons.tsx): add support for endpoint prop in GenerationButtons component
feat(OptionsBar.tsx): pass endpoint prop to GenerationButtons component
feat(useGenerations.ts): create useGenerations hook to handle generation logic
fix(schemas.ts): add searchResult field to tMessageSchema

* refactor(HoverButtons): convert to TSX and utilize new useGenerations hook

* fix(abortMiddleware): handle error with res headers set, or abortController not found, to ensure proper API error is sent to the client, chore(BaseClient): remove console log for onStart message meant for debugging

* refactor(api): remove librechat-data-provider dep for now as it complicates deployed docker build stage, re-use code in CJS, located in server/endpoints/schemas

* chore: remove console.logs from test files

* ci: add backend tests for AnthropicClient, focusing on new buildMessages logic

* refactor(FakeClient): use actual BaseClient sendMessage method for testing

* test(BaseClient.test.js): add test for loading chat history
test(BaseClient.test.js): add test for sendMessage logic with isEdited flag

* fix(buildEndpointOption.js): add support for azureOpenAI in buildFunction object
wip(endpoints.js): fetch Azure models from Azure OpenAI API if opts.azure is true

* fix(Button.tsx): add data-testid attribute to button component
fix(SelectDropDown.tsx): add data-testid attribute to Listbox.Button component
fix(messages.spec.ts): add waitForServerStream function to consolidate logic for awaiting the server response
feat(messages.spec.ts): add test for stopping and continuing message and improve browser/page context order and closing

* refactor(onProgress): speed up time to save initial message for editable routes

* chore: disable AI message editing (for now), was accidentally allowed

* refactor: ensure continue is only supported for latest message style: improve styling in dark mode and across all hover buttons/icons, including making edit icon for AI invisible (for now)

* fix: add test id to generation buttons so they never resolve to 2+ items

* chore(package.json): add 'packages/' to the list of ignored directories
chore(data-provider/package.json): bump version to 0.1.5
  • Loading branch information
@danny-avila
danny-avila authored Aug 17, 2023
1 parent ae5b7d3 commit afd43af
Show file tree
Hide file tree
Showing 113 changed files with 3,029 additions and 1,549 deletions.
10 changes: 3 additions & 7 deletions .github/workflows/backend-review.yml
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,5 @@
name: Backend Unit Tests
on:
# push:
# branches:
# - main
# - dev
# - release/*
pull_request:
branches:
- main
Expand All @@ -23,6 +18,7 @@ jobs:
JWT_SECRET: ${{ secrets.JWT_SECRET }}
CREDS_KEY: ${{ secrets.CREDS_KEY }}
CREDS_IV: ${{ secrets.CREDS_IV }}
NODE_ENV: ci
steps:
- uses: actions/checkout@v2
- name: Use Node.js 20.x
Expand All @@ -34,8 +30,8 @@ jobs:
- name: Install dependencies
run: npm ci

# - name: Install Linux X64 Sharp
# run: npm install --platform=linux --arch=x64 --verbose sharp
- name: Install Data Provider
run: npm run build:data-provider

- name: Run unit tests
run: cd api && npm run test:ci
Expand Down
46 changes: 37 additions & 9 deletions api/app/clients/AnthropicClient.js
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
const Keyv = require('keyv');
// const { Agent, ProxyAgent } = require('undici');
const BaseClient = require('./BaseClient');
const {
Expand All @@ -15,8 +14,6 @@ const tokenizersCache = {};
class AnthropicClient extends BaseClient {
constructor(apiKey, options = {}, cacheOptions = {}) {
super(apiKey, options, cacheOptions);
cacheOptions.namespace = cacheOptions.namespace || 'anthropic';
this.conversationsCache = new Keyv(cacheOptions);
this.apiKey = apiKey || process.env.ANTHROPIC_API_KEY;
this.sender = 'Anthropic';
this.userLabel = HUMAN_PROMPT;
Expand Down Expand Up @@ -107,6 +104,23 @@ class AnthropicClient extends BaseClient {
content: message?.content ?? message.text,
}));

let lastAuthor = '';
let groupedMessages = [];

for (let message of formattedMessages) {
// If last author is not same as current author, add to new group
if (lastAuthor !== message.author) {
groupedMessages.push({
author: message.author,
content: [message.content],
});
lastAuthor = message.author;
// If same author, append content to the last group
} else {
groupedMessages[groupedMessages.length - 1].content.push(message.content);
}
}

let identityPrefix = '';
if (this.options.userLabel) {
identityPrefix = `\nHuman's name: ${this.options.userLabel}`;
Expand All @@ -129,8 +143,12 @@ class AnthropicClient extends BaseClient {
promptPrefix = `${identityPrefix}${promptPrefix}`;
}

const promptSuffix = `${promptPrefix}${this.assistantLabel}\n`; // Prompt AI to respond.
let currentTokenCount = this.getTokenCount(promptSuffix);
// Prompt AI to respond, empty if last message was from AI
let isEdited = lastAuthor === this.assistantLabel;
const promptSuffix = isEdited ? '' : `${promptPrefix}${this.assistantLabel}\n`;
let currentTokenCount = isEdited
? this.getTokenCount(promptPrefix)
: this.getTokenCount(promptSuffix);

let promptBody = '';
const maxTokenCount = this.maxPromptTokens;
Expand All @@ -148,10 +166,13 @@ class AnthropicClient extends BaseClient {
};

const buildPromptBody = async () => {
if (currentTokenCount < maxTokenCount && formattedMessages.length > 0) {
const message = formattedMessages.pop();
if (currentTokenCount < maxTokenCount && groupedMessages.length > 0) {
const message = groupedMessages.pop();
const isCreatedByUser = message.author === this.userLabel;
const messageString = `${message.author}\n${message.content}${this.endToken}\n`;
// Use promptPrefix if message is edited assistant'
const messagePrefix =
isCreatedByUser || !isEdited ? message.author : `${promptPrefix}${message.author}`;
const messageString = `${messagePrefix}\n${message.content}${this.endToken}\n`;
let newPromptBody = `${messageString}${promptBody}`;

context.unshift(message);
Expand Down Expand Up @@ -182,6 +203,12 @@ class AnthropicClient extends BaseClient {
}
promptBody = newPromptBody;
currentTokenCount = newTokenCount;

// Switch off isEdited after using it for the first time
if (isEdited) {
isEdited = false;
}

// wait for next tick to avoid blocking the event loop
await new Promise((resolve) => setImmediate(resolve));
return buildPromptBody();
Expand All @@ -197,7 +224,8 @@ class AnthropicClient extends BaseClient {
context.shift();
}

const prompt = `${promptBody}${promptSuffix}`;
let prompt = `${promptBody}${promptSuffix}`;

// Add 2 tokens for metadata after all messages have been counted.
currentTokenCount += 2;

Expand Down
71 changes: 45 additions & 26 deletions api/app/clients/BaseClient.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@ const { ChatOpenAI } = require('langchain/chat_models/openai');
const { loadSummarizationChain } = require('langchain/chains');
const { refinePrompt } = require('./prompts/refinePrompt');
const { getConvo, getMessages, saveMessage, updateMessage, saveConvo } = require('../../models');
const { addSpaceIfNeeded } = require('../../server/utils');

class BaseClient {
constructor(apiKey, options = {}) {
this.apiKey = apiKey;
this.sender = options.sender || 'AI';
this.sender = options.sender ?? 'AI';
this.contextStrategy = null;
this.currentDateString = new Date().toLocaleDateString('en-us', {
year: 'numeric',
Expand Down Expand Up @@ -51,18 +52,20 @@ class BaseClient {
if (opts && typeof opts === 'object') {
this.setOptions(opts);
}
const user = opts.user || null;
const conversationId = opts.conversationId || crypto.randomUUID();
const parentMessageId = opts.parentMessageId || '00000000-0000-0000-0000-000000000000';
const userMessageId = opts.overrideParentMessageId || crypto.randomUUID();
const responseMessageId = crypto.randomUUID();
const user = opts.user ?? null;
const conversationId = opts.conversationId ?? crypto.randomUUID();
const parentMessageId = opts.parentMessageId ?? '00000000-0000-0000-0000-000000000000';
const userMessageId = opts.overrideParentMessageId ?? crypto.randomUUID();
const responseMessageId = opts.responseMessageId ?? crypto.randomUUID();
const saveOptions = this.getSaveOptions();
this.abortController = opts.abortController || new AbortController();
this.currentMessages = (await this.loadHistory(conversationId, parentMessageId)) ?? [];
const head = opts.isEdited ? responseMessageId : parentMessageId;
this.currentMessages = (await this.loadHistory(conversationId, head)) ?? [];
this.abortController = opts.abortController ?? new AbortController();

return {
...opts,
user,
head,
conversationId,
parentMessageId,
userMessageId,
Expand All @@ -72,27 +75,35 @@ class BaseClient {
}

createUserMessage({ messageId, parentMessageId, conversationId, text }) {
const userMessage = {
return {
messageId,
parentMessageId,
conversationId,
sender: 'User',
text,
isCreatedByUser: true,
};
return userMessage;
}

async handleStartMethods(message, opts) {
const { user, conversationId, parentMessageId, userMessageId, responseMessageId, saveOptions } =
await this.setMessageOptions(opts);

const userMessage = this.createUserMessage({
messageId: userMessageId,
parentMessageId,
const {
user,
head,
conversationId,
text: message,
});
parentMessageId,
userMessageId,
responseMessageId,
saveOptions,
} = await this.setMessageOptions(opts);

const userMessage = opts.isEdited
? this.currentMessages[this.currentMessages.length - 2]
: this.createUserMessage({
messageId: userMessageId,
parentMessageId,
conversationId,
text: message,
});

if (typeof opts?.getIds === 'function') {
opts.getIds({
Expand All @@ -109,6 +120,7 @@ class BaseClient {
return {
...opts,
user,
head,
conversationId,
responseMessageId,
saveOptions,
Expand Down Expand Up @@ -373,7 +385,7 @@ class BaseClient {

if (this.options.debug) {
console.debug('<-------------------------PAYLOAD/TOKEN COUNT MAP------------------------->');
console.debug('Payload:', payload);
// console.debug('Payload:', payload);
console.debug('Token Count Map:', tokenCountMap);
console.debug('Prompt Tokens', promptTokens, remainingContextTokens, this.maxContextTokens);
}
Expand All @@ -382,13 +394,16 @@ class BaseClient {
}

async sendMessage(message, opts = {}) {
const { user, conversationId, responseMessageId, saveOptions, userMessage } =
const { user, head, isEdited, conversationId, responseMessageId, saveOptions, userMessage } =
await this.handleStartMethods(message, opts);

this.user = user;
// It's not necessary to push to currentMessages
// depending on subclass implementation of handling messages
this.currentMessages.push(userMessage);
// When this is an edit, all messages are already in currentMessages, both user and response
if (!isEdited) {
this.currentMessages.push(userMessage);
}

let {
prompt: payload,
Expand All @@ -398,13 +413,13 @@ class BaseClient {
this.currentMessages,
// When the userMessage is pushed to currentMessages, the parentMessage is the userMessageId.
// this only matters when buildMessages is utilizing the parentMessageId, and may vary on implementation
userMessage.messageId,
isEdited ? head : userMessage.messageId,
this.getBuildMessagesOptions(opts),
);

if (this.options.debug) {
console.debug('payload');
console.debug(payload);
// console.debug(payload);
}

if (tokenCountMap) {
Expand All @@ -423,15 +438,19 @@ class BaseClient {
this.handleTokenCountMap(tokenCountMap);
}

await this.saveMessageToDatabase(userMessage, saveOptions, user);
if (!isEdited) {
await this.saveMessageToDatabase(userMessage, saveOptions, user);
}

const generation = isEdited ? this.currentMessages[this.currentMessages.length - 1].text : '';
const responseMessage = {
messageId: responseMessageId,
conversationId,
parentMessageId: userMessage.messageId,
isCreatedByUser: false,
model: this.modelOptions.model,
sender: this.sender,
text: await this.sendCompletion(payload, opts),
text: addSpaceIfNeeded(generation) + (await this.sendCompletion(payload, opts)),
promptTokens,
};

Expand All @@ -453,7 +472,7 @@ class BaseClient {
console.debug('Loading history for conversation', conversationId, parentMessageId);
}

const messages = (await getMessages({ conversationId })) || [];
const messages = (await getMessages({ conversationId })) ?? [];

if (messages.length === 0) {
return [];
Expand Down
9 changes: 9 additions & 0 deletions api/app/clients/OpenAIClient.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -314,13 +314,18 @@ class OpenAIClient extends BaseClient {
async sendCompletion(payload, opts = {}) {
let reply = '';
let result = null;
let streamResult = null;
if (typeof opts.onProgress === 'function') {
await this.getCompletion(
payload,
(progressMessage) => {
if (progressMessage === '[DONE]') {
return;
}

if (progressMessage.choices) {
streamResult = progressMessage;
}
const token = this.isChatCompletion
? progressMessage.choices?.[0]?.delta?.content
: progressMessage.choices?.[0]?.text;
Expand Down Expand Up @@ -355,6 +360,10 @@ class OpenAIClient extends BaseClient {
}
}

if (streamResult && typeof opts.addMetadata === 'function') {
const { finish_reason } = streamResult.choices[0];
opts.addMetadata({ finish_reason });
}
return reply.trim();
}

Expand Down
3 changes: 2 additions & 1 deletion api/app/clients/PluginsClient.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,8 @@ Only respond with your conversational reply to the following User Message:
}

async sendMessage(message, opts = {}) {
const completionMode = this.options.tools.length === 0;
// If a message is edited, no tools can be used.
const completionMode = this.options.tools.length === 0 || opts.isEdited;
if (completionMode) {
this.setOptions(opts);
return super.sendMessage(message, opts);
Expand Down
Loading

0 comments on commit afd43af

Please sign in to comment.