Chat: waitlist for OpenAI-o1 & OpenAI-o1 mini #5508

abeatrix · 2024-09-09T15:34:53Z

CLOSE https://linear.app/sourcegraph/issue/CODY-3681/improve-support-for-previewexperimental-models

Related to https://github.com/sourcegraph/sourcegraph/pull/323

Added OpenAI o1 and OpenAI o1 mini to dropdown list where users can join the waitlist
Clicking on Join Waitlist will open the blog post, fire a telemetry event called cody.joinLlmWaitlist and update the Join Waitlist label to On Waitlist
Added a new ModelTag.StreamDisabled tag to indicate models that do not support streaming
Updated the AssistantMessageCell and HumanMessageEditor components to handle models without streaming support
- Displayed a message to the user when a non-streaming model is used
- Filtered out the initial codebase context when using a non-streaming model to avoid longer processing times
- Display Try again with a different model instead of Try again with different context below each assistant response
Updated the ModelSelectField component to display an "Early Access" badge for models with the ModelTag.Preview tag
Expanded the ModelRef interface and ModelsService to better handle model ID matching
Adds the ability to duplicate the current chat session in the Cody chat interface.
- When the "Try again with a different model" button is clicked, a new chat session is created that is a copy of the current session, allowing the user to resubmit their request with a different model.
- The changes include:
  - Adding a new chatSession command to the webview message protocol, with duplicate and new actions
  - Implementing the duplicateSession method in the ChatController to create a new session with a unique ID based on the current session
  - Updating the ContextFocusActions component to use the new chatSession command when the "Try again with a different model" button is clicked
Models with the ModelTag.StreamDisabled tag will no longer display the initial codebase context to avoid longer processing times.
Updated the current OpenAI o1 and OpenAI o1 mini models to use the stable version instead of -latest versions
Fix the model provider logo not matching issue

Test plan

Build from this branch and verify the following in your debug modes:

OpenAI o1 & OpenAI o1 mini are added to the model dropdown list
These models have the Join Waitlist label
Clicking on one of the model will open a webpage in browser
The Join Waitlist label now turned to On Waitlist

When using these models:

When submitting a question using one of these models, you will see Model without streaming support takes longer to response. when waiting for the LLM response
You should not find Try again with different context underneath the LLM response when using these models.
You should see Try again with a different model instead
Verify that when the "Try again with a different model" button is clicked, a new chat session is created that is a copy of the current session, and the user is able to resubmit their request with a different model.

Changelog

feat(chat): add support for preview models Gemini 1.5 Pro Latest & Gemini 1.5 Flash Latest
feat(chat): Added ability to duplicate chat sessions

- Added a new `ModelTag.StreamDisabled` tag to indicate models that do not support streaming - Updated the `AssistantMessageCell` and `HumanMessageEditor` components to handle models without streaming support - Displayed a message to the user when a non-streaming model is used - Filtered out the initial codebase context when using a non-streaming model to avoid longer processing times - Updated the `ModelSelectField` component to display an "Early Access" badge for models with the `ModelTag.Preview` tag - Expanded the `ModelRef` interface and `ModelsService` to better handle model ID matching Models with the `ModelTag.StreamDisabled` tag will no longer display the initial codebase context to avoid longer processing times.

abeatrix · 2024-09-12T07:18:43Z

vscode/src/completions/nodeClient.ts

@@ -200,6 +200,17 @@ export class SourcegraphNodeCompletionsClient extends SourcegraphCompletionsClie
                        bufferText += str
                        bufferBin = buf

+                        // HACK: Handles non-stream request.
+                        // TODO: Implement a function to make and process non-stream requests.
+                        if (params.stream === false) {


Don't have time to implement a proper function to process non-streaming requests so will require a follow-up.

abeatrix · 2024-09-12T07:20:52Z

vscode/webviews/components/modelSelectField/ModelSelectField.tsx

@@ -83,11 +84,12 @@ export const ModelSelectField: React.FunctionComponent<{
                })
                return
            }
-            getVSCodeAPI().postMessage({
-                command: 'event',


slimsag · 2024-09-12T18:54:32Z

vscode/CHANGELOG.md

@@ -6,6 +6,8 @@ This is a log of all notable changes to Cody for VS Code. [Unreleased] changes a

 ### Added

+- Chat: The new `Gemini 1.5 Pro Latest` and `Gemini 1.5 Flash Latest` models are now available for Cody Pro users. [pull/5508](https://github.com/sourcegraph/cody/pull/5508)


is this correct

Good catch! 🦅

slimsag

Really impressive work @abeatrix !

Follow up on #5508 CLOSE https://linear.app/sourcegraph/issue/CODY-3741/oai-o1-hits-parsing-error This change adds support for non-streaming requests, particularly for new OpenAI models that do not support streaming. It also fixes a "Parse Error: JS exception" issue that occurred when handling non-streaming responses in the streaming client. - Remove HACK for handling non-streaming requests in the stream method in nodeClient.ts that's resulting in parsing error - Implement a new `_fetchWithCallbacks` method in the `SourcegraphCompletionsClient` class to handle non-streaming requests - Update the `SourcegraphNodeCompletionsClient` and `SourcegraphBrowserCompletionsClient` classes to use the new `_fetchWithCallbacks` method when `params.stream` is `false` - This change ensures that non-streaming requests, such as those for the new OpenAI models that do not support streaming, are handled correctly and avoid issues like "Parse Error: JS exception" that can occur when the response is not parsed correctly in the streaming client The `SourcegraphCompletionsClient` class now has a new abstract method `_fetchWithCallbacks` that must be implemented by subclasses ## Test plan  Verify that chat requests using non-streaming models (e.g., OpenAI O1, OpenAI O1-mini) work correctly without any parsing errors. Example: ask Cody about cody repo with "cody what are the latest updates in this repo?" ### After First try ![image](https://github.com/user-attachments/assets/24fbd1b0-185d-4b07-90be-385fdd5da975) Second try ![image](https://github.com/user-attachments/assets/becdc7f6-18bb-4fec-8fcc-8263072680f8) ### Before First try ![image](https://github.com/user-attachments/assets/8975cb5a-b605-4e0a-ac2c-dab053124920) ## Changelog  feat(chat): support non-streaming requests --------- Co-authored-by: Valery Bugakov <skymk1@gmail.com>

abeatrix added 4 commits September 6, 2024 17:44

Update agent bindings and changelog

310315f

Merge branch 'main' into bee/preview-models

4992d7a

add ability to duplicate chat sessions

7cc6544

abeatrix changed the title ~~wip - feat(chat): add support for preview models~~ feat(chat): add support for preview models Sep 9, 2024

abeatrix requested review from beyang, kukicado and aramaraju September 9, 2024 20:00

abeatrix added 2 commits September 11, 2024 22:51

add waitlist support

d1dcc2b

wip

b17213b

abeatrix force-pushed the bee/preview-models branch from 43bd8e8 to b17213b Compare September 12, 2024 06:25

abeatrix added 3 commits September 11, 2024 23:26

Merge branch 'main' into bee/preview-models

134d75e

clean up merge

2785ced

Update agent bindings

5b1ba8f

abeatrix commented Sep 12, 2024

View reviewed changes

abeatrix changed the title ~~feat(chat): add support for preview models~~ wip - feat(chat): add support for preview models Sep 12, 2024

abeatrix added 4 commits September 12, 2024 01:05

fix params and test

80586d0

Update preview model names

a2c0c0e

Update o1 blog link

d7bfd41

Merge branch 'main' into bee/preview-models

12d7bc7

abeatrix changed the title ~~wip - feat(chat): add support for preview models~~ Chat: waitlist for OpenAI-o1 & OpenAI-o1 mini Sep 12, 2024

abeatrix marked this pull request as ready for review September 12, 2024 18:39

abeatrix requested review from a team and slimsag September 12, 2024 18:39

slimsag reviewed Sep 12, 2024

View reviewed changes

slimsag approved these changes Sep 12, 2024

View reviewed changes

Update changelog

7df7e5b

abeatrix merged commit 162162d into main Sep 12, 2024
19 checks passed

abeatrix deleted the bee/preview-models branch September 12, 2024 21:17

abeatrix mentioned this pull request Sep 13, 2024

fix(chat): support non-streaming chat completion requests #5565

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Chat: waitlist for OpenAI-o1 & OpenAI-o1 mini #5508

Chat: waitlist for OpenAI-o1 & OpenAI-o1 mini #5508

abeatrix commented Sep 9, 2024 •

edited

Loading

abeatrix Sep 12, 2024

abeatrix Sep 12, 2024

slimsag Sep 12, 2024

abeatrix Sep 12, 2024

slimsag left a comment

		@@ -6,6 +6,8 @@ This is a log of all notable changes to Cody for VS Code. [Unreleased] changes a

		### Added

		- Chat: The new `Gemini 1.5 Pro Latest` and `Gemini 1.5 Flash Latest` models are now available for Cody Pro users. [pull/5508](https://github.com/sourcegraph/cody/pull/5508)

Chat: waitlist for OpenAI-o1 & OpenAI-o1 mini #5508

Chat: waitlist for OpenAI-o1 & OpenAI-o1 mini #5508

Conversation

abeatrix commented Sep 9, 2024 • edited Loading

Test plan

Changelog

abeatrix Sep 12, 2024

Choose a reason for hiding this comment

abeatrix Sep 12, 2024

Choose a reason for hiding this comment

slimsag Sep 12, 2024

Choose a reason for hiding this comment

abeatrix Sep 12, 2024

Choose a reason for hiding this comment

slimsag left a comment

Choose a reason for hiding this comment

abeatrix commented Sep 9, 2024 •

edited

Loading