feat(chore): Adding tags and usage for LLM Models #1894

jamesmcnamara · 2024-07-09T01:43:03Z

This update is to align with a refactor in the vscode repo around model types to remove certain hard coded fields and replace them with ModelTags.

This is very rough around the edges because I don't know how to access the generated code in that PR (nor am I aware of the context on this repo) so any feedback is welcome!

Test plan

Green CI

jamesmcnamara · 2024-07-09T01:44:06Z

src/main/kotlin/com/sourcegraph/cody/history/state/LLMState.kt

+  @get:OptionTag(tag = "tags", nameAttribute = "") var tags: MutableList<String> by list()
+
+  @get:OptionTag(tag = "usage", nameAttribute = "") var usage: MutableList<String> by list()
+


I don't know the implications of this. Where is it getting this state from? Is this going to break immediately?

This is state saved in a IntelliJ xml settings files.
You will need to add migration which will convert old format to the new one .
For the reference you can look at the DeprecatedChatLlmMigration.

In practice you will most likely have to:

Keep ChatModelProvider::deprecated field but mark it as @Deprecated.

In the migration function (see DeprecatedChatLlmMigration::migrate for reference):

read all the old chats

for each chat read the content from deprecated field

set tags appropriately

reset deprecated to be null (as it won't be used anymore, we will remove it in future)

Alright. I created ChatTagsLllmMigration that I believe accomplishes this.

Thank you! It looks good to me now, I just added one question about the default but otherwise I think it is mergeable.

odisseus · 2024-07-09T08:51:50Z

In order for the tests to work, you'll need to update the cody.commit field in gradle.properties.

pkukielka · 2024-07-09T12:37:42Z

src/test/kotlin/com/sourcegraph/cody/config/SettingsMigrationTest.kt

@@ -190,22 +198,20 @@ class SettingsMigrationTest : BasePlatformTestCase() {
  fun `test DeprecatedChatLlmMigration`() {


That is incorrect I think.
That is migration function, so it specifically migrates data form one old format to another.
It needs to work after your changes.
Then your migration function can pick up state after all existing migrations, and migrate it further (so you will have to add new test for it)

Understood. I reverted those changes and added a new test for my migration.

pkukielka · 2024-07-10T08:51:10Z

src/main/kotlin/com/sourcegraph/cody/chat/ui/LlmDropdown.kt

-        if (selectedModel?.codyProOnly == true && isCurrentUserFree())
-            availableModels.find { it.default }
+        if (selectedModel?.isCodyProOnly() == true && isCurrentUserFree())
+            availableModels.getOrNull(0)


Out of curiosity, if we do not have default anymore isn't there any tag which would replace it?
Like recommended LLM?

I've just replaced it with the fallback convention of the head of the list. The model structs are immutable having a mutable field tracking the user's selection was a bit tricky and also easy to get wrong (multiple default's was possible). And we always fell back to just grabbing the head of the list if we couldn't find one.

I could see adding a default tag, but I would think to only use it for what the server originally sent as the default, not as the user's current selection.

pkukielka

LGTM, but with two caveats:

Isn't default replace by any tag? Don't we have any LLMs which we recommend to use?
Are those changes already in the Cody?
We bind cody version we are using by setting cody.commit there: https://github.com/sourcegraph/jetbrains/blob/main/gradle.properties#L27
Please make sure version from this commit already contains changes to the model. If not, you need to bump that commit in this PR (you can just set it to the hash of the last commit on Cody main)

jamesmcnamara · 2024-07-10T17:46:29Z

@pkukielka

default is now the head of the list always
I will update cody.commit when this PR merges.

pkukielka · 2024-07-15T07:45:37Z

@pkukielka

default is now the head of the list always

I will update cody.commit when this PR merges.

Thanks! BTW. Looks like it's merged now :)

…-enterprise-models

pkukielka · 2024-07-17T14:46:41Z

src/main/kotlin/com/sourcegraph/cody/chat/ui/LlmDropdown.kt

-        if (selectedModel?.codyProOnly == true && isCurrentUserFree())
-            availableModels.find { it.default }
+        if (selectedModel?.isCodyProOnly() == true && isCurrentUserFree())
+            availableModels.getOrNull(0)
        else selectedModel

    val isEnterpriseAccount =


@jamesmcnamara

We need to remove this line and just set isVisible = true always.

Also, I noticed that there is now enterprise tag present, shouldn't we filter out LLMs displayed for enterprise users based on if that tag is present? Or maybe for an enterprise account we are guaranteed to get a correct list?

I pushed first change to your branch already.

I'm not sure if we need the second.

I updated this because we don't want it to be visible for enterprise users that don't have multiple models enabled. However I'm still struggling with the integration tests failing and I'm not sure how to debug them. Do you know any good tricks for figuring out why they keep timing out? I tried opening the IDE to the test file and trying the document code action but it seemed to work fine.

To your second point, I think the current idea is that an enterprise user can use any model that the server provided (but most likely they will all be enterprise models).

Also, unrelated, but do you know how to access the generated kotlin code from the Cody repo? I don't see it referenced anywhere.

Problem with integration tests was that updated recordings were missing (requires run of the ./gradlew :recordingIntegrationTest). I updated them.
Interestingly it seems we were not reporting that fact properly, I will make sure we fixed that.

Also, unrelated, but do you know how to access the generated kotlin code from the Cody repo? I don't see it referenced anywhere.

Right now they are just working as a manual reference, BUT it will change very soon as there is @RXminuS PR in flight which adress that.

This change adds a new `serverSentModels` feature to the `ConfigFeatures` class, which is used to track whether the server supports sending models directly to the client. The `LlmDropdown` class is updated to subscribe to changes in the `CurrentConfigFeatures` and show/hide the dropdown based on the value of `serverSentModels`. Additionally, the `CodyAgentClientTest` and `CurrentConfigFeaturesTest` are updated to reflect the new feature.

…etbrains into jsm/cody-enterprise-models

pkukielka · 2024-07-18T08:35:53Z

I updated recordings for integration tests + reformatted the code to make spotless happy.
I also tested it manually with pro account and two enterprise accounts (demo and gs02) and now it seems to be working perfectly fine.

Thanks @jamesmcnamara and @abeatrix for fixing bug with the auth and models!

first pass at adding tags and usage for LLM Models

57bc481

jamesmcnamara requested a review from abeatrix July 9, 2024 01:43

jamesmcnamara commented Jul 9, 2024

View reviewed changes

fixed param that flipped

0742e7e

abeatrix requested review from mkondratek and a team July 9, 2024 02:11

pkukielka reviewed Jul 9, 2024

View reviewed changes

added migration

fb59ec8

jamesmcnamara marked this pull request as ready for review July 10, 2024 02:17

pkukielka reviewed Jul 10, 2024

View reviewed changes

pkukielka approved these changes Jul 10, 2024

View reviewed changes

jamesmcnamara and others added 4 commits July 15, 2024 10:40

updated cody commit

7264e72

Merge branch 'main' of github.com:sourcegraph/jetbrains into jsm/cody…

24b7a72

…-enterprise-models

Merge branch 'main' of github.com:sourcegraph/jetbrains into jsm/cody…

dd26937

…-enterprise-models

Show LLM selector for enterprise users

0c63ea5

pkukielka reviewed Jul 17, 2024

View reviewed changes

jamesmcnamara and others added 5 commits July 17, 2024 17:19

Merge branch 'jsm/cody-enterprise-models' of github.com:sourcegraph/j…

dec0c16

…etbrains into jsm/cody-enterprise-models

Added missing bool

ffe710b

Add missing recordings

986cc76

Reformat code with spotless

2a92540

pkukielka merged commit 2a28a0e into main Jul 18, 2024
7 checks passed

pkukielka deleted the jsm/cody-enterprise-models branch July 18, 2024 08:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(chore): Adding tags and usage for LLM Models #1894

feat(chore): Adding tags and usage for LLM Models #1894

jamesmcnamara commented Jul 9, 2024 •

edited

Loading

jamesmcnamara Jul 9, 2024

pkukielka Jul 9, 2024 •

edited

Loading

jamesmcnamara Jul 10, 2024

pkukielka Jul 10, 2024

odisseus commented Jul 9, 2024

pkukielka Jul 9, 2024 •

edited

Loading

jamesmcnamara Jul 10, 2024

pkukielka Jul 10, 2024

jamesmcnamara Jul 10, 2024

pkukielka left a comment •

edited

Loading

jamesmcnamara commented Jul 10, 2024

pkukielka commented Jul 15, 2024

pkukielka Jul 17, 2024 •

edited

Loading

pkukielka Jul 17, 2024

jamesmcnamara Jul 18, 2024 •

edited

Loading

pkukielka Jul 18, 2024

pkukielka commented Jul 18, 2024

		@get:OptionTag(tag = "tags", nameAttribute = "") var tags: MutableList<String> by list()

		@get:OptionTag(tag = "usage", nameAttribute = "") var usage: MutableList<String> by list()

		@@ -190,22 +198,20 @@ class SettingsMigrationTest : BasePlatformTestCase() {
		fun `test DeprecatedChatLlmMigration`() {

feat(chore): Adding tags and usage for LLM Models #1894

feat(chore): Adding tags and usage for LLM Models #1894

Conversation

jamesmcnamara commented Jul 9, 2024 • edited Loading

Test plan

Choose a reason for hiding this comment

pkukielka Jul 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

odisseus commented Jul 9, 2024

pkukielka Jul 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkukielka left a comment • edited Loading

Choose a reason for hiding this comment

jamesmcnamara commented Jul 10, 2024

pkukielka commented Jul 15, 2024

pkukielka Jul 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesmcnamara Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkukielka commented Jul 18, 2024

jamesmcnamara commented Jul 9, 2024 •

edited

Loading

pkukielka Jul 9, 2024 •

edited

Loading

pkukielka Jul 9, 2024 •

edited

Loading

pkukielka left a comment •

edited

Loading

pkukielka Jul 17, 2024 •

edited

Loading

jamesmcnamara Jul 18, 2024 •

edited

Loading