Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(go/plugins/vertexai): add context caching to vertexai #1478

Closed
wants to merge 5 commits into from

Conversation

falonso81
Copy link
Contributor

@falonso81 falonso81 commented Dec 10, 2024

Add context caching to vertexai
#1443

How to test:

func main() {
    fmt.Println("hello world!!!!")
    ctx := context.Background()
    if err := vertexai.Init(ctx, nil); err != nil {
        log.Fatal(err)
    }
    m := vertexai.Model("gemini-1.5-flash-002")
    if m == nil {
        log.Fatal(errors.New("vertexai init failed"))
    }

    strrr, err := ai.GenerateText(ctx, m, ai.WithTextPrompt("Tell me a joke about golang developers."))
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(strrr)

    strrr, err = ai.GenerateText(ctx, m, ai.WithTextPrompt("Tell me a joke about golang developers. "))
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(strrr)
}

Checklist (if applicable):

@falonso81 falonso81 changed the title feat(go): add context caching to vertexai feat(go/plugins/vertexai): add context caching to vertexai Dec 10, 2024
"gemini-1.5-flash": gemini.Multimodal,
"gemini-1.0-pro": gemini.BasicText,
"gemini-1.5-pro": gemini.Multimodal,
"gemini-1.5-flash-002": gemini.Multimodal,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context caching is working only with stable Gemini model versions using number prefix like https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions#stable-versions-available

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get rid of the plain gemini-1.5-flash. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.

@falonso81 falonso81 requested a review from apascal07 December 10, 2024 14:45
@falonso81 falonso81 linked an issue Dec 13, 2024 that may be closed by this pull request
@apascal07
Copy link
Collaborator

Please add tests for the caching business logic and a sample where this is used (by setting the cache TTL, which enables context caching). See #1297 for test coverage.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go style guide uses tabs (as 4 spaces) instead of 2 spaces. Please update here and in the other PR.

"gemini-1.5-flash": gemini.Multimodal,
"gemini-1.0-pro": gemini.BasicText,
"gemini-1.5-pro": gemini.Multimodal,
"gemini-1.5-flash-002": gemini.Multimodal,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get rid of the plain gemini-1.5-flash. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.

@falonso81 falonso81 force-pushed the 1443-add-context-caching-to-vertex-ai branch from 8562c77 to 7ef5090 Compare December 17, 2024 17:57
@falonso81
Copy link
Contributor Author

closing , new PR here #1566

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants