Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BYOM]: Allow setting a custom max context size for custom models #41167

Closed
LorenzoMinto opened this issue Sep 20, 2024 · 4 comments · Fixed by brave/brave-core#25864
Closed

[BYOM]: Allow setting a custom max context size for custom models #41167

LorenzoMinto opened this issue Sep 20, 2024 · 4 comments · Fixed by brave/brave-core#25864
Assignees
@jonathansampson
Labels
browser-ai feature-request good first issue OS/Desktop priority/P3 The next thing for us to work on. It'll ride the trains. QA Pass-Win64 QA/Yes release-notes/include
Milestone
1.74.x - Release

Comments

@LorenzoMinto
Copy link

Platforms

Linux, macOS, Windows

Description

BYOM currently uses a fixed context size for all custom models. Make the context size editable in the model details cards.
@LorenzoMinto LorenzoMinto changed the title [BYOM]: Allow setting a custom max context size for a model [BYOM]: Allow setting a custom max context size for custom models Sep 20, 2024
@The-unknown-Shadowman
Copy link

I would like to see this too.

@jonathansampson
Copy link
Contributor

It would be even cooler if, for supported endpoints, Brave could ask for this information. For example, ollama show phi3.5 relays quite a bit of metadata regarding the model, including the context window size. More info at https://github.com/ollama/ollama/blob/main/docs/api.md#show-model-information.

@jonathansampson jonathansampson self-assigned this Oct 5, 2024
This was referenced Oct 5, 2024
Closed
Merged
@mattmcalister mattmcalister moved this to In Progress in Browser AI Oct 8, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Browser AI Oct 17, 2024
@brave-builds brave-builds added this to the 1.73.x - Nightly milestone Oct 17, 2024
@srirambv
Copy link
Contributor

Blocked on #42590

@srirambv
Copy link
Contributor

srirambv commented Dec 4, 2024

Verification passed on

Brave 1.74.18 Chromium: 131.0.6778.85 (Official Build) beta (64-bit)
Revision 9e238f4e6036c0b250ffefd506a55813f3aad501
OS Windows 11 Version 23H2 (Build 22631.4460)
  • Verified steps from brave/brave-core#25864
  • Verified with low context size, query shows warning message about conversation length
  • Encountered #42631
41167.mp4

@LaurenWags LaurenWags mentioned this issue Jan 14, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jonathansampson jonathansampson

Labels
browser-ai feature-request good first issue OS/Desktop priority/P3 The next thing for us to work on. It'll ride the trains. QA Pass-Win64 QA/Yes release-notes/include
Projects
Browser AI
Status: Done
Milestone
1.74.x - Release
Development

Successfully merging a pull request may close this issue.

[BYOM] Introduces support for custom context sizes brave/brave-core
5 participants
@jonathansampson @LorenzoMinto @srirambv @brave-builds @The-unknown-Shadowman