Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Agent Context window + context window refactor. #2126

Merged
merged 3 commits into from
Aug 15, 2024

Conversation

timothycarambat
Copy link
Member

Pull Request Type

  • ✨ feat
  • πŸ› fix
  • ♻️ refactor
  • πŸ’„ style
  • πŸ”¨ chore
  • πŸ“ docs

Relevant Issues

resolves #2121

What is in this change?

  • Added proper context window referencing for provider/model on @agent directive
  • Refactored model/context mapping to external file so it is all in one place (only applied to fixed model selection providers)
  • Added tiktoken counter to summarize/web-scraping assessment for model.

Additional Information

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated
  • I have tested my code functionality
  • Docker build succeeds locally

Add token count to document length instead of char-count
refernce promptWindowLimit from AIProvider in central location
@timothycarambat timothycarambat merged commit 99f2c25 into master Aug 15, 2024
@timothycarambat timothycarambat deleted the 2121-agent-context-windows branch August 15, 2024 19:13
TuanBC pushed a commit to TuanBC/anything-llm that referenced this pull request Aug 26, 2024
* Enable agent context windows to be accurate per provider:model

* Refactor model mapping to external file
Add token count to document length instead of char-count
refernce promptWindowLimit from AIProvider in central location

* remove unused imports
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG]: Scraping with agent does not leverage full context window
1 participant