Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the Content caching feature #167

Merged
merged 15 commits into from
Jun 17, 2024
Merged

Add the Content caching feature #167

merged 15 commits into from
Jun 17, 2024

Conversation

hsubox76
Copy link
Collaborator

@hsubox76 hsubox76 commented Jun 6, 2024 •
edited
Loading

Feature: Content Caching

Design doc (API) (internal): https://docs.google.com/document/d/1kJTdoz-RO7mkzsjE1QDfJfD1YvSxJXuzmqTAtsS1hJ0/edit?resourcekey=0-cv2C2J8SLDn__yoetTPvBw&tab=t.0
Design doc (Python SDK) (internal): https://docs.google.com/document/d/11en7rPCuXIu-hfF71jUDWkfQctWeFEoq_5KgkinnxWg/edit?resourcekey=0-ntvB0-Y_GBQXOfkO-KDaXg&tab=t.0

APIs added

  • GoogleAICacheManager class (exported from new /server path) with create, update, get, list, delete methods.
  • GoogleGenerativeAI.getGenerativeModelFromContextCache() added

Infrastructure changed

  • /files subpath changed to /server. GoogleAIFileManager and GoogleAICacheManager will be exported from here as they are only meant to be used in a server environment.
  • All types moved to a single folder.
  • Generate common/api-review md files for easier review of API mistakes/gaps
  • Refactor makeRequest-related functions to reduce code duplication

DellaBitta
DellaBitta reviewed Jun 6, 2024
View reviewed changes
packages/main/server/package.json Outdated Show resolved Hide resolved
packages/main/src/requests/request.ts Outdated Show resolved Hide resolved
packages/main/src/requests/request.ts Outdated Show resolved Hide resolved
packages/main/src/server/cache-manager.ts Outdated Show resolved Hide resolved
packages/main/src/server/cache-manager.ts Show resolved Hide resolved
packages/main/src/server/cache-manager.ts Show resolved Hide resolved
packages/main/src/server/request.ts Show resolved Hide resolved
packages/main/types/function-calling.ts Show resolved Hide resolved
packages/main/types/server/caching.ts Show resolved Hide resolved
packages/main/types/server/caching.ts Outdated Show resolved Hide resolved
@DellaBitta DellaBitta changed the title DO NOT MERGE: Add context caching - draft DO NOT MERGE: Add context caching Jun 13, 2024
@DellaBitta
Add Integration Tests for Content Caching (#171)
39a4f8d 
Updates to the Content Caching feature in development.

 - adds integration tests
 - adds display name to the creation of content caching.
 - fixes the use of a context cache in the sample implementation.

Documentation PR is separate: #172
@DellaBitta
Cached Content generated documentation (#172)
8f5c338 
An isolated pull request with the Cached Content documentation.
@DellaBitta
Copy link
Collaborator

Fixes merged.
Documentation merged.

@hsubox76 hsubox76 marked this pull request as ready for review June 17, 2024 19:50
@hsubox76 hsubox76 changed the title DO NOT MERGE: Add context caching DO NOT MERGE: Add content caching Jun 17, 2024
@DellaBitta DellaBitta changed the title DO NOT MERGE: Add content caching Add the Content caching feature Jun 17, 2024
@DellaBitta DellaBitta merged commit 5df61d1 into main Jun 17, 2024
4 checks passed
@DellaBitta DellaBitta deleted the ch-caching branch June 17, 2024 23:51
@melvinchia3636 melvinchia3636 mentioned this pull request Jul 10, 2024
Open
@Anomaly-1 Anomaly-1 mentioned this pull request Jul 10, 2024
Open
@nestorzamili nestorzamili mentioned this pull request Jul 10, 2024
Closed
@leoo1992 leoo1992 mentioned this pull request Jul 10, 2024
Merged
@melvinchia3636 melvinchia3636 mentioned this pull request Jul 18, 2024
Open
@nestorzamili nestorzamili mentioned this pull request Jul 18, 2024
Closed
@Anomaly-1 Anomaly-1 mentioned this pull request Jul 18, 2024
Open
@leoo1992 leoo1992 mentioned this pull request Jul 19, 2024
Merged
@aliakseikrauchanka aliakseikrauchanka mentioned this pull request Jul 21, 2024
Closed
@Anomaly-1 Anomaly-1 mentioned this pull request Jul 23, 2024
Open
@nestorzamili nestorzamili mentioned this pull request Jul 23, 2024
Closed
@aliakseikrauchanka aliakseikrauchanka mentioned this pull request Jul 23, 2024
Merged
@melvinchia3636 melvinchia3636 mentioned this pull request Jul 23, 2024
Open
This was referenced Aug 1, 2024
Open
Open
@Anomaly-1 Anomaly-1 mentioned this pull request Aug 6, 2024
Open
@nestorzamili nestorzamili mentioned this pull request Aug 6, 2024
Merged
@melvinchia3636 melvinchia3636 mentioned this pull request Aug 6, 2024
Open
@Anomaly-1 Anomaly-1 mentioned this pull request Aug 15, 2024
Open
@melvinchia3636 melvinchia3636 mentioned this pull request Aug 15, 2024
Open
@paulorezende007 paulorezende007 mentioned this pull request Aug 21, 2024
Open
@melvinchia3636 melvinchia3636 mentioned this pull request Aug 25, 2024
Open
@melvinchia3636 melvinchia3636 mentioned this pull request Sep 4, 2024
Open
@Anomaly-1 Anomaly-1 mentioned this pull request Sep 4, 2024
Open
This was referenced Sep 7, 2024
Open
Open
@Anomaly-1 Anomaly-1 mentioned this pull request Sep 11, 2024
Open
@melvinchia3636 melvinchia3636 mentioned this pull request Oct 2, 2024
Open
This was referenced Oct 2, 2024
Open
Open
@melvinchia3636 melvinchia3636 mentioned this pull request Oct 3, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DellaBitta DellaBitta DellaBitta approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hsubox76 @DellaBitta