-
Notifications
You must be signed in to change notification settings - Fork 11.7k
feat(sdk): Implement dynamic system instructions #18863
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
d0ca053
feat(sdk): initial package bootstrap for SDK
mbleigh bf9d04f
Merge branch 'main' into sdk-01-bootstrap
mbleigh 5412b9d
fix(sdk): add missing src/index.ts export file
mbleigh 07ad7a0
fix(sdk): handle JSON string tool arguments and add TODO for AbortSignal
mbleigh 78aa0ac
Merge branch 'main' into sdk-01-bootstrap
mbleigh 5045cad
fix(sdk): enforce tool execution confirmation in GeminiCliAgent and r…
mbleigh 46b7f07
feat(sdk): implements SessionContext for SDK tool calls
mbleigh bcd5f21
fix(sdk): use ShellTool for policy enforcement in SdkAgentShell
mbleigh 57441ed
fix(sdk): resolve build and lint errors in SDK
mbleigh 854a12b
feat(sdk): implements SessionContext for SDK tool calls\n\n- adds tes…
mbleigh 984d63c
feat(sdk): Implement dynamic system instructions
mbleigh 72c2417
fix(sdk): throw error when dynamic instructions fail
mbleigh c787f33
fix(sdk): fix TS4111 error by using bracket notation for process.env …
mbleigh d274655
Merge branch 'main' into sdk-03-system-instructions
mbleigh 836bc9a
feat(sdk): Support async dynamic system instructions and ensure lazy …
mbleigh 1289779
Apply suggestions from code review
mbleigh 8a0a7ac
Merge branch 'main' into sdk-03-system-instructions
mbleigh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| /** | ||
| * @license | ||
| * Copyright 2026 Google LLC | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| import { describe, it, expect } from 'vitest'; | ||
| import { GeminiCliAgent } from './agent.js'; | ||
| import * as path from 'node:path'; | ||
| import { fileURLToPath } from 'node:url'; | ||
| import { dirname } from 'node:path'; | ||
|
|
||
| const __filename = fileURLToPath(import.meta.url); | ||
| const __dirname = dirname(__filename); | ||
|
|
||
| // Set this to true locally when you need to update snapshots | ||
| const RECORD_MODE = process.env['RECORD_NEW_RESPONSES'] === 'true'; | ||
|
|
||
| const getGoldenPath = (name: string) => | ||
| path.resolve(__dirname, '../test-data', `${name}.json`); | ||
|
|
||
| describe('GeminiCliAgent Integration', () => { | ||
| it('handles static instructions', async () => { | ||
| const goldenFile = getGoldenPath('agent-static-instructions'); | ||
|
|
||
| const agent = new GeminiCliAgent({ | ||
| instructions: 'You are a pirate. Respond in pirate speak.', | ||
| model: 'gemini-2.0-flash', | ||
| recordResponses: RECORD_MODE ? goldenFile : undefined, | ||
| fakeResponses: RECORD_MODE ? undefined : goldenFile, | ||
| }); | ||
|
|
||
| const events = []; | ||
| const stream = agent.sendStream('Say hello.'); | ||
|
|
||
| for await (const event of stream) { | ||
| events.push(event); | ||
| } | ||
|
|
||
| const textEvents = events.filter((e) => e.type === 'content'); | ||
| const responseText = textEvents | ||
| .map((e) => (typeof e.value === 'string' ? e.value : '')) | ||
| .join(''); | ||
|
|
||
| // Expect pirate speak | ||
| expect(responseText.toLowerCase()).toMatch(/ahoy|matey|arrr/); | ||
| }, 30000); | ||
|
|
||
| it('handles dynamic instructions', async () => { | ||
| const goldenFile = getGoldenPath('agent-dynamic-instructions'); | ||
|
|
||
| let callCount = 0; | ||
| const agent = new GeminiCliAgent({ | ||
| instructions: (_ctx) => { | ||
| callCount++; | ||
| return `You are a helpful assistant. The secret number is ${callCount}. Always mention the secret number when asked.`; | ||
| }, | ||
| model: 'gemini-2.0-flash', | ||
| recordResponses: RECORD_MODE ? goldenFile : undefined, | ||
| fakeResponses: RECORD_MODE ? undefined : goldenFile, | ||
| }); | ||
|
|
||
| // First turn | ||
| const stream1 = agent.sendStream('What is the secret number?'); | ||
| const events1 = []; | ||
| for await (const event of stream1) { | ||
| events1.push(event); | ||
| } | ||
| const responseText1 = events1 | ||
| .filter((e) => e.type === 'content') | ||
| .map((e) => (typeof e.value === 'string' ? e.value : '')) | ||
| .join(''); | ||
|
|
||
| expect(responseText1).toContain('1'); | ||
| expect(callCount).toBe(1); | ||
|
|
||
| // Second turn | ||
| const stream2 = agent.sendStream('What is the secret number now?'); | ||
| const events2 = []; | ||
| for await (const event of stream2) { | ||
| events2.push(event); | ||
| } | ||
| const responseText2 = events2 | ||
| .filter((e) => e.type === 'content') | ||
| .map((e) => (typeof e.value === 'string' ? e.value : '')) | ||
| .join(''); | ||
|
|
||
| // Should still be 1 because instructions are only loaded once per session | ||
| expect(responseText2).toContain('1'); | ||
| expect(callCount).toBe(1); | ||
| }, 30000); | ||
|
|
||
| it('handles async dynamic instructions', async () => { | ||
| const goldenFile = getGoldenPath('agent-async-instructions'); | ||
|
|
||
| let callCount = 0; | ||
| const agent = new GeminiCliAgent({ | ||
| instructions: async (_ctx) => { | ||
| await new Promise((resolve) => setTimeout(resolve, 10)); // Simulate async work | ||
| callCount++; | ||
| return `You are a helpful assistant. The secret number is ${callCount}. Always mention the secret number when asked.`; | ||
| }, | ||
| model: 'gemini-2.0-flash', | ||
| recordResponses: RECORD_MODE ? goldenFile : undefined, | ||
| fakeResponses: RECORD_MODE ? undefined : goldenFile, | ||
| }); | ||
|
|
||
| // First turn | ||
| const stream1 = agent.sendStream('What is the secret number?'); | ||
| const events1 = []; | ||
| for await (const event of stream1) { | ||
| events1.push(event); | ||
| } | ||
| const responseText1 = events1 | ||
| .filter((e) => e.type === 'content') | ||
| .map((e) => (typeof e.value === 'string' ? e.value : '')) | ||
| .join(''); | ||
|
|
||
| expect(responseText1).toContain('1'); | ||
| expect(callCount).toBe(1); | ||
|
|
||
| // Second turn | ||
| const stream2 = agent.sendStream('What is the secret number now?'); | ||
| const events2 = []; | ||
| for await (const event of stream2) { | ||
| events2.push(event); | ||
| } | ||
| const responseText2 = events2 | ||
| .filter((e) => e.type === 'content') | ||
| .map((e) => (typeof e.value === 'string' ? e.value : '')) | ||
| .join(''); | ||
|
|
||
| // Should still be 1 because instructions are only loaded once per session | ||
| expect(responseText2).toContain('1'); | ||
| expect(callCount).toBe(1); | ||
| }, 30000); | ||
|
|
||
| it('throws when dynamic instructions fail', async () => { | ||
| const agent = new GeminiCliAgent({ | ||
| instructions: () => { | ||
| throw new Error('Dynamic instruction failure'); | ||
| }, | ||
| model: 'gemini-2.0-flash', | ||
| }); | ||
|
|
||
| const stream = agent.sendStream('Say hello.'); | ||
|
|
||
| await expect(async () => { | ||
| for await (const _event of stream) { | ||
| // Just consume the stream | ||
| } | ||
| }).rejects.toThrow('Dynamic instruction failure'); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9831,"totalTokenCount":9831,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9831}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7098,"candidatesTokenCount":8,"totalTokenCount":7106,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7098}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9848,"totalTokenCount":9848,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9848}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7113,"candidatesTokenCount":8,"totalTokenCount":7121,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7113}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9853,"totalTokenCount":9853,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9853}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7120,"candidatesTokenCount":8,"totalTokenCount":7128,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7120}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9870,"totalTokenCount":9870,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9870}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7135,"candidatesTokenCount":8,"totalTokenCount":7143,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7135}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9831,"totalTokenCount":9831,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9831}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7098,"candidatesTokenCount":8,"totalTokenCount":7106,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7098}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9848,"totalTokenCount":9848,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9848}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7113,"candidatesTokenCount":8,"totalTokenCount":7121,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7113}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9853,"totalTokenCount":9853,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9853}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7120,"candidatesTokenCount":8,"totalTokenCount":7128,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7120}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"The secret number is"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9870,"totalTokenCount":9870,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9870}]}},{"candidates":[{"content":{"parts":[{"text":" 1.\n"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7135,"candidatesTokenCount":8,"totalTokenCount":7143,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7135}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":8}]}}]} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| {"method":"generateContentStream","response":[{"candidates":[{"content":{"parts":[{"text":"Ah"}],"role":"model"}}],"usageMetadata":{"promptTokenCount":9828,"totalTokenCount":9828,"promptTokensDetails":[{"modality":"TEXT","tokenCount":9828}]}},{"candidates":[{"content":{"parts":[{"text":"oy, matey! Ready to chart a course through the code?"}],"role":"model"},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":7095,"candidatesTokenCount":15,"totalTokenCount":7110,"promptTokensDetails":[{"modality":"TEXT","tokenCount":7095}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":15}]}}]} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
intended?