Skip to content

Commit 647c93b

Browse files
committed
feat: Introduce new tools and refactor existing ones
- Added `FileChangePlanTool` for managing file change plans with user approval. - Introduced `RejectFileChangesTool` to handle rejection of file changes with reasons. - Implemented `WebSearchTool` for performing web searches and returning results. - Created `WriteToFileTool` for writing content to files with user confirmation. - Added `ReadFileTool` for reading file content with structured responses. - Refactored existing tools to use a unified `definitions` module for parameter schemas. - Enhanced `ToolExecutor` to include new tools and manage their execution. - Developed utility functions for generating prompts and documentation for tools. - Updated `ToolParser` to support both standard and legacy XML formats for tool calls. - Improved error handling and user feedback mechanisms across tools.
1 parent b7d0488 commit 647c93b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+3525
-15
lines changed

codemcp.toml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[commands]
2+
format = ["./run_format.sh"]
3+
test = ["./run_test.sh"]
Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# Migration Plan: Unified Tool Definitions
2+
3+
This document outlines the steps to migrate from the existing separate tool definitions in `/schema` and `/prompts/tools` to the new unified architecture in `/definitions`.
4+
5+
## Overview
6+
7+
The migration involves:
8+
9+
1. Replacing schema imports with unified definition imports
10+
2. Updating the ToolExecutor to use the new definitions
11+
3. Modifying the ToolParser to work with the unified format
12+
4. Updating tool runners to reference the new definitions
13+
5. Integrating the system prompt generator
14+
15+
## Detailed Steps
16+
17+
### 1. Update Tool Executor
18+
19+
The `tool-executor.ts` file needs to be modified to use the new unified definitions:
20+
21+
```typescript
22+
// Replace this import
23+
import { tools, writeToFileTool } from "./schema"
24+
25+
// With this import
26+
import { toolDefinitions, writeToFileTool } from "./definitions"
27+
28+
// Update the ToolParser initialization
29+
this.toolParser = new ToolParser(
30+
toolDefinitions.map((tool) => ({ name: tool.name, schema: tool.schema })),
31+
{
32+
onToolUpdate: this.handleToolUpdate.bind(this),
33+
onToolEnd: this.handleToolEnd.bind(this),
34+
onToolError: this.handleToolError.bind(this),
35+
}
36+
)
37+
38+
// Make the createTool method more dynamic
39+
private createTool(params: FullToolParams<any>) {
40+
const toolName = params.name as ToolName;
41+
42+
// Define the mapping of tool names to their implementation classes
43+
const toolImplementations = {
44+
read_file: ReadFileTool,
45+
search_files: SearchFilesTool,
46+
execute_command: ExecuteCommandTool,
47+
list_files: ListFilesTool,
48+
file_editor: FileEditorTool,
49+
ask_followup_question: AskFollowupQuestionTool,
50+
search_symbol: SearchSymbolTool,
51+
url_screenshot: UrlScreenshotTool,
52+
attempt_completion: AttemptCompletionTool,
53+
explore_repo_folder: ExploreRepoFolderTool,
54+
spawn_agent: SpawnAgentTool,
55+
exit_agent: ExitAgentTool,
56+
server_runner: ServerRunnerTool,
57+
web_search: WebSearchTool,
58+
write_to_file: WriteToFileTool,
59+
add_interested_file: AddInterestedFileTool,
60+
file_changes_plan: FileChangePlanTool,
61+
submit_review: SubmitReviewTool,
62+
reject_file_changes: RejectFileChangesTool,
63+
} as const;
64+
65+
const ToolClass = toolImplementations[toolName];
66+
if (!ToolClass) {
67+
throw new Error(`Unknown tool: ${params.name}`);
68+
}
69+
70+
return new ToolClass(params, this.options);
71+
}
72+
```
73+
74+
### 2. Update Tool Runners
75+
76+
Each tool runner should be updated to import its types from the new unified definitions:
77+
78+
```typescript
79+
// Replace this import
80+
import { ReadFileToolParams } from "../schema/read_file";
81+
82+
// With this import
83+
import { ReadFileToolParams } from "../definitions";
84+
```
85+
86+
### 3. Integrate System Prompt Generator
87+
88+
Update the agent initialization to use the new system prompt generator:
89+
90+
```typescript
91+
// Import the generator
92+
import { generateToolsSystemPrompt } from "../tools/utils/system-prompt-generator";
93+
94+
// Use it in the system prompt
95+
const systemPrompt = `
96+
${baseSystemPrompt}
97+
98+
${generateToolsSystemPrompt()}
99+
100+
${additionalInstructions}
101+
`;
102+
```
103+
104+
### 4. Update Tool Parser
105+
106+
Enhance the ToolParser to support both the standard and legacy XML formats:
107+
108+
```typescript
109+
// Add support for both formats in the pattern matching
110+
const standardPattern = /<tool name="([^"]+)">([\s\S]*?)<\/tool>/g;
111+
const legacyPattern = /<([^>]+)>([\s\S]*?)<\/\1>/g;
112+
113+
// Check both patterns when parsing
114+
function parseToolUse(text: string) {
115+
let match;
116+
117+
// Try standard format first
118+
match = standardPattern.exec(text);
119+
if (match) {
120+
const [fullMatch, toolName, paramsText] = match;
121+
// Process params and return
122+
return { toolName, params, fullMatch };
123+
}
124+
125+
// Try legacy format as fallback
126+
match = legacyPattern.exec(text);
127+
if (match) {
128+
const [fullMatch, toolName, paramsText] = match;
129+
// Process params and return
130+
return { toolName, params, fullMatch };
131+
}
132+
133+
return null;
134+
}
135+
```
136+
137+
### 5. Phase Out Old Definitions
138+
139+
Once the new system is in place and tested:
140+
141+
1. Create a deprecation notice in the `/schema` and `/prompts/tools` directories
142+
2. Mark the old imports as deprecated in the codebase
143+
3. Establish a timeline for removing the old files entirely
144+
145+
## Testing Strategy
146+
147+
1. Create unit tests for the new unified definitions
148+
2. Test each tool with both standard and legacy XML formats
149+
3. Perform integration tests with the ToolExecutor
150+
4. Verify the system prompt generation includes all tools correctly
151+
5. Conduct end-to-end tests with the Claude agent using the new formats
152+
153+
## Rollout Plan
154+
155+
1. Implement the changes in a separate branch
156+
2. Review and test thoroughly
157+
3. Merge to development branch and test in the full system
158+
4. Schedule the production deployment
159+
5. Monitor for any issues after deployment
160+
6. After successful rollout, create a cleanup ticket to remove the deprecated code
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Next Steps for Tool Definition Unification
2+
3+
## Completed Work
4+
5+
1. ✅ Created a unified tool call template with consistent XML format
6+
2. ✅ Implemented 19 tool definitions using the new unified format:
7+
- read_file
8+
- search_files
9+
- execute_command
10+
- list_files
11+
- file_editor
12+
- ask_followup_question
13+
- search_symbol
14+
- url_screenshot
15+
- attempt_completion
16+
- explore_repo_folder
17+
- spawn_agent
18+
- exit_agent
19+
- server_runner
20+
- web_search
21+
- write_to_file
22+
- add_interested_file
23+
- file_changes_plan
24+
- submit_review
25+
- reject_file_changes
26+
3. ✅ Created helper functions to manage tool definitions
27+
4. ✅ Developed system prompt generator for consistent LLM instructions
28+
5. ✅ Documented the new architecture and migration strategy
29+
30+
## Pending Tasks
31+
32+
1. 🔲 Update the ToolExecutor to use unified definitions
33+
- Modify imports to use new definitions
34+
- Update ToolParser initialization
35+
- Make createTool method more dynamic
36+
37+
2. 🔲 Update tool runners to reference unified definitions
38+
- Replace schema imports with definition imports
39+
- Ensure type compatibility
40+
41+
3. 🔲 Integrate the system prompt generator
42+
- Connect to agent initialization
43+
- Test generation with all tools
44+
45+
4. 🔲 Enhance the ToolParser
46+
- Add support for both standard and legacy XML formats
47+
- Ensure parameter extraction works with new format
48+
49+
5. 🔲 Testing
50+
- Create unit tests for the unified definitions
51+
- Test LLM understanding of the new format
52+
- End-to-end testing with all tools
53+
54+
6. 🔲 Documentation
55+
- Update developer documentation to reflect the new architecture
56+
- Add examples of how to create new tool definitions
57+
58+
## Benefits of the New Architecture
59+
60+
1. **Single Source of Truth**: All information about a tool is in one place
61+
2. **Type Safety**: TypeScript types are derived directly from Zod schemas
62+
3. **Consistent Calling Format**: All tools use the same XML format
63+
4. **Better LLM Understanding**: Standardized examples improve Claude's tool usage
64+
5. **Easier Maintenance**: Adding or modifying tools requires changes in fewer places
65+
6. **Automated Documentation**: System prompts can be generated from definitions
66+
7. **Future Compatibility**: The format supports both current and future Claude models
67+
68+
## Implementation Priority
69+
70+
1. Start with the ToolExecutor updates, as this is the central integration point
71+
2. Update the most commonly used tool runners first
72+
3. Integrate the system prompt generator to improve LLM understanding
73+
4. Add testing throughout the process
74+
5. Continue with less frequently used tools
75+
6. Complete documentation updates last
76+
77+
## Timeline Estimate
78+
79+
- **Week 1**: Implement ToolExecutor changes and update common runners
80+
- **Week 2**: Update remaining runners and integrate system prompt generator
81+
- **Week 3**: Testing, bug fixing, and documentation
82+
- **Week 4**: Phase out old definitions and monitor for issues
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Unified Tool Definitions
2+
3+
This directory contains unified definitions for all tools used in the Claude Dev agent system. Each tool definition combines:
4+
5+
1. **Schema** - Zod schema for validation
6+
2. **Types** - TypeScript types derived from the schema
7+
3. **Prompt Information** - Descriptions, capabilities, and examples for LLM consumption
8+
4. **Calling Format** - Standardized XML format for tool invocation
9+
10+
## Architecture
11+
12+
The unified tool definitions provide a single source of truth for each tool, ensuring consistency between:
13+
- Runtime validation (Zod schema)
14+
- Type safety (TypeScript types)
15+
- LLM instructions (prompt definitions)
16+
- Tool invocation format (XML template)
17+
18+
### Key Components
19+
20+
- **tool-call-template.ts** - Defines the XML template format and helpers
21+
- **index.ts** - Registry of all tool definitions with helper functions
22+
- **{tool-name}.tool.ts** - Individual tool definitions
23+
24+
## XML Tool Calling Format
25+
26+
All tools use a consistent XML format:
27+
28+
```xml
29+
<tool name="tool_name">
30+
<parameter1>value1</parameter1>
31+
<parameter2>value2</parameter2>
32+
</tool>
33+
```
34+
35+
This provides Claude with a standardized way to invoke tools.
36+
37+
## Usage
38+
39+
1. **Adding a new tool**:
40+
- Create a new file `{tool-name}.tool.ts`
41+
- Define the schema, types, and prompt information
42+
- Register it in `index.ts`
43+
44+
2. **Fetching tool information**:
45+
- `getToolDefinition(name)` - Get a specific tool definition
46+
- `getToolCallFormat(name)` - Get XML format for a specific tool
47+
- `getAllToolExamples()` - Get examples for system prompts
48+
49+
## Tool List
50+
51+
The system currently provides definitions for the following tools:
52+
53+
- `read_file` - Read file contents
54+
- `search_files` - Search for files with a pattern
55+
- `execute_command` - Execute a shell command
56+
- `list_files` - List files in a directory
57+
- `file_editor` - Edit file contents
58+
- `ask_followup_question` - Ask a clarifying question
59+
- `search_symbol` - Search for code symbols
60+
- `url_screenshot` - Take a screenshot of a URL
61+
- `attempt_completion` - Attempt to complete a task
62+
- `explore_repo_folder` - Explore a repository folder
63+
- `spawn_agent` - Create a new agent
64+
- `exit_agent` - Exit the current agent
65+
- `server_runner` - Run a development server
66+
- `web_search` - Search the web for information
67+
- `write_to_file` - Write content to a file
68+
- `add_interested_file` - Track files relevant to the task
69+
- `file_changes_plan` - Plan file changes
70+
- `submit_review` - Submit a review of progress
71+
- `reject_file_changes` - Reject proposed file changes
72+
73+
## Integration
74+
75+
This unified architecture integrates with:
76+
- Tool execution logic in `/agent/v1/tools/runners/`
77+
- Tool schema validation in `/agent/v1/tools/schema/`
78+
- System prompt generation in `/agent/v1/prompts/tools/`

0 commit comments

Comments
 (0)