feat: Introduce new tools and refactor existing ones

codemcp.toml

@@ -0,0 +1,3 @@
+[commands]
+format = ["./run_format.sh"]
+test = ["./run_test.sh"]

extension/src/agent/v1/tools/definitions/MIGRATION.md

@@ -0,0 +1,160 @@
+# Migration Plan: Unified Tool Definitions
+
+This document outlines the steps to migrate from the existing separate tool definitions in `/schema` and `/prompts/tools` to the new unified architecture in `/definitions`.
+
+## Overview
+
+The migration involves:
+
+1. Replacing schema imports with unified definition imports
+2. Updating the ToolExecutor to use the new definitions
+3. Modifying the ToolParser to work with the unified format
+4. Updating tool runners to reference the new definitions
+5. Integrating the system prompt generator
+
+## Detailed Steps
+
+### 1. Update Tool Executor
+
+The `tool-executor.ts` file needs to be modified to use the new unified definitions:
+
+```typescript
+// Replace this import
+import { tools, writeToFileTool } from "./schema"
+
+// With this import
+import { toolDefinitions, writeToFileTool } from "./definitions"
+
+// Update the ToolParser initialization
+this.toolParser = new ToolParser(
+  toolDefinitions.map((tool) => ({ name: tool.name, schema: tool.schema })),
+  {
+    onToolUpdate: this.handleToolUpdate.bind(this),
+    onToolEnd: this.handleToolEnd.bind(this),
+    onToolError: this.handleToolError.bind(this),
+  }
+)
+
+// Make the createTool method more dynamic
+private createTool(params: FullToolParams<any>) {
+  const toolName = params.name as ToolName;
+
+  // Define the mapping of tool names to their implementation classes
+  const toolImplementations = {
+    read_file: ReadFileTool,
+    search_files: SearchFilesTool,
+    execute_command: ExecuteCommandTool,
+    list_files: ListFilesTool,
+    file_editor: FileEditorTool,
+    ask_followup_question: AskFollowupQuestionTool,
+    search_symbol: SearchSymbolTool,
+    url_screenshot: UrlScreenshotTool,
+    attempt_completion: AttemptCompletionTool,
+    explore_repo_folder: ExploreRepoFolderTool,
+    spawn_agent: SpawnAgentTool,
+    exit_agent: ExitAgentTool,
+    server_runner: ServerRunnerTool,
+    web_search: WebSearchTool,
+    write_to_file: WriteToFileTool,
+    add_interested_file: AddInterestedFileTool,
+    file_changes_plan: FileChangePlanTool,
+    submit_review: SubmitReviewTool,
+    reject_file_changes: RejectFileChangesTool,
+  } as const;
+
+  const ToolClass = toolImplementations[toolName];
+  if (!ToolClass) {
+    throw new Error(`Unknown tool: ${params.name}`);
+  }
+
+  return new ToolClass(params, this.options);
+}
+```
+
+### 2. Update Tool Runners
+
+Each tool runner should be updated to import its types from the new unified definitions:
+
+```typescript
+// Replace this import
+import { ReadFileToolParams } from "../schema/read_file";
+
+// With this import
+import { ReadFileToolParams } from "../definitions";
+```
+
+### 3. Integrate System Prompt Generator
+
+Update the agent initialization to use the new system prompt generator:
+
+```typescript
+// Import the generator
+import { generateToolsSystemPrompt } from "../tools/utils/system-prompt-generator";
+
+// Use it in the system prompt
+const systemPrompt = `
+${baseSystemPrompt}
+
+${generateToolsSystemPrompt()}
+
+${additionalInstructions}
+`;
+```
+
+### 4. Update Tool Parser
+
+Enhance the ToolParser to support both the standard and legacy XML formats:
+
+```typescript
+// Add support for both formats in the pattern matching
+const standardPattern = /<tool name="([^"]+)">([\s\S]*?)<\/tool>/g;
+const legacyPattern = /<([^>]+)>([\s\S]*?)<\/\1>/g;
+
+// Check both patterns when parsing
+function parseToolUse(text: string) {
+  let match;
+
+  // Try standard format first
+  match = standardPattern.exec(text);
+  if (match) {
+    const [fullMatch, toolName, paramsText] = match;
+    // Process params and return
+    return { toolName, params, fullMatch };
+  }
+
+  // Try legacy format as fallback
+  match = legacyPattern.exec(text);
+  if (match) {
+    const [fullMatch, toolName, paramsText] = match;
+    // Process params and return
+    return { toolName, params, fullMatch };
+  }
+
+  return null;
+}
+```
+
+### 5. Phase Out Old Definitions
+
+Once the new system is in place and tested:
+
+1. Create a deprecation notice in the `/schema` and `/prompts/tools` directories
+2. Mark the old imports as deprecated in the codebase
+3. Establish a timeline for removing the old files entirely
+
+## Testing Strategy
+
+1. Create unit tests for the new unified definitions
+2. Test each tool with both standard and legacy XML formats
+3. Perform integration tests with the ToolExecutor
+4. Verify the system prompt generation includes all tools correctly
+5. Conduct end-to-end tests with the Claude agent using the new formats
+
+## Rollout Plan
+
+1. Implement the changes in a separate branch
+2. Review and test thoroughly
+3. Merge to development branch and test in the full system
+4. Schedule the production deployment
+5. Monitor for any issues after deployment
+6. After successful rollout, create a cleanup ticket to remove the deprecated code

extension/src/agent/v1/tools/definitions/NEXT_STEPS.md

@@ -0,0 +1,82 @@
+# Next Steps for Tool Definition Unification
+
+## Completed Work
+
+1. ✅ Created a unified tool call template with consistent XML format
+2. ✅ Implemented 19 tool definitions using the new unified format:
+   - read_file
+   - search_files
+   - execute_command
+   - list_files
+   - file_editor
+   - ask_followup_question
+   - search_symbol
+   - url_screenshot
+   - attempt_completion
+   - explore_repo_folder
+   - spawn_agent
+   - exit_agent
+   - server_runner
+   - web_search
+   - write_to_file
+   - add_interested_file
+   - file_changes_plan
+   - submit_review
+   - reject_file_changes
+3. ✅ Created helper functions to manage tool definitions
+4. ✅ Developed system prompt generator for consistent LLM instructions
+5. ✅ Documented the new architecture and migration strategy
+
+## Pending Tasks
+
+1. 🔲 Update the ToolExecutor to use unified definitions
+   - Modify imports to use new definitions
+   - Update ToolParser initialization
+   - Make createTool method more dynamic
+
+2. 🔲 Update tool runners to reference unified definitions
+   - Replace schema imports with definition imports
+   - Ensure type compatibility
+
+3. 🔲 Integrate the system prompt generator
+   - Connect to agent initialization
+   - Test generation with all tools
+
+4. 🔲 Enhance the ToolParser
+   - Add support for both standard and legacy XML formats
+   - Ensure parameter extraction works with new format
+
+5. 🔲 Testing
+   - Create unit tests for the unified definitions
+   - Test LLM understanding of the new format
+   - End-to-end testing with all tools
+
+6. 🔲 Documentation
+   - Update developer documentation to reflect the new architecture
+   - Add examples of how to create new tool definitions
+
+## Benefits of the New Architecture
+
+1. **Single Source of Truth**: All information about a tool is in one place
+2. **Type Safety**: TypeScript types are derived directly from Zod schemas
+3. **Consistent Calling Format**: All tools use the same XML format
+4. **Better LLM Understanding**: Standardized examples improve Claude's tool usage
+5. **Easier Maintenance**: Adding or modifying tools requires changes in fewer places
+6. **Automated Documentation**: System prompts can be generated from definitions
+7. **Future Compatibility**: The format supports both current and future Claude models
+
+## Implementation Priority
+
+1. Start with the ToolExecutor updates, as this is the central integration point
+2. Update the most commonly used tool runners first
+3. Integrate the system prompt generator to improve LLM understanding
+4. Add testing throughout the process
+5. Continue with less frequently used tools
+6. Complete documentation updates last
+
+## Timeline Estimate
+
+- **Week 1**: Implement ToolExecutor changes and update common runners
+- **Week 2**: Update remaining runners and integrate system prompt generator
+- **Week 3**: Testing, bug fixing, and documentation
+- **Week 4**: Phase out old definitions and monitor for issues

extension/src/agent/v1/tools/definitions/README.md

@@ -0,0 +1,78 @@
+# Unified Tool Definitions
+
+This directory contains unified definitions for all tools used in the Claude Dev agent system. Each tool definition combines:
+
+1. **Schema** - Zod schema for validation
+2. **Types** - TypeScript types derived from the schema 
+3. **Prompt Information** - Descriptions, capabilities, and examples for LLM consumption
+4. **Calling Format** - Standardized XML format for tool invocation
+
+## Architecture
+
+The unified tool definitions provide a single source of truth for each tool, ensuring consistency between:
+- Runtime validation (Zod schema)
+- Type safety (TypeScript types)
+- LLM instructions (prompt definitions)
+- Tool invocation format (XML template)
+
+### Key Components
+
+- **tool-call-template.ts** - Defines the XML template format and helpers
+- **index.ts** - Registry of all tool definitions with helper functions
+- **{tool-name}.tool.ts** - Individual tool definitions
+
+## XML Tool Calling Format
+
+All tools use a consistent XML format:
+
+```xml
+<tool name="tool_name">
+  <parameter1>value1</parameter1>
+  <parameter2>value2</parameter2>
+</tool>
+```
+
+This provides Claude with a standardized way to invoke tools.
+
+## Usage
+
+1. **Adding a new tool**:
+   - Create a new file `{tool-name}.tool.ts`
+   - Define the schema, types, and prompt information
+   - Register it in `index.ts`
+
+2. **Fetching tool information**:
+   - `getToolDefinition(name)` - Get a specific tool definition
+   - `getToolCallFormat(name)` - Get XML format for a specific tool
+   - `getAllToolExamples()` - Get examples for system prompts
+
+## Tool List
+
+The system currently provides definitions for the following tools:
+
+- `read_file` - Read file contents
+- `search_files` - Search for files with a pattern
+- `execute_command` - Execute a shell command
+- `list_files` - List files in a directory
+- `file_editor` - Edit file contents
+- `ask_followup_question` - Ask a clarifying question
+- `search_symbol` - Search for code symbols
+- `url_screenshot` - Take a screenshot of a URL
+- `attempt_completion` - Attempt to complete a task
+- `explore_repo_folder` - Explore a repository folder
+- `spawn_agent` - Create a new agent
+- `exit_agent` - Exit the current agent
+- `server_runner` - Run a development server
+- `web_search` - Search the web for information
+- `write_to_file` - Write content to a file
+- `add_interested_file` - Track files relevant to the task
+- `file_changes_plan` - Plan file changes
+- `submit_review` - Submit a review of progress
+- `reject_file_changes` - Reject proposed file changes
+
+## Integration
+
+This unified architecture integrates with:
+- Tool execution logic in `/agent/v1/tools/runners/`
+- Tool schema validation in `/agent/v1/tools/schema/`
+- System prompt generation in `/agent/v1/prompts/tools/`