Add comprehensive analysis of Anthropic Claude Skills and Sourcegraph Toolbox integration strategies

[Copilot]

Overview

This PR delivers a comprehensive analysis of integrating Anthropic Claude Skills and Sourcegraph Toolbox capabilities into the PSAI ecosystem, as requested in the issue. Based on successful proof-of-concept work, this analysis provides three detailed implementation plans with recommendations to guide the decision-making process.

What's Included

Documentation Deliverables

Three comprehensive documentation files have been added to a new docs/ directory:

1. docs/DECISION_SUMMARY.md (185 lines) - Quick reference guide

   • Side-by-side comparison of all implementation options
   • Clear recommendations with supporting rationale
   • Decision-making framework and key questions

2. docs/CLAUDE_SKILLS_TOOLBOX_ANALYSIS.md (965 lines) - Complete analysis

   • Review of current PSAI architecture and design patterns
   • Deep dive into Anthropic Claude Skills (markdown-based, declarative workflows)
   • Deep dive into Sourcegraph Toolbox (JSON-based, dynamic tool registration)
   • Comparative analysis showing these approaches are complementary
   • Three detailed implementation plans with code examples
   • Decision matrix evaluating 10 criteria
   • Usage scenarios, schemas, and timelines

3. docs/README.md (49 lines) - Navigation guide

   • Quick links for decision makers and implementers
   • Contributing guidelines

Implementation Plans Summary

Plan A: Core Integration into PSAI (Recommended ⭐)

Integrate both Skills and Toolbox as built-in features of the PSAI module.

Key Features:

• Add Get-ClaudeSkill, Import-ClaudeSkill, Get-ToolboxPath, Import-Toolbox functions
• Enhance New-Agent with -Skills and -AutoLoadToolbox parameters
• Include default Skills and Toolboxes directories with examples

Timeline: 6 weeks

Usage Example:

$agent = New-Agent `
    -Skills @('code-review', 'security-audit') `
    -AutoLoadToolbox `
    -Name "CodeReviewer"

$agent | Get-AgentResponse "Review the PowerShell files in ./src"

Rationale:

• Best user experience (single module installation)
• PoC proves seamless integration
• Skills and Toolboxes work better together
• Positions PSAI as complete PowerShell AI solution

Plan B: Separate Companion Modules (Alternative)

Create independent PSAISkills and PSAIToolbox modules that extend PSAI.

Timeline: 8 weeks

Usage Example:

Import-Module PSAI, PSAISkills, PSAIToolbox

$agent = New-AgentWithSkills -Skills @('code-review') `
    | New-AgentWithToolbox -AutoLoadFromEnv

When to Choose:

• If adoption is uncertain and you want to experiment
• If rapid iteration without PSAI core changes is desired
• If community ownership of extensions is preferred
• Can migrate to Plan A later if successful

Plan C: Hybrid Provider Pattern (Future)

Core infrastructure in PSAI with content in provider modules, enabling a plugin ecosystem.

Timeline: 8 weeks

Best For:

• Supporting multiple skill/toolbox formats beyond Anthropic and Sourcegraph
• Building long-term extensible architecture
• Enterprise scenarios requiring custom providers

Key Findings

1. Complementary Technologies: Claude Skills provide high-level workflow guidance while Toolboxes provide low-level executable capabilities. They enhance each other rather than compete.

2. Proven Integration: The PoC demonstrated both approaches integrate naturally with PSAI's existing New-Agent, Register-Tool, and instruction system.

3. Market Opportunity: These capabilities position PSAI as the leading PowerShell AI automation solution.

Technical Details

Claude Skills Pattern

• Markdown files with YAML frontmatter containing metadata
• Natural language instructions optimized for LLM comprehension
• Discovered via directory scanning
• Injected as system instructions via New-Agent -Instructions

Sourcegraph Toolbox Pattern

• JSON tool definitions with parameter schemas
• Discovered via environment variable ($env:PSAI_TOOLBOX_PATH)
• Registered dynamically via Register-Tool
• Integrated via New-Agent -Tools

Impact

• No breaking changes - All additions are documentation only
• No code modifications - PSAI module remains unchanged
• CI/CD safe - Documentation changes ignored by test workflows
• Ready for decision - Complete analysis enables informed choice

Next Steps

1. Review docs/DECISION_SUMMARY.md for quick overview
2. Review docs/CLAUDE_SKILLS_TOOLBOX_ANALYSIS.md for complete details
3. Submit preferred implementation approach (Plan A, B, or C)
4. Implementation begins based on selection

Related

• Anthropic Claude Skills: https://docs.claude.com/en/docs/agents-and-tools/agent-skills/overview
• GitHub Skills Repository: https://github.com/anthropics/skills
• Sourcegraph Toolboxes: https://ampcode.com/manual#toolboxes

---

This analysis provides everything needed to make an informed decision about integrating these powerful capabilities into the PSAI ecosystem. All three plans are viable and include detailed implementation guidance, code examples, schemas, and timelines.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Analysis of Anthropic Claude Skills and Sourcegraph Toolbox approach</issue_title>
<issue_description>I have done separate PoCs for both Anthropic Claude Skills and Sourcegraph Toolbox in PowerShell using this module PSAI and the New-Agent function,

it was pretty straightforward. Doing dir listings to get files. For the Sourcegraph Toolbox, setting the $env var and in my ps1 files seeing if it is set and returning the json from Register-Tool and setting that up via the New-Agent -Tools - worked like a charm

Similar with Anthropic Claude Skills. They open the mardown file and return the frontmatter. I do that and parse it with PowerShell. I set that up with New-Agent -Instrurctions and add more instructions so AI model can tell New-Agent what directory and md file should then be fully loaded in the context - worked like a charm.

Expected Output

Review the the "Reference links" and other internet sources about these offerings from the two companies.

• Review the current implementation of PSAI
• Should the toolbox and skills capbilities be implemented in this module
• Should the toolbox and skills capbilities be implemented in separate modules apart from PSAI

Base on the analysis, create detailed plans of which makes sense. plans for each are fine.

I will review the plans and would like to be able to submit my choice back to AI for the implementation.

Reference links

Anthropic Claude Skills links:

• Agent Skills Overview - https://docs.claude.com/en/docs/agents-and-tools/agent-skills/overview
• GitHub Repository - https://github.com/anthropics/skills

Sourcegraph Toolbox links:

• Toolboxes - https://ampcode.com/manual#toolboxes
• https://ampcode.com/manual/appendix#toolboxes-reference</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #106

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.