🤖 Feature: Agent Discovery & Orchestration Interface

[manavgup]

🤖 Feature: Agent Discovery & Orchestration Interface

📋 Overview

Create an intuitive interface for discovering, configuring, and orchestrating AI agents within the RAG system. Users should be able to browse available agents, understand their capabilities, configure parameters, and create multi-agent workflows through a visual interface.

🎯 User Story

As a user
I want to discover and configure AI agents for my specific tasks
So that I can automate complex document analysis workflows without technical expertise

✅ Acceptance Criteria

Agent Discovery & Catalog

• Agent Marketplace: Browse available agents organized by categories (Analysis, Summarization, Research, etc.)
• Capability Descriptions: Clear explanations of what each agent does with use case examples
• Search & Filter: Find agents by functionality, document type, or use case
• Agent Ratings: Display usage statistics and user ratings for each agent
• Favorites System: Save frequently used agents for quick access
• Recommendations: AI-powered agent suggestions based on user behavior and context

Agent Configuration Interface

• Dynamic Forms: Auto-generated configuration forms based on agent schemas
• Parameter Validation: Real-time validation with helpful error messages
• Preview Mode: Preview agent output with sample data before execution
• Configuration Templates: Pre-built configurations for common use cases
• Advanced Options: Collapsible advanced settings for power users
• Version Control: Save, load, and rollback configuration changes

Contextual Agent Suggestions

• Smart Recommendations: Suggest relevant agents during chat conversations
• Context Awareness: Recommendations based on current document types and user intent
• Inline Integration: Seamlessly invoke agents from chat interface
• Explanation System: Clear reasoning for why specific agents are recommended
• Learning System: Improve recommendations based on user acceptance/rejection

Agent Execution Monitoring

• Real-time Progress: Live progress indicators for agent execution
• Detailed Logs: Comprehensive execution logs and intermediate results
• Performance Metrics: Execution time, resource usage, and success rates
• Control Actions: Pause, resume, cancel, or retry agent execution
• Error Handling: Clear error messages with suggested resolutions
• Historical Data: Track agent performance and usage patterns over time

🏗️ Technical Implementation

Component Architecture

// Main agent management interface
<AgentOrchestrator>
  <AgentCatalog 
    agents={availableAgents}
    onSelectAgent={handleAgentSelect}
    filters={activeFilters}
  />
  <AgentConfigPanel 
    selectedAgent={selectedAgent}
    configuration={agentConfig}
    onConfigChange={handleConfigChange}
  />
  <AgentExecutionMonitor 
    runningJobs={activeJobs}
    onJobAction={handleJobAction}
  />
</AgentOrchestrator>

Key Components

Agent Catalog

• AgentCard.jsx: Individual agent display with capabilities and ratings
• AgentCatalog.jsx: Grid/list view of available agents with search
• AgentFilter.jsx: Category and capability filtering controls
• AgentDetails.jsx: Detailed agent information modal
• AgentRecommendations.jsx: AI-powered agent suggestions

Configuration System

• AgentConfigPanel.jsx: Main configuration interface
• DynamicForm.jsx: Auto-generated forms from agent schemas
• ConfigPresets.jsx: Template management for common configurations
• ParameterInput.jsx: Specialized inputs for different parameter types
• ConfigValidator.jsx: Real-time validation and error display

Execution Monitoring

• ExecutionDashboard.jsx: Overview of all running and completed jobs
• JobMonitor.jsx: Individual job progress and control
• ExecutionLogs.jsx: Detailed logging and intermediate results
• PerformanceMetrics.jsx: Agent performance analytics
• ErrorHandler.jsx: Error display and recovery options

State Management

// Agent context for global agent state
const AgentContext = {
  availableAgents: [],
  selectedAgent: null,
  agentConfig: {},
  executionJobs: [],
  recommendations: [],
  filters: {
    category: 'all',
    capabilities: [],
    rating: 0
  }
}

Agent Schema System

// Example agent schema for dynamic form generation
const agentSchema = {
  id: 'document-summarizer',
  name: 'Document Summarizer',
  description: 'Generates concise summaries of document content',
  category: 'Analysis',
  parameters: {
    maxLength: {
      type: 'number',
      default: 250,
      min: 50,
      max: 1000,
      description: 'Maximum summary length in words'
    },
    style: {
      type: 'select',
      options: ['bullet-points', 'paragraph', 'executive'],
      default: 'paragraph',
      description: 'Summary format style'
    },
    includeKeywords: {
      type: 'boolean',
      default: true,
      description: 'Include key terms and phrases'
    }
  }
}

🎨 Design Specifications

Visual Design

• Agent Cards: Consistent card design with IBM Carbon styling
• Icons & Indicators: Clear visual indicators for agent status and capabilities
• Color Coding: Category-based color coding for easy agent identification
• Progress Visualization: Animated progress bars and status indicators
• Responsive Layout: Adaptive layout for different screen sizes

User Experience

• Intuitive Navigation: Easy browsing and discovery of agent capabilities
• Contextual Help: Tooltips and help text for complex configuration options
• Smart Defaults: Sensible default configurations to reduce setup time
• Quick Actions: Common actions accessible with minimal clicks
• Search Excellence: Fast, accurate search with auto-complete suggestions

📱 Responsive Design

Desktop Layout (1024px+)

• Three-column layout: Catalog | Configuration | Monitoring
• Full-featured interface: All configuration options visible
• Advanced workflows: Multi-agent orchestration tools
• Detailed analytics: Comprehensive performance dashboards

Tablet Layout (768-1024px)

• Two-column layout: Collapsible panels for space efficiency
• Touch-optimized controls: Larger touch targets and gestures
• Simplified configuration: Progressive disclosure of advanced options
• Essential monitoring: Key metrics and controls prominently displayed

Mobile Layout (320-768px)

• Single-column layout: Full-screen panels with navigation
• Simplified workflows: Focus on essential agent operations
• Swipe navigation: Gesture-based interface navigation
• Condensed information: Prioritized content display

🔌 API Integration

Backend Endpoints

// Agent management API endpoints
GET    /api/agents                     // List available agents
GET    /api/agents/:id                 // Get agent details and schema
POST   /api/agents/:id/execute         // Execute agent with configuration
GET    /api/jobs                       // List execution jobs
GET    /api/jobs/:id                   // Get job status and results
DELETE /api/jobs/:id                   // Cancel running job
GET    /api/agents/recommendations     // Get personalized recommendations

Agent Execution Flow

// Example agent execution
const executeAgent = async (agentId, config, collectionId) => {
  const job = await agentService.execute(agentId, {
    configuration: config,
    inputs: { collectionId },
    callback_url: '/api/jobs/callback'
  });
  
  // Monitor job progress
  const monitor = new JobMonitor(job.id);
  monitor.onProgress(updateProgressUI);
  monitor.onComplete(handleCompletion);
  monitor.onError(handleError);
  
  return job;
};

🧪 Testing Strategy

Unit Tests

• Agent catalog filtering and search functionality
• Dynamic form generation from schemas
• Configuration validation logic
• Job monitoring and status updates
• Recommendation engine accuracy

Integration Tests

• End-to-end agent discovery and execution flow
• Real-time job monitoring and control
• Configuration persistence and loading
• Error handling and recovery scenarios
• Cross-browser compatibility testing

User Experience Tests

• Agent discovery and selection workflows
• Configuration ease-of-use for non-technical users
• Mobile interaction and touch responsiveness
• Accessibility compliance with screen readers
• Performance with large numbers of agents and jobs

📊 Success Metrics

Discovery & Adoption

• Agent Discovery Rate: >70% of users explore agent catalog within first week
• Configuration Success: >85% of users successfully configure agents on first attempt
• Agent Utilization: Average user executes 3+ different agents per week
• Recommendation Accuracy: >80% of suggested agents are accepted and used

User Experience

• Task Completion Time: <5 minutes to discover, configure, and execute agent
• Error Rate: <5% of agent executions fail due to configuration errors
• User Satisfaction: >4.5/5 rating for agent management experience
• Return Usage: >75% of users return to use agents within one week

Performance Metrics

• Catalog Load Time: <2 seconds to display full agent catalog
• Configuration Response: <100ms for form updates and validation
• Execution Feedback: <500ms from user action to UI feedback
• Job Monitoring: Real-time updates with <1 second latency

🚧 Implementation Phases

Phase 1: Agent Discovery (Week 1-2)

• Basic agent catalog with search and filtering
• Agent detail views and capability descriptions
• Responsive layout and mobile optimization
• Integration with backend agent registry

Phase 2: Configuration System (Week 3-4)

• Dynamic form generation from agent schemas
• Parameter validation and error handling
• Configuration templates and presets
• Preview mode for configuration testing

Phase 3: Execution Monitoring (Week 5-6)

• Real-time job monitoring dashboard
• Progress tracking and intermediate results
• Job control actions (pause, cancel, retry)
• Performance analytics and historical data

Phase 4: Smart Recommendations (Week 7-8)

• Context-aware agent suggestions
• Integration with chat interface for inline recommendations
• Learning system for improving suggestions
• A/B testing framework for recommendation algorithms

🔗 Dependencies

Internal Dependencies

• Backend Agent Registry: Catalog of available agents and their schemas
• Job Execution Engine: System for running and monitoring agent jobs
• User Analytics: Data for generating personalized recommendations
• Chat Interface: Integration point for contextual agent suggestions

External Libraries

{
  "react-hook-form": "^7.45.0",
  "yup": "^1.2.0",
  "react-select": "^5.7.0",
  "react-virtualized": "^9.22.0",
  "lodash": "^4.17.21"
}

🎯 Definition of Done

• All acceptance criteria implemented and tested
• Agent catalog loads and displays correctly across all devices
• Dynamic configuration forms generate properly from schemas
• Real-time job monitoring works reliably
• Recommendation system provides relevant suggestions
• Performance meets specified benchmarks
• Accessibility compliance (WCAG 2.1 AA)
• Comprehensive test coverage (>90%)
• Code review completed and approved
• User testing feedback incorporated
• Integration testing with backend services completed

---

Estimated Effort: 8 weeks
Story Points: 34 points
Priority: High (enables core agentic capabilities)
Risk: High (complex state management, real-time monitoring)

This feature enables the core agentic AI capabilities that distinguish the platform from traditional RAG solutions, providing users with powerful automation tools through an intuitive, no-code interface.

[manavgup]

@claude how do we accomplish this with MCP protocol?

[claude]

Claude finished @manavgup's task —— View job

---

MCP Protocol Architecture for Agent Orchestration

Current State Analysis

Existing Agent Infrastructure:

• Frontend agent context exists (frontend/src/contexts/AgentContext.tsx:1)
• Mock agent types: research, synthesis, validation, planner
• Basic orchestration interfaces (sessions, tasks, agents)
• IBM MCP Context Forge mentioned as planned technology

Current MCP Status:

• No Model Context Protocol implementation exists yet
• MCP mentioned as recommended orchestration framework
• Ready for implementation

---

How MCP Protocol Enables Agent Orchestration

The Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to AI models. Here's how it maps to your agent orchestration requirements:

1. Agent Discovery & Catalog (MCP Resources)

MCP uses Resources to expose discoverable, contextual data:

# Backend: MCP Server Implementation
# backend/rag_solution/mcp/server.py

from mcp.server import Server
from mcp.types import Resource

class AgentMCPServer:
    def __init__(self):
        self.server = Server("rag-agent-orchestrator")
        
    @self.server.list_resources()
    async def list_agents() -> list[Resource]:
        """Expose available agents as MCP resources"""
        return [
            Resource(
                uri="agent://research-agent",
                name="Research Agent",
                description="Document research and information gathering",
                mimeType="application/json",
                metadata={
                    "capabilities": ["document_search", "information_extraction"],
                    "category": "Analysis",
                    "version": "1.0.0"
                }
            ),
            # ... more agents
        ]

Benefits:

• ✅ Automatic Discovery: Clients query available agents via MCP
• ✅ Standardized Metadata: Agent capabilities in consistent format
• ✅ Dynamic Updates: Agent catalog updates without client changes

2. Agent Configuration (MCP Prompts)

MCP Prompts provide reusable, parameterized templates:

from mcp.types import Prompt, PromptArgument

@self.server.list_prompts()
async def list_agent_configs() -> list[Prompt]:
    return [
        Prompt(
            name="configure-research-agent",
            description="Configure research agent parameters",
            arguments=[
                PromptArgument(
                    name="max_results",
                    description="Maximum search results",
                    required=False,
                    default=10
                ),
                PromptArgument(
                    name="confidence_threshold",
                    description="Minimum confidence score",
                    required=False,
                    default=0.8
                )
            ]
        )
    ]

Benefits:

• ✅ Schema Validation: MCP handles parameter validation
• ✅ Dynamic Forms: Frontend auto-generates UI from prompt schemas
• ✅ Configuration Reuse: Templates enable preset configurations

3. Agent Execution (MCP Tools)

MCP Tools enable agents to perform actions:

from mcp.types import Tool

@self.server.list_tools()
async def list_agent_tools() -> list[Tool]:
    return [
        Tool(
            name="execute_research_agent",
            description="Execute research agent with configuration",
            inputSchema={
                "type": "object",
                "properties": {
                    "query": {"type": "string"},
                    "collection_id": {"type": "string"},
                    "config": {"type": "object"}
                },
                "required": ["query", "collection_id"]
            }
        ),
        Tool(
            name="orchestrate_multi_agent",
            description="Coordinate multiple agents",
            inputSchema={
                "type": "object",
                "properties": {
                    "agents": {"type": "array", "items": {"type": "string"}},
                    "workflow": {"type": "string", "enum": ["sequential", "parallel"]}
                }
            }
        )
    ]

Benefits:

• ✅ Standardized Execution: Consistent interface for all operations
• ✅ Type Safety: JSON Schema validation
• ✅ Composability: Tools chain together for complex workflows

4. Real-time Monitoring (MCP Sampling)

from mcp.types import SamplingMessage

@self.server.create_message()
async def stream_agent_progress(session_id: str):
    """Stream real-time agent execution progress"""
    async for update in orchestration_service.monitor_session(session_id):
        yield SamplingMessage(
            role="assistant",
            content={
                "type": "progress",
                "session_id": session_id,
                "status": update.status,
                "progress": update.progress_percentage,
                "logs": update.logs
            }
        )

5. Context-Aware Recommendations

@self.server.read_resource()
async def get_agent_recommendations(uri: str) -> str:
    """Provide agent recommendations based on context"""
    recommendations = await recommendation_service.suggest_agents(
        user_id=context["user_id"],
        collection_id=context.get("collection_id"),
        conversation_history=context.get("conversation")
    )
    return json.dumps(recommendations)

---

Implementation Architecture

Backend Structure

backend/rag_solution/
├── mcp/
│   ├── server.py           # MCP server setup
│   ├── resources.py        # Agent discovery
│   ├── prompts.py          # Configuration templates
│   ├── tools.py            # Agent execution
│   └── sampling.py         # Real-time monitoring
├── services/
│   ├── agent_service.py
│   ├── orchestration_service.py
│   └── recommendation_service.py
└── router/
    └── agent_router.py

Frontend Integration

// frontend/src/services/mcpClient.ts
import { Client } from '@modelcontextprotocol/sdk/client/index.js';

class AgentMCPClient {
  async discoverAgents() {
    const resources = await this.client.listResources();
    return resources.resources.filter(r => r.uri.startsWith('agent://'));
  }

  async executeAgent(agentId: string, config: any) {
    return await this.client.callTool({
      name: `execute_${agentId}`,
      arguments: config
    });
  }

  async monitorExecution(sessionId: string, callback: (update) => void) {
    this.client.onSamplingMessage((message) => {
      if (message.content.session_id === sessionId) {
        callback(message.content);
      }
    });
  }
}

---

Key Advantages of MCP

1. Standardized Protocol: Industry-standard for AI context
2. Self-Describing: Agents expose their own schemas
3. Type Safety: Built-in JSON Schema validation
4. Real-time Updates: Native streaming support
5. Context-Aware: Designed for contextual AI
6. Interoperability: Works with any MCP client
7. Extensibility: Easy to add new agents

Implementation Roadmap

Phase 1: MCP Server (Week 1)

• Install MCP SDK: pip install mcp
• Implement basic server with agent resources

Phase 2: Agent Discovery (Week 2)

• MCP resources for agent catalog
• Frontend MCP client integration

Phase 3: Configuration (Week 3)

• MCP prompts for configuration
• Auto-generated forms from schemas

Phase 4: Execution (Week 4-5)

• MCP tools for execution
• Real-time monitoring via sampling

Phase 5: Recommendations (Week 6)

• Context-aware suggestions
• Conversation integration

Next Steps

1. Install MCP SDK: poetry add mcp
2. Create backend/rag_solution/mcp/server.py
3. Expose agents as MCP resources
4. Add frontend MCP client
5. Update AgentContext to use MCP

Would you like me to start implementing the MCP server infrastructure?

---

Branch