Skip to content

Add real-time event handling and streaming support #10157

@Mossaka

Description

@Mossaka

Description

Implement real-time event streaming from Copilot SDK to enable progress updates, intermediate results, and reactive workflow patterns.

Part of Epic

#10154 - Copilot SDK Integration for Advanced Agentic Workflows

Problem Statement

CLI Limitation:

  • Must wait for full CLI execution to complete
  • No intermediate progress updates
  • Cannot react to tool calls in real-time
  • All-or-nothing execution model

SDK Opportunity:

session.on((event) => {
  switch (event.type) {
    case "assistant.message":
      // Stream response as it generates
      console.log(event.data.content);
      break;
    case "tool.call":
      // Log tool usage in real-time
      logToolUsage(event.data.tool, event.data.args);
      break;
    case "session.idle":
      // React to completion
      processComplete();
      break;
  }
});

Use Cases

  1. Progress Indicators - Show real-time progress in GitHub Actions logs
  2. Tool Validation - Intercept and validate tool calls before execution
  3. Streaming Responses - Display partial results as they generate
  4. Error Detection - React to errors immediately, not after full completion
  5. Cost Monitoring - Track token usage in real-time with budget alerts

Event Types to Support

1. Core Events

  • session.started - Session initialization complete
  • assistant.message - Model generating response
  • assistant.complete - Model finished response
  • tool.call - Model requesting tool execution
  • tool.result - Tool execution complete
  • session.idle - Session waiting for input
  • session.error - Error occurred

2. Metrics Events

  • tokens.used - Token usage update
  • cost.update - Cost accumulation update
  • turn.complete - Conversation turn finished

3. Control Events

  • approval.required - Human approval needed
  • timeout.warning - Approaching timeout
  • budget.warning - Approaching budget limit

Design Considerations

Event Handler Registration

engine:
  id: copilot
  mode: sdk
  streaming:
    enabled: true
    handlers:
      - event: tool.call
        action: log_and_validate
      - event: cost.update
        action: check_budget_limit
      - event: assistant.message
        action: stream_to_output

GitHub Actions Integration

Challenge: GitHub Actions has limited real-time output streaming

Solutions:

  1. Grouped Output - Buffer events and emit in batches
  2. Annotations - Use workflow annotations for important events
  3. Summary - Real-time summary table updated as events occur
  4. Artifacts - Write event stream to artifact for analysis

Implementation Pattern

// SDK event handler in action
class WorkflowEventHandler {
  constructor(private context: ActionContext) {}

  async handleEvent(event: SDKEvent) {
    switch (event.type) {
      case "assistant.message":
        await this.streamToOutput(event.data.content);
        break;
      case "tool.call":
        await this.logToolUsage(event.data);
        const approved = await this.validateTool(event.data);
        if (!approved) {
          throw new Error("Tool call rejected by validation");
        }
        break;
      case "cost.update":
        await this.checkBudget(event.data.cost);
        break;
    }
  }
}

Implementation Tasks

  • Design event handler architecture
  • Implement event streaming in SDK engine
  • Add GitHub Actions-compatible output formatting
  • Create event handler registry
  • Implement built-in handlers (logging, validation, budgeting)
  • Add event filtering and routing
  • Create event stream artifact for analysis
  • Update documentation with event handling patterns

Example Handlers

1. Budget Monitor:

async function checkBudget(event: CostUpdateEvent) {
  if (event.data.cost > process.env.MAX_COST) {
    await session.send({
      prompt: "Budget limit reached. Summarize findings and stop."
    });
    await session.destroy();
  }
}

2. Tool Validator:

async function validateToolCall(event: ToolCallEvent) {
  const { tool, args } = event.data;
  
  if (tool === "bash" && isDestructiveCommand(args.command)) {
    core.warning(`Blocked destructive command: ${args.command}`);
    return { approved: false, reason: "Destructive command blocked" };
  }
  
  return { approved: true };
}

3. Progress Reporter:

async function reportProgress(event: AssistantMessageEvent) {
  const summary = await core.summary
    .addHeading("Workflow Progress")
    .addTable([
      ["Turn", session.metrics.turns],
      ["Tokens", session.metrics.tokens.total],
      ["Cost", `$${session.metrics.cost.toFixed(4)}`]
    ])
    .write();
}

Success Criteria

  • Event streaming working in SDK engine
  • At least 5 built-in event handlers
  • GitHub Actions log integration
  • Real-time summary updates
  • Event filtering and routing
  • Documentation with examples
  • Integration tests for event handling

Performance Considerations

  • Event handler execution should not block SDK
  • Batch events to reduce GitHub Actions API calls
  • Event stream artifacts should be compressed
  • Consider event sampling for high-volume streams

References

  • SDK event documentation
  • GitHub Actions output and annotations API
  • Real-time progress patterns

Priority: Medium - Enhances observability
Estimated Effort: 5-7 days
Dependencies: #10155 (POC), #10156 (session design)
Skills Required: TypeScript, event-driven architecture, GitHub Actions

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions