Design session state management for SDK workflows

[Mossaka]

Description

Design how SDK session state will be managed in GitHub Actions workflows, including persistence across job boundaries, state serialization, and multi-turn conversation support.

Part of Epic

#10154 - Copilot SDK Integration for Advanced Agentic Workflows

Problem Statement

CLI Limitation:

• Each CLI invocation is independent
• No programmatic access to conversation history
• State reconstruction requires parsing log files
• Cannot pause/resume sessions

SDK Opportunity:

• Persistent sessions with full state control
• Programmatic access to conversation history
• Ability to pause/resume sessions
• State serialization for cross-job persistence

Design Considerations

1. State Storage Options

Option A: In-Memory (Single Job)

• Pros: Fast, simple
• Cons: Lost on job completion, no cross-job support

Option B: File-Based Artifacts

• Pros: Persists across jobs, human-readable
• Cons: Serialization overhead, size limits

Option C: GitHub Cache

• Pros: Fast, persistent, restore keys
• Cons: Cache limits, eventual eviction

Option D: Hybrid (Memory + Artifacts)

• Pros: Best of both worlds
• Cons: More complex implementation

2. State Structure

interface SessionState {
  id: string;
  created: timestamp;
  model: string;
  systemPrompt?: string;
  conversationHistory: Message[];
  tools: ToolDefinition[];
  metrics: {
    tokens: { input: number; output: number };
    cost: number;
    turns: number;
  };
  metadata: Record<string, unknown>;
}

3. Workflow Integration

engine:
  id: copilot
  mode: sdk
  session:
    persistent: true          # Enable state persistence
    storage: artifacts        # or: memory, cache, hybrid
    max-turns: 10            # Limit conversation length
    restore-from: previous   # or: specific-run-id

Key Decisions Needed

1. Default storage mechanism - Which option should be the default?
2. State size limits - How much history to retain?
3. Cross-job restore - How to reference previous sessions?
4. Security - How to protect sensitive data in state?
5. Cleanup - When to expire old session state?

Success Criteria

• Design document with detailed state management approach
• API specification for state operations
• Security considerations documented
• Performance impact analysis
• Migration path for existing workflows

Implementation Guidance

New Files:

• pkg/workflow/copilot_sdk_session.go - Session management
• pkg/workflow/copilot_sdk_state.go - State serialization
• actions/copilot-sdk-agent/session-manager.ts - Runtime state

Frontmatter Extensions:

engine:
  session:
    persistent: boolean
    storage: "memory" | "artifacts" | "cache" | "hybrid"
    max-turns: number
    max-history-size: string  # e.g., "10MB"
    restore-from: string      # run ID or "previous"
    cleanup-after: string     # e.g., "7d"

References

• Session management best practices
• GitHub Actions artifacts API
• GitHub Actions cache API

---

Priority: High - Critical for multi-turn workflows
Estimated Effort: 5-7 days
Dependencies: #10155 (POC must validate approach)
Skills Required: System design, Go, GitHub Actions