Skip to content

perf: reduce monomorphization bloat from provider generic P #407

@bug-ops

Description

@bug-ops

Problem

Agent<P: LlmProvider + Clone + 'static, C, T> and SemanticMemory<P> cause Rust to generate separate machine code for every provider type combination.

Files:

  • crates/zeph-core/src/agent/mod.rs lines 99-131
  • crates/zeph-memory/src/semantic.rs lines 58-116

Impact

  • Binary size: +200-500KB per monomorphization
  • Compile time: +5-10 seconds incremental rebuilds when changing agent code
  • Instruction cache: Multiple copies reduce CPU cache efficiency

Current usage

  • Agent only calls P::embed() and P::chat()
  • SemanticMemory only calls P::embed()

Solution

Replace generic P with dynamic dispatch:

Short-term:

pub struct Agent<C: Channel, T: ToolExecutor> {
    provider: Arc<AnyProvider>,  // Arc makes clone cheap
}

Long-term:

trait EmbedProvider: Send + Sync {
    async fn embed(&self, text: &str) -> Result<Vec<f32>, LlmError>;
}

pub struct SemanticMemory {
    embedder: Arc<dyn EmbedProvider>,
}

Priority: P2
Effort: Large (1-2 days, requires careful refactoring)
Related to #391

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance optimization

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions