-
Notifications
You must be signed in to change notification settings - Fork 1
Closed
Labels
performancePerformance optimizationPerformance optimization
Description
Problem
Agent<P: LlmProvider + Clone + 'static, C, T> and SemanticMemory<P> cause Rust to generate separate machine code for every provider type combination.
Files:
crates/zeph-core/src/agent/mod.rslines 99-131crates/zeph-memory/src/semantic.rslines 58-116
Impact
- Binary size: +200-500KB per monomorphization
- Compile time: +5-10 seconds incremental rebuilds when changing agent code
- Instruction cache: Multiple copies reduce CPU cache efficiency
Current usage
- Agent only calls
P::embed()andP::chat() - SemanticMemory only calls
P::embed()
Solution
Replace generic P with dynamic dispatch:
Short-term:
pub struct Agent<C: Channel, T: ToolExecutor> {
provider: Arc<AnyProvider>, // Arc makes clone cheap
}Long-term:
trait EmbedProvider: Send + Sync {
async fn embed(&self, text: &str) -> Result<Vec<f32>, LlmError>;
}
pub struct SemanticMemory {
embedder: Arc<dyn EmbedProvider>,
}Priority: P2
Effort: Large (1-2 days, requires careful refactoring)
Related to #391
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
performancePerformance optimizationPerformance optimization