feat: integrate structured output into core workflows#563
Merged
Conversation
github-actions
bot
added
llm
bug-ops
force-pushed
the
feat/epic-549-structured-output
branch
2 times, most recently
from
bd269de to
e59516e
Compare
Add chat_typed/Extractor-based structured output to three subsystems: - Model selection (#551): ModelSelection struct in orchestrator with opt-in LLM-assisted routing and rule-based fallback - Structured summarization (#552): StructuredSummary with key_facts stored as individual Qdrant points for granular retrieval - Self-learning evaluation (#553): SkillEvaluation gate before skill improvement to avoid unnecessary LLM calls
Cover all untested paths from epic #549: - try_llm_routing with llm_routing enabled (valid/unknown/error paths) - summarize chat_typed fallback to plain text - SkillEvaluation gate (should_improve false/true/parse error) - check_improvement_allowed (cooldown, min_failures, threshold) - attempt_self_reflection early returns - ModelSelection/SkillEvaluation deserialization edge cases - proptest for classify, estimate_tokens, chrono_parse, build_evaluation_prompt
bug-ops
force-pushed
the
feat/epic-549-structured-output
branch
from
e59516e to
97648d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integrates
chat_typed/Extractorstructured output (from M27) into three core workflows.ModelSelectionstruct in orchestrator with opt-in LLM-assisted routing, rule-based fallback when disabled or on failureStructuredSummarywithkey_factsandentitiesstored as individual Qdrant points for granular semantic retrievalSkillEvaluationgate before skill improvement — skips unnecessary LLM rewrite calls whenshould_improve=falseCloses #549, closes #551, closes #552, closes #553
Changes
crates/zeph-skills/src/evolution.rs—SkillEvaluationstruct +build_evaluation_promptcrates/zeph-core/src/agent/learning.rs— evaluation step beforecall_improvement_llmcrates/zeph-llm/src/orchestrator/classifier.rs—ModelSelectionstructcrates/zeph-llm/src/orchestrator/mod.rs—try_llm_routing,with_llm_routingonModelOrchestratorcrates/zeph-memory/src/semantic.rs—StructuredSummary,store_key_facts,search_key_factscrates/zeph-memory/Cargo.toml— addedschemarsdependencyTest plan