Epic: Compaction Update

[riatzukiza]

Summary

Compaction currently loses important conversation context and instructions, producing regressions like memory loss after compaction, rules not being followed, and poor continuation in BUILD mode. This epic addresses boundary selection, rules preservation, summary quality, pruning, and observability so conversations remain coherent and safe after compaction.

Problem Statement

• Boundary detection uses first prior assistant summary as a hard cut (message-v2.ts:672-680), which can discard too much or wrong context and makes the model forget the current task.
• Rules and user constraints (e.g. no git push) are not reliably preserved in the conversational context after compaction.
• The current summary prompt is too light to act as a handoff; it lacks detailed structure for continuing work (files, errors/fixes, pending tasks, last action).
• Pruning is underutilized and protects the exact region where most tokens accrue (tool calls between last two user turns), leading to premature full compaction instead of selective deletion.
• Minimal observability: hard to see which messages/parts are included, and token usage breakdown for tool calls.

Related Issues

• compact causes llm to forget convo anomalyco/opencode#2234 compact causes LLM to forget conversation (memory loss after compact)
• Session automatically compacted, destroying the entire working context anomalyco/opencode#2945 Session automatically compacted, destroys working context
• Model in BUILD mode does not have enough context to continue after compaction anomalyco/opencode#3031 BUILD mode lacks context after compaction (needs handoff)
• Soft compaction / AI global workspace metabolism anomalyco/opencode#3032 Soft compaction / better metabolism (selective removal, keep recent context)
• Agent no follow rules after compact session anomalyco/opencode#3099 Rules not followed after compact (AGENTS.md and constraints)
• TypeError: undefined is not an object (evaluating 'toSummarize.at(-1).info') anomalyco/opencode#2559 Edge case empty slice during summarize (robustness)
• BUG: @ai-sdk/openai 2.0.50 breaks OpenAI OAuth plugin requiring store: false (missing providerOptions in compaction) anomalyco/opencode#3118 ProviderOptions missing in compaction path (token/usage)
• I can not configure the limit for litellm anomalyco/opencode#2787 includeUsage needed for openai-compatible (usage accounting)

Goals

• Preserve critical context for seamless continuation in BUILD mode.
• Keep user rules/constraints alive in the conversation post-compaction.
• Reduce token pressure via smarter pruning before resorting to full compaction.
• Improve transparency (what was kept/dropped and why).

Non‑Goals

• UI redesign of the session view.
• Provider-specific tuning beyond ensuring usage accounting and stable headers/options.

Plan / Tasks

• Boundary: replace filterCompacted with smarter selection that preserves the last N turns intact (configurable), includes the last assistant message, and then compacts earlier history only.
  • File: packages/opencode/src/session/message-v2.ts:672-680
  • Add defensive check that all included messages have info.sessionID === input.sessionID (belt-and-suspenders).
  • Emit debug logs: selected message IDs, last summary index, preserved turn count.
• Rules Preservation: ensure user constraints survive compaction.
  • Minimal: require the summary to explicitly include a "Rules & Constraints" section capturing active directives (AGENTS.md, permission-sensitive rules, user instructions).
  • Stretch: introduce a pinned RulesPart to persist constraints across summaries (design spike; may land later).
• Summary Prompt: upgrade to a detailed handoff format (primary intent, files touched, code snippets, errors/fixes, pending tasks, current work, rules, optional next step). Use a compact but structured template.
  • File: packages/opencode/src/session/compaction.ts (user prompt around L169-L178)
  • Ensure consistency with SystemPrompt.summarize and custom instructions.
• Pruning Improvements: make pruning more effective before full compaction.
  • Revisit PRUNE_PROTECT/PRUNE_MINIMUM and turns-protection strategy; optionally configurable.
  • Prefer deleting old tool outputs earlier; consider a soft-compaction approach that drops tool call outputs selectively.
  • Add /status or log output that shows tokens by category (user/assistant/tool/reasoning) to guide tuning.
• Provider Options & Usage
  • Ensure providerOptions (headers, includeUsage) flow in compaction path (see BUG: @ai-sdk/openai 2.0.50 breaks OpenAI OAuth plugin requiring store: false (missing providerOptions in compaction) anomalyco/opencode#3118, I can not configure the limit for litellm anomalyco/opencode#2787).
  • Verify accurate usage accounting during compaction and pruning.
• Tests
  • Unit: boundary selection (empty slice, multiple summaries, preserve last N turns).
  • Unit: rules preservation (rules section present in summary).
  • Unit: pruning selection logic.
  • Integration: BUILD-mode continuation across compaction retains current task context and rules.
• Telemetry/Logging
  • Log token inputs for compaction, before/after counts.
  • Log pruning decisions and compaction boundaries.

Definition of Done

• After compaction, the assistant reliably continues the task in BUILD mode without re-asking for context.
• User rules/constraints remain in force and visible in the conversation context.
• Selective pruning reduces token usage without harming recency.
• Added tests pass and prevent regressions in boundary selection and rules persistence.
• Logs clearly show messages kept/dropped and token breakdown.

Acceptance Tests

• 8+ hour session with dozens of compactions retains rules (e.g., no git push without permission) and continues coding flows.
• Last N turns are preserved verbatim; summary contains a clear handoff and rules.
• No cross-session contamination; compaction operates strictly within sessionID.

[riatzukiza]

Related Issues & Context

This Compaction Update epic is fundamentally related to the Subagent State Handoff epic (#56) because both address the same core architectural problem: context boundary fragility.

Direct Relationship:

• Compaction failures (Compaction: Preserve rules/constraints across compaction #48, Compaction: Structured BUILD-mode handoff summary prompt #49, Compaction: Smarter boundary selection and cross-session guard #47, Compaction: Pruning/soft compaction and selective tool-output deletion #50, Compaction: Telemetry and /status for context + decisions #52) = Context transfer within same agent across time
• Subagent failures (link to sst/opencode issues Subagent results not always properly handled anomalyco/opencode#3173, Feature request: let subagents inherit context anomalyco/opencode#2588, Sub-agent's request for permission is not shown in the primary agent's view anomalyco/opencode#4078, Feature: Subagents should support auto compaction anomalyco/opencode#3153) = Context transfer between different agents

Shared Problems:

1. State Loss: Rules/constraints lost during compaction (Compaction: Preserve rules/constraints across compaction #48) mirrors rules lost in subagent delegation
2. Poor Handoff: Inadequate summaries for BUILD mode (Compaction: Structured BUILD-mode handoff summary prompt #49) mirrors incomplete task context in subagents
3. Boundary Selection: Wrong context preserved (Compaction: Smarter boundary selection and cross-session guard #47) mirrors subagent not inheriting proper context
4. Observability: No visibility into compaction decisions (Compaction: Telemetry and /status for context + decisions #52) mirrors no visibility into subagent failures

Implementation Synergy:

• Rules Preservation (Compaction: Preserve rules/constraints across compaction #48) and Context Inheritance in subagents solve the same problem differently
• Handoff Documents (Compaction: Structured BUILD-mode handoff summary prompt #49) work for both compaction recovery AND subagent state handoff
• Telemetry (Compaction: Telemetry and /status for context + decisions #52) benefits both compaction debugging AND subagent monitoring

Solving compaction issues will provide the foundation for solving subagent state handoff, as both are fundamentally about reliable context transfer across boundaries.