Soft compaction / AI global workspace metabolism

[agladysh]

Current /compact replaces entire context window with a brief summary.

It is often not desirable, besides obvious working context loss, AI assistant "personality" gets lost, and has to be whipped into order and fed nuances again.

Most of the context is consumed by tool calls, often unsuccessful or no longer relevant.

Offer a soft compaction operation. When activated:

1/ Mark tool calls with identifiers, clearly visible to the model. Leave most recent 10-15% of conversation by message count (will require tuning) unmarked.
2/ Offer model to return the list of identifiers to be deleted from the context, and produce a soft compaction message to preserve the context.
3/ Delete the tool calls with the identifiers the model selected.
4/ Insert the soft compaction message at the end of the conversation data.

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• Model in BUILD mode does not have enough context to continue after compaction #3031: Model in BUILD mode does not have enough context to continue after compaction - requests better handoff documentation and context preservation after compaction
• [Feature Request] Configuration to Limit Input Tokens per Request #2637: Configuration to Limit Input Tokens per Request - proposes maxInputTokens setting to control context sent to API
• [Feature Request] Add User Controls for Context Management #1990: Add User Controls for Context Management - requests /drop, /list, /tokens commands for fine-grained context control
• Feature request: let subagents inherit context #2588: Feature request: let subagents inherit context - proposes context forking mechanisms
• feature: Context editing #1014: Context editing - requests ability to manually edit conversation context

These issues all address similar concerns about intelligent context window management, selective content removal, and preserving working context while managing token usage.

Feel free to ignore if none of these address your specific case.

[agladysh]

Looks like a very similar automated functionality already exists: prune().

I have not seen its effects, and I don't think user and model messages consume all context in my conversations.

Perhaps display details about how much of the context window is taken by the tool calls, either near on the main screen, or in /status command, so this is easier to discern and tune?

[agladysh]

For many larger tasks I chat with the agent, and then it codes for a long while without any user input. This often consumes most of the context window.

The model (at least Grok Code Fast 1) seems to be aware one way or another of how much context window is left, and often prematurely stops the work and/or takes bad shortcuts in order to avoid running out. Full conversation compaction is often not triggered, model stops at 70-80%.

It seems that pruning is not triggered as well. If I understand the code correctly, it protects the context between the two last user calls — exactly where almost all the tool calls are. Which means pruning is useless — at least in my scenarios.

Triggering full compaction is not desirable, I would prefer pruning, even if manual.

Ideally, pruning would be smarter (perhaps as I described in the original issue text or similar), and would NOT protect last user messages.

[agladysh]

Workaround:

We need to trigger the prune function of opencode. Reply "ok"

[rekram1-node]

ik @thdxr has played around with different compact strategies

assigning to him

[anuramat]

• avante.nvim does exactly this with delete_tool_use_messages and remove_file_from_context; imho "leave most recent x% unmarked" part sounds like a bad idea in terms of caching
• new /compact prompt in cc is pretty good, but ig it could be problematic with smaller context windows; also, at least the "user messages" part could be hardcoded; related: feat(compact): override compact default model and prompt #2465

claude code compact prompt

Your task is to create a detailed summary of the conversation so far, paying close attention to the user's explicit requests and your previous actions.
This summary should be thorough in capturing technical details, code patterns, and architectural decisions that would be essential for continuing development work without losing context.

Before providing your final summary, wrap your analysis in <analysis> tags to organize your thoughts and ensure you've covered all necessary points. In your analysis process:

1. Chronologically analyze each message and section of the conversation. For each section thoroughly identify:
   - The user's explicit requests and intents
   - Your approach to addressing the user's requests
   - Key decisions, technical concepts and code patterns
   - Specific details like:
     - file names
     - full code snippets
     - function signatures
     - file edits
  - Errors that you ran into and how you fixed them
  - Pay special attention to specific user feedback that you received, especially if the user told you to do something differently.
2. Double-check for technical accuracy and completeness, addressing each required element thoroughly.

Your summary should include the following sections:

1. Primary Request and Intent: Capture all of the user's explicit requests and intents in detail
2. Key Technical Concepts: List all important technical concepts, technologies, and frameworks discussed.
3. Files and Code Sections: Enumerate specific files and code sections examined, modified, or created. Pay special attention to the most recent messages and include full code snippets where applicable and include a summary of why this file read or edit is important.
4. Errors and fixes: List all errors that you ran into, and how you fixed them. Pay special attention to specific user feedback that you received, especially if the user told you to do something differently.
5. Problem Solving: Document problems solved and any ongoing troubleshooting efforts.
6. All user messages: List ALL user messages that are not tool results. These are critical for understanding the users' feedback and changing intent.
6. Pending Tasks: Outline any pending tasks that you have explicitly been asked to work on.
7. Current Work: Describe in detail precisely what was being worked on immediately before this summary request, paying special attention to the most recent messages from both user and assistant. Include file names and code snippets where applicable.
8. Optional Next Step: List the next step that you will take that is related to the most recent work you were doing. IMPORTANT: ensure that this step is DIRECTLY in line with the user's most recent explicit requests, and the task you were working on immediately before this summary request. If your last task was concluded, then only list next steps if they are explicitly in line with the users request. Do not start on tangential requests or really old requests that were already completed without confirming with the user first.
                       If there is a next step, include direct quotes from the most recent conversation showing exactly what task you were working on and where you left off. This should be verbatim to ensure there's no drift in task interpretation.

Here's an example of how your output should be structured:

<example>
<analysis>
[Your thought process, ensuring all points are covered thoroughly and accurately]
</analysis>

<summary>
1. Primary Request and Intent:
   [Detailed description]

2. Key Technical Concepts:
   - [Concept 1]
   - [Concept 2]
   - [...]

3. Files and Code Sections:
   - [File Name 1]
      - [Summary of why this file is important]
      - [Summary of the changes made to this file, if any]
      - [Important Code Snippet]
   - [File Name 2]
      - [Important Code Snippet]
   - [...]

4. Errors and fixes:
    - [Detailed description of error 1]:
      - [How you fixed the error]
      - [User feedback on the error if any]
    - [...]

5. Problem Solving:
   [Description of solved problems and ongoing troubleshooting]

6. All user messages: 
    - [Detailed non tool use user message]
    - [...]

7. Pending Tasks:
   - [Task 1]
   - [Task 2]
   - [...]

8. Current Work:
   [Precise description of current work]

9. Optional Next Step:
   [Optional Next step to take]

</summary>
</example>

Please provide your summary based on the conversation so far, following this structure and ensuring precision and thoroughness in your response. 

There may be additional summarization instructions provided in the included context. If so, remember to follow these instructions when creating the above summary. Examples of instructions include:
<example>
## Compact Instructions
When summarizing the conversation focus on typescript code changes and also remember the mistakes you made and how you fixed them.
</example>

<example>
# Summary instructions
When you are using compact - please focus on test output and code changes. Include file reads verbatim.
</example>

[agladysh]

See also: https://ampcode.com/news/handoff

[rekram1-node]

yeah internally we were discussing some better approaches to this

[mschenk42]

I find the handoff concept appealing. Compaction alone provides insufficient headroom, resulting in an endless cycle of auto-compaction. This has been my recent experience. Additionally, I am curious about the overhead from opencode for the context window. I am unsure if it is just me or if I am not comprehending how much context I am creating. When starting a new session, initial work seems to indicate a larger than anticipated context size. Perhaps the size estimates are inaccurate?