feat: add file persistence for large tool outputs #6234
Conversation
|
I'm constantly having the prompt is too long error so this PR would be greatly appreciated. |
|
Can we get this prioritized @fwang |
|
This seems likely to fix the single biggest usability issue with opencode that I keep hitting over and over and over... literally daily, multiple times per day with normal use of opencode. |
It's worked for me over the past few days. Haven't hit the "prompt too long" issue once. |
Cherry-picked from PR anomalyco#6234 (ben-vargas/ai-opencode) When tool outputs exceed 30,000 characters, save to disk instead of passing to model context. This prevents 'prompt is too long' errors caused by large WebFetch, Bash, or MCP tool outputs. Changes: - Add tool-results.ts: Storage helper for large outputs - Update bash.ts, webfetch.ts, websearch.ts, codesearch.ts: File persistence - Update prompt.ts: MCP tools save large outputs to file - Update session/index.ts: Cleanup tool_results on session delete - Add OPENCODE_EXPERIMENTAL_MCP_MAX_OUTPUT_LENGTH flag for configurability
Just wanted to note, this actually hasn't completely fixed the bug for me. It has significantly lowered the frequency that I run into it, but it still happens a few times a day now. Seems like the team is working on a fix though. |
|
@codyw912 - yeah, this is just addressing one possible cause (tool use and specifically mcp tools) which often caused it for me. But i'm certain there are other edge cases to be addressed, hopefully their team gets all of them if not wanting to merge this PR. I do hope they retain or implement a feature similar to writing to a file rather than just returning truncated data, the models are much more effective when they can be warned of the truncation and take a couple attempts at reading/exploring the large data, especially useful when using tools in enterprise (think reading data from salesforce or servicenow APIs). Flat out truncation of what the model gets back from tools alone isn't great. @thdxr - Any thoughts on having this kind of file write rather than pure truncation of tools? |
ben-vargas
force-pushed
the
mcp-output-file-persistence
branch
from
496ae0e to
0339c06
Compare
|
+1 |
Refactors the file persistence feature to work with the new centralized
truncation architecture in upstream. Instead of modifying each tool
individually, this integrates file persistence directly into the
Truncate namespace.
When tool output exceeds 50KB/2000 lines, the full output is saved to
disk and the model receives a head+tail preview with instructions for
exploring the data using Read/Grep/jq tools.
Changes:
- truncation.ts: Add outputWithPersistence() function with:
- Head (67%) + tail (33%) preview respecting caller's maxLines
- Preview byte cap respecting min(caller's maxBytes, 20KB)
- JSON detection (.json extension + jq hint)
- 10MB hard cap on persisted files (using Buffer for both detection
AND truncation to handle multibyte chars correctly)
- Tool name sanitization (64 char limit, safe chars only)
- Path traversal protection (sessionID validation + resolve check)
- cleanupSessionFiles() for session deletion
- tool.ts: Update Tool.define() wrapper to use outputWithPersistence
- registry.ts: Update fromPlugin() to use outputWithPersistence
- session/index.ts: Call Truncate.cleanupSessionFiles on session delete
Benefits over previous per-tool approach:
- Single point of change - all tools automatically get file persistence
- Aligns with upstream's centralized truncation architecture
- No duplicate truncation logic across multiple files
- Cleaner integration with existing session lifecycle
ben-vargas
force-pushed
the
mcp-output-file-persistence
branch
from
2e5a12e to
ce28fd3
Compare
MCP tools have a separate execution path in prompt.ts that wasn't covered by the centralized truncation changes. This adds Truncate.outputWithPersistence() to the MCP tool wrapper so that large outputs from MCP tools (like firecrawl) are also saved to disk instead of overflowing the context window. Also improves the truncation notice to be more explicit that the preview is INCOMPLETE and the model MUST read the file for accurate results - prevents models from answering based on partial data.
When a tool output is truncated and saved to file, the TUI now shows a [truncated] indicator in the tool call line so users can see at a glance that the output was truncated. Shows in warning color on both InlineTool and BlockTool displays.
|
@rekram1-node one thing I found really helpful on the tool truncation persistence was the small little indicator to the user I added in the refactor which was done after your PR merging concepts. You might consider adding something similar to 5e0aca1 in this branch so the user knows when a tool was truncated... helpful because sometimes I have to know truncation happened and prompt it to look a little deeper at the persisted tool output, and user doesn't know unless there is an indicator of some kind. |
|
Also, thanks for considering and landing the persistence!! |
5 participants
Summary
Enhances the centralized truncation system with file persistence for large tool outputs.
Background: Upstream recently added centralized truncation (
Truncate.output()intool.tsandregistry.ts) which addresses "prompt is too long" errors by capping outputs at 50KB/2000 lines. However, this truncation permanently loses data - the model has no way to access content beyond the cutoff.This PR retains the benefits of file persistence from the original approach: when outputs exceed the threshold, the full output is saved to disk and the model receives a preview with instructions for exploring the data using Read, Grep, or jq tools. The model can then make targeted requests to examine specific portions of large outputs.
Closes #4560
Related: #4826, #4845, #5360
Changes
truncation.ts- Extends the centralized truncation namespace with:outputWithPersistence()- saves large outputs to~/.local/share/opencode/storage/tool_results/{sessionID}/.jsonextension + jq hint for structured queries)cleanupSessionFiles()for session deletion lifecycletool.ts- UpdatesTool.define()wrapper to useoutputWithPersistenceregistry.ts- UpdatesfromPlugin()to useoutputWithPersistencesession/index.ts- Calls cleanup on session deleteBenefits over pure truncation
Coverage
All tools automatically get file persistence:
Tool.define()wrapperfromPlugin()