Vertex AI - Context Length / Comprehension Broken

[TheFynx]

App Version

3.23.12

API Provider

Google Vertex AI

Model Used

Claude 4.0 Sonnet, Gemini 2.5 Pro

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

OS: 15.5 (24F74)
VSCode: 1.101.1

Settings: Using Codebase Indexing, Reasoning Enabled, Context Condensing at 75%

Exact Actions: General Use - any new tasks/convo; every sessions context is jacked/reporting incorrectly.

Context starts at 0 and just randomizes after, never going about 1500 was the highest I've seen so far. Last was at 371 and it just fell over due to prompt/context length

I switched to openrouter as a test and everything went back to fully functional, seems to only affect Vertex (at least for the services I have access to/use)

Normal conversations jump to 5-12K context on the first interaction - with vertex is has like 5-7 listed.

💥 Outcome Summary

Expected

to condense context and keep going

Got

400 {"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 192514 + 16384 > 200000, decrease input length or max_tokens and try again"}}

Attempted to manually condense context

Failed to condense context
Context size increased during condensing; skipping this attempt

📄 Relevant Logs or Errors (Optional)

[roomote]

I am investigating this Vertex AI context length issue. The problem appears to be that context tracking is not working correctly with Vertex AI providers, showing incorrect context counts and failing to properly condense context when limits are reached. I will analyze the codebase to identify the root cause and implement a fix.

[roomote]

✅ Fix Implemented

I have successfully implemented a fix for the Vertex AI context tracking issue and created pull request #5831.

🔧 What was fixed:

1. Enhanced token counting reliability: Fixed the GeminiHandler.countTokens() method to properly handle cases where the Gemini API returns 0 or undefined token counts
2. Forced reliable token counting for Claude models: Added a countTokens() override in AnthropicVertexHandler to use tiktoken instead of the unreliable Gemini API for Claude models on Vertex AI
3. Added robust validation: Enhanced VertexHandler with additional token counting validation and fallback mechanisms
4. Improved error handling: Added better logging and error messages to help diagnose token counting issues

🎯 Expected results:

• Context counts should now display correctly (no more starting at 0 or random low values)
• Context condensing should work properly when limits are approached
• No more "context limit exceeded" errors due to incorrect token counting
• Both Vertex AI Claude and Gemini models should have consistent, reliable context tracking

🔗 Pull Request: #5831

The CI checks are currently running to validate the changes. Once they pass, the fix will be ready for review and merge.

[ryan-smylski]

I'm seeing similar context issues using claude-3-7-sonnet and claude-sonnet-4 from Amazon Bedrock. Would this fix address those as well? Specifically, in orchestrator mode, each subtask resets its context to 0, and on like a 3rd or 4th subtask, with its context showing maybe 1.5k, I start getting the "input length and max_tokens exceed context limit" error message.