Fix provider exhaustion after model redirect (refs #629)

[ding113]

Summary

• Fixes provider failover logic to use the original (pre-redirect) model for provider selection, preventing premature provider exhaustion when a previous provider applied modelRedirects.
• Makes Claude non-stream response converters tolerant of tool_result content blocks (which do not have a .text field), avoiding crashes like ClaudeContentBlockToolResult has no attribute 'text'.

Problem

Fixes #629 - When a provider redirects the model (e.g. claude-* -> glm-*), subsequent failover attempts were selecting/validating providers against the redirected model name, which could incorrectly filter out the remaining healthy Claude providers and end in "no available providers" after only a couple attempts.

The issue manifested as:

• System repeatedly retrying the same failed provider instead of switching to backup providers
• Provider exhaustion occurring prematurely due to model name mismatch after redirect
• Circuit breaker triggering but failover not executing properly

Solution

Use session.getOriginalModel() instead of session.getCurrentModel() throughout the provider selection pipeline to ensure:

• Provider selection is based on the user's originally requested model
• Model support validation checks against the correct model name
• Session reuse logic matches providers correctly
• Decision context tracking reflects the actual user request

Additionally, make response converters resilient to unexpected tool_result blocks in Claude responses.

Changes

Core Changes

• src/app/v1/_lib/proxy/provider-selector.ts (5 locations): Replace getCurrentModel() with getOriginalModel() for:
  • Provider reuse validation
  • Model support checks
  • Decision context tracking
  • Provider selection logic

Supporting Changes

• src/app/v1/_lib/converters/openai-to-claude/response.ts: Add tool_result case handler to ignore these blocks in non-stream conversion
• src/app/v1/_lib/converters/claude-to-codex/response.ts: Add tool_result case handler to ignore these blocks in non-stream conversion
• tests/unit/proxy/converters-tool-result-nonstream.test.ts: New test coverage for tool_result tolerance (86 lines)
• tests/unit/proxy/provider-selector-total-limit.test.ts: Update mock session to include getOriginalModel() method

Testing

Automated Tests

• Unit tests added for tool_result block handling
• Existing provider selector tests updated
• All tests pass (bun run test)

Verification

• bun run typecheck (pass)
• bun run lint (pass)
• bun run test (pass)
• bun run build (pass)

Related

• Related to feat(providers): add vendor endpoints with probes and failover #608 - Vendor endpoints architecture (different approach to provider availability)

---

Description enhanced by Claude AI

[coderabbitai]

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

Walkthrough

在两个 Claude 非流响应转换器中添加对 tool_result 块的显式忽略逻辑；将 provider-selector 中所有 session.getCurrentModel() 调用替换为 session.getOriginalModel()；新增/更新单元测试覆盖这些更改。

Changes

Cohort / File(s)	变更摘要
Claude → Codex / OpenAI 非流响应转换器 src/app/v1/_lib/converters/claude-to-codex/response.ts, src/app/v1/_lib/converters/openai-to-claude/response.ts	在非流处理的内容块 switch 中新增 tool_result 分支（显式忽略），并在 switch 添加默认分支以跳过未知块类型。
Provider 选择器（模型来源改用 original） src/app/v1/_lib/proxy/provider-selector.ts	将所有 session.getCurrentModel() 调用替换为 session.getOriginalModel()，影响 ensure / pickRandomProvider / reuse 等决策路径中使用的模型值。
单元测试 tests/unit/proxy/converters-tool-result-nonstream.test.ts, tests/unit/proxy/provider-selector-total-limit.test.ts, tests/unit/proxy/provider-selector-model-redirect.test.ts	新增测试以验证 tool_result 在 Claude→OpenAI/Claude→Codex 转换中被忽略；更新/新增 provider-selector 相关测试以包含 getOriginalModel mock。

Estimated code review effort

🎯 3 (中等) | ⏱️ ~22 minutes

Possibly related PRs

• Fix provider exhaustion after model redirect (refs #629) #633: 对 Claude 非流转换器中显式忽略 tool_result 块并将 provider-selector 改为使用 getOriginalModel() 的相同代码级更改。

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	标题清晰准确地总结了PR的主要变更：修复模型重定向后的提供商耗尽问题，并引用了相关的issue #629。
Description check	✅ Passed	描述详细阐述了问题、解决方案和变更内容，与代码改动紧密相关，包括核心问题诊断和测试验证。
Linked Issues check	✅ Passed	PR完全满足issue #629的目标：通过将provider-selector中的getCurrentModel()替换为getOriginalModel()来修复模型重定向后的提供商选择问题；并通过增加tool_result块处理来解决Claude响应转换崩溃问题。
Out of Scope Changes check	✅ Passed	所有改动都与修复provider exhaustion和tool_result处理直接相关，包括provider-selector核心逻辑、响应转换器更新和相关单元测试，无超出范围的变更。
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/629-provider-fallback-toolresult

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Summary of Changes

Hello @ding113, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses two critical areas: improving provider resilience and enhancing response converter robustness. It refactors the provider failover mechanism to consistently use the initially requested model for selection, even after a model redirect, which prevents incorrect filtering and premature exhaustion of available providers. Additionally, it fortifies Claude's non-stream response converters by making them tolerant to tool_result content blocks, thereby preventing application crashes when such blocks are encountered in responses.

Highlights

• Provider Failover Logic: The system now correctly uses the original (pre-redirect) model for provider selection during failover, preventing premature provider exhaustion when a model redirect occurs.
• Claude Response Conversion Robustness: Non-stream response converters for Claude are updated to gracefully handle tool_result content blocks, avoiding crashes that previously occurred when these blocks lacked a .text field.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

🧪 测试结果

测试类型	状态
代码质量	✅
单元测试	✅
集成测试	✅
API 测试	✅

总体结果: ✅ 所有测试通过

[gemini-code-assist]

Code Review

The pull request effectively addresses the identified issues related to provider exhaustion during model redirects and handling tool_result blocks in non-stream responses. The core logic change in provider-selector.ts correctly switches to using the original model for provider selection, which is crucial for proper failover. The converter updates gracefully ignore tool_result blocks, preventing crashes, and new unit tests have been added to ensure this behavior. Overall, the changes enhance the system's robustness and reliability.

[gemini-code-assist]

The default case currently ignores unknown block types. While this prevents crashes, it might be beneficial to log a warning if an unknown block type is encountered. This would provide better observability and help detect if new, potentially important, block types are introduced by the Claude API that should be explicitly handled by the converter.

Suggested change

	default:
	// Unknown block types are ignored for non-stream output.
	break;
	default:
	logger.warn("[Claude->Codex] Unknown content block type encountered", { blockType });
	break;

[gemini-code-assist]

Similar to the Claude-to-Codex converter, silently ignoring unknown block types in the default case could lead to missed opportunities for handling new API features or debugging unexpected responses. Consider adding a warning log here.

Suggested change

	default:
	// Unknown block types are ignored for non-stream output.
	break;
	default:
	logger.warn("[Claude->OpenAI] Unknown content block type encountered", { blockType });
	break;

[greptile-apps]

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[github-actions]

Code Review Summary

No significant issues identified in this PR. The changes correctly address the provider exhaustion bug by consistently using getOriginalModel() instead of getCurrentModel() for provider selection, and properly handle tool_result content blocks in response converters.

PR Size: S

• Lines changed: 118
• Files changed: 5

Review Coverage

• Logic and correctness - Clean
• Security (OWASP Top 10) - Clean
• Error handling - Clean
• Type safety - Clean
• Documentation accuracy - Clean
• Test coverage - Adequate
• Code clarity - Good

Validation Notes

Provider Selector Changes (5 locations):
All getCurrentModel() → getOriginalModel() changes are in provider selection/validation contexts and correctly ensure the original user-requested model is used for provider matching during failover scenarios.

Converter Changes:
The tool_result case handlers with empty break statements are intentional no-ops - they gracefully handle unexpected content blocks that may appear in Claude responses from certain proxies. This is a defensive fix preventing crashes, not a silent failure.

Test Coverage:

• New unit tests added for tool_result block handling in both OpenAI and Codex converters
• Mock session object updated to include getOriginalModel() method

---

Automated review by Claude AI

[github-actions]

Code Review Summary

No significant issues identified in this PR. The changes correctly address the provider exhaustion bug by consistently using getOriginalModel() instead of getCurrentModel() for provider selection, and properly handle tool_result content blocks in response converters.

PR Size: S

• Lines changed: 118
• Files changed: 5

Review Coverage

• Logic and correctness - Clean
• Security (OWASP Top 10) - Clean
• Error handling - Clean
• Type safety - Clean
• Documentation accuracy - Clean
• Test coverage - Adequate
• Code clarity - Good

Validation Notes

Provider Selector Changes (5 locations):
All getCurrentModel() to getOriginalModel() changes are in provider selection/validation contexts and correctly ensure the original user-requested model is used for provider matching during failover scenarios.

Converter Changes:
The tool_result case handlers with empty break statements are intentional no-ops - they gracefully handle unexpected content blocks that may appear in Claude responses from certain proxies. This is a defensive fix preventing crashes, not a silent failure.

Test Coverage:

• New unit tests added for tool_result block handling in both OpenAI and Codex converters
• Mock session object updated to include getOriginalModel() method

---

Automated review by Claude AI

[github-actions]

[HIGH] [TEST-MISSING-CRITICAL] No regression test for model-redirect provider selection

Why this is a problem: This change switches provider filtering to use session.getOriginalModel() instead of the (potentially redirected) current model. This is core failover behavior; without a unit test that simulates currentModel being redirected (e.g. claude-* -> glm-*) while originalModel remains claude-*, it is easy to reintroduce the "no available providers" exhaustion bug during future refactors.

Suggested fix:

// tests/unit/proxy/provider-selector-model-redirect.test.ts
import { describe, expect, test, vi } from "vitest";
import type { Provider } from "@/types/provider";
import { ProxyProviderResolver } from "@/app/v1/_lib/proxy/provider-selector";

describe("ProxyProviderResolver.pickRandomProvider - model redirect", () => {
  test("filters providers using the original model (not redirected current model)", async () => {
    vi.spyOn(ProxyProviderResolver as any, "filterByLimits").mockImplementation(
      async (providers: Provider[]) => providers
    );
    vi.spyOn(ProxyProviderResolver as any, "selectTopPriority").mockImplementation(
      (providers: Provider[]) => providers
    );
    vi.spyOn(ProxyProviderResolver as any, "selectOptimal").mockImplementation(
      (providers: Provider[]) => providers[0] as Provider
    );

    const providers: Provider[] = [
      {
        id: 1,
        name: "p1",
        isEnabled: true,
        providerType: "claude",
        groupTag: null,
        weight: 1,
        priority: 0,
        costMultiplier: 1,
        allowedModels: ["claude-test"],
      } as unknown as Provider,
    ];

    const session = {
      originalFormat: "claude",
      authState: null,
      getProvidersSnapshot: async () => providers,
      getOriginalModel: () => "claude-test",
      getCurrentModel: () => "glm-test",
      clientRequestsContext1m: () => false,
    } as any;

    const { provider, context } = await (ProxyProviderResolver as any).pickRandomProvider(
      session,
      []
    );

    expect(context.requestedModel).toBe("claude-test");
    expect(provider?.id).toBe(1);
  });
});

[github-actions]

Code Review Summary

This PR adjusts provider selection to use the original (pre-redirect) model across multiple paths and makes non-stream converters ignore Claude tool_result blocks.
Converter tolerance is covered by a new unit test, but the provider-selection change lacks a regression test for the redirected-model scenario that triggered provider exhaustion.

PR Size: S

• Lines changed: 118
• Files changed: 5

Issues Found

Category	Critical	High	Medium	Low
Logic/Bugs	0	0	0	0
Security	0	0	0	0
Error Handling	0	0	0	0
Types	0	0	0	0
Comments/Docs	0	0	0	0
Tests	0	1	0	0
Simplification	0	0	0	0

Critical Issues (Must Fix)

None.

High Priority Issues (Should Fix)

• [TEST-MISSING-CRITICAL] Add a unit test that simulates currentModel being redirected while originalModel remains claude-*, and asserts provider selection still filters by the original model (Confidence: 90)

Review Coverage

• Logic and correctness
• Security (OWASP Top 10)
• Error handling
• Type safety
• Documentation accuracy
• Test coverage
• Code clarity

---

Automated review by Codex AI

[github-actions]

Applied PR size label size/S and submitted the review on PR #633 (including 1 inline comment + summary).

Findings (diff-only)

• [HIGH] [TEST-MISSING-CRITICAL] Missing regression coverage for the redirected-model provider selection path in src/app/v1/_lib/proxy/provider-selector.ts:651
  • Evidence (new behavior line):

    const requestedModel = session?.getOriginalModel() || "";

  • Why this is a problem: this line is the key behavioral change that prevents failover from filtering providers using a redirected model name. Without a unit test that forces getCurrentModel() to differ from getOriginalModel() (e.g. claude-* → glm-*), a future refactor can silently revert the bug (“no available providers” after redirect) with no test catching it.
  • Concrete fix (test suggestion): add a unit test that sets:
    • getOriginalModel(): "claude-test"
    • getCurrentModel(): "glm-test"
    • providers: only providerType: "claude" with allowedModels: ["claude-test"]
    • and asserts pickRandomProvider returns a provider + context.requestedModel === "claude-test"
      (I included a full, copy/paste-ready test snippet in the inline PR comment attached to this line.)