fix(proxy): add OpenAI chat completion format support in usage extraction

[ding113]

Summary

Fixes OpenAI chat completion format token usage extraction bug where prompt_tokens/completion_tokens were not being extracted, resulting in 0 cost/token display.

Related Issues:

• Fixes [bug] 使用openai compatible 接口，无法对token进行计费 #705 - OpenAI compatible interface token billing not working
• Closes [bug] 使用openai compatible 接口，无法对token进行计费 #700 - Duplicate issue reported by another user

Problem

When using the /v1/chat/completions endpoint with OpenAI-compatible providers, the system failed to extract token usage from responses, causing:

• Usage dashboard showing 0 tokens
• Cost calculation showing $0.00
• Missing billing data despite successful API calls

Root Cause:
The extractUsageMetrics function (response-handler.ts:1193-1468) only handled:

1. Claude format: input_tokens, output_tokens
2. Gemini format: promptTokenCount, candidatesTokenCount

But OpenAI chat completion format uses prompt_tokens/completion_tokens, which were never extracted. This affected both JSON responses and SSE streaming (where usage appears in the final chunk).

Solution

Enhanced extractUsageMetrics to support OpenAI format with priority-based extraction:

• Priority: Claude format > Gemini format > OpenAI format
• Added extraction for prompt_tokens → input_tokens (if input_tokens undefined)
• Added extraction for completion_tokens → output_tokens (if output_tokens undefined)

This ensures Claude and Gemini formats take precedence when present, while OpenAI format serves as fallback.

Changes

Core Changes

• src/app/v1/_lib/proxy/response-handler.ts (+14 lines)
  • Added OpenAI prompt_tokens extraction at line 1199-1202
  • Added OpenAI completion_tokens extraction at line 1288-1291
  • Preserves existing format priority (Claude > Gemini > OpenAI)

Testing

• tests/unit/proxy/extract-usage-metrics.test.ts (+82 lines)
  • 5 new test cases for OpenAI format:
    1. Basic prompt_tokens/completion_tokens extraction
    2. Format priority verification (Claude takes precedence)
    3. SSE streaming with usage in final chunk
    4. completion_tokens_details handling
    5. Individual field extraction

Test Results

• ✅ All 43 unit tests pass
• ✅ TypeCheck passes
• ✅ Build succeeds
• ✅ New test coverage: 5 test cases covering OpenAI format edge cases

Verification

Tested with actual OpenAI-compatible provider response (from Issue #705):

{
  "usage": {
    "completion_tokens": 57,
    "total_tokens": 123,
    "prompt_tokens": 66,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    }
  }
}

✅ Now correctly extracts input_tokens: 66, output_tokens: 57

---

Generated with Claude Code
Description enhanced by Claude AI PR Agent

[gemini-code-assist]

Summary of Changes

Hello @ding113, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug where usage metrics from OpenAI chat completion responses were not being properly recorded. It enhances the extractUsageMetrics function to correctly identify and process prompt_tokens and completion_tokens, ensuring accurate token accounting for OpenAI-compatible services. The changes also formalize the priority for extracting usage data across different provider formats (Claude, Gemini, OpenAI) and are backed by comprehensive new unit tests to guarantee reliability.

Highlights

• OpenAI Chat Completion Usage Extraction: The extractUsageMetrics function now correctly extracts prompt_tokens and completion_tokens from OpenAI chat completion responses, resolving an issue where these metrics were previously ignored.
• Token Extraction Priority: A clear priority has been established for token extraction: Claude format (input_tokens, output_tokens) takes precedence, followed by Gemini format (promptTokenCount, candidatesTokenCount), and then OpenAI format (prompt_tokens, completion_tokens).
• New Unit Tests: Five new unit test cases have been added to thoroughly validate the correct extraction of OpenAI chat completion usage metrics, including scenarios for streaming responses and completion_tokens_details.

Changelog

• src/app/v1/_lib/proxy/response-handler.ts
  • Implemented logic to map OpenAI's prompt_tokens to input_tokens within the extractUsageMetrics function, respecting existing Claude and Gemini format priorities.
  • Implemented logic to map OpenAI's completion_tokens to output_tokens within the extractUsageMetrics function, respecting existing Claude and Gemini format priorities.
• tests/unit/proxy/extract-usage-metrics.test.ts
  • Added a new test suite specifically for 'OpenAI chat completion format (prompt_tokens/completion_tokens)'.
  • Included test cases to verify the extraction of prompt_tokens as input_tokens and completion_tokens as output_tokens.
  • Added a test to confirm that Claude format's input_tokens and output_tokens are preferred over OpenAI's prompt_tokens and completion_tokens when both are present.
  • Added a test case to ensure correct usage metric extraction from OpenAI streaming chunks in the final event.
  • Added a test case to handle completion_tokens_details (e.g., reasoning_tokens) in OpenAI-compatible responses.

Activity

• The author has added 5 new unit tests to cover the new functionality.
• All 43 existing tests passed successfully after the changes.
• TypeCheck passed, indicating no type-related issues were introduced.
• The build process completed successfully.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[coderabbitai]

📝 Walkthrough

Walkthrough

本次变更为响应处理器添加了兼容性映射，支持OpenAI/Claude风格的token字段（prompt_tokens/completion_tokens映射到input_tokens/output_tokens），并增加了相应的测试覆盖以验证跨provider的token使用统计功能。

Changes

Cohort / File(s)	Summary
Token兼容性映射 src/app/v1/_lib/proxy/response-handler.ts	在extractUsageMetrics中添加条件逻辑，当input_tokens/output_tokens未定义时，从prompt_tokens/completion_tokens字段进行回退映射，以支持OpenAI兼容接口的token统计。
单元测试覆盖 tests/unit/proxy/extract-usage-metrics.test.ts	新增测试套件验证OpenAI风格usage字段解析，包含prompt_tokens→input_tokens、completion_tokens→output_tokens的映射，以及字段优先级和流式数据聚合场景。

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• PR #691：修改Gemini SSE处理逻辑中usageMetadata的聚合策略，同样涉及usage提取代码的改进
• PR #664：在extractUsageMetrics中添加基于模态的image/text token字段提取，属于同一模块的provider兼容性增强

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	标题清晰准确地描述了主要变更：为使用量提取函数添加OpenAI聊天完成格式支持。
Linked Issues check	✅ Passed	代码变更完全满足issue #705的要求：添加了OpenAI chat completion格式（prompt_tokens/completion_tokens）的支持，实现了字段映射。
Out of Scope Changes check	✅ Passed	所有变更都在scope范围内，仅涉及extractUsageMetrics函数的增强和相应的单元测试，无额外不相关的改动。
Description check	✅ Passed	The pull request description clearly relates to the changeset, detailing the fix for OpenAI-compatible token extraction with root cause analysis and test coverage.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/705-openai-usage-extraction

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request correctly adds support for extracting usage metrics from OpenAI-compatible chat completion responses, including prompt_tokens and completion_tokens. The changes follow the existing structure and the new unit tests are comprehensive, covering various scenarios like format priority and streaming responses.

I've identified one related issue regarding priority handling for Gemini tokens that you may want to address for consistency. Overall, this is a good fix that improves provider compatibility.

[github-actions]

🧪 测试结果

测试类型	状态
代码质量	✅
单元测试	✅
集成测试	✅
API 测试	✅

总体结果: ✅ 所有测试通过

[greptile-apps]

_{2 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (2)

src/app/v1/_lib/proxy/response-handler.ts
Pre-existing issue (not introduced by this PR): Gemini's promptTokenCount doesn't check if result.input_tokens is already defined, which could override Claude's input_tokens if both formats are present. OpenAI format correctly checks result.input_tokens === undefined on line 1199. Consider adding the same check here for consistency:

  if (result.input_tokens === undefined && typeof usage.promptTokenCount === "number") {
    const cachedTokens =
      typeof usage.cachedContentTokenCount === "number" ? usage.cachedContentTokenCount : 0;
    result.input_tokens = Math.max(usage.promptTokenCount - cachedTokens, 0);
    hasAny = true;
  }

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/app/v1/_lib/proxy/response-handler.ts
Line: 1186:1191

Comment:
Pre-existing issue (not introduced by this PR): Gemini's `promptTokenCount` doesn't check if `result.input_tokens` is already defined, which could override Claude's `input_tokens` if both formats are present. OpenAI format correctly checks `result.input_tokens === undefined` on line 1199. Consider adding the same check here for consistency:

```suggestion
  if (result.input_tokens === undefined && typeof usage.promptTokenCount === "number") {
    const cachedTokens =
      typeof usage.cachedContentTokenCount === "number" ? usage.cachedContentTokenCount : 0;
    result.input_tokens = Math.max(usage.promptTokenCount - cachedTokens, 0);
    hasAny = true;
  }
```

How can I resolve this? If you propose a fix, please make it concise.

---

src/app/v1/_lib/proxy/response-handler.ts
Pre-existing issue (not introduced by this PR): candidatesTokenCount should check if result.output_tokens is already defined to maintain stated priority. OpenAI format correctly implements this on line 1290.

  if (result.output_tokens === undefined && typeof usage.candidatesTokenCount === "number") {
    result.output_tokens = usage.candidatesTokenCount;
    hasAny = true;
  }

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/app/v1/_lib/proxy/response-handler.ts
Line: 1192:1195

Comment:
Pre-existing issue (not introduced by this PR): `candidatesTokenCount` should check if `result.output_tokens` is already defined to maintain stated priority. OpenAI format correctly implements this on line 1290.

```suggestion
  if (result.output_tokens === undefined && typeof usage.candidatesTokenCount === "number") {
    result.output_tokens = usage.candidatesTokenCount;
    hasAny = true;
  }
```

How can I resolve this? If you propose a fix, please make it concise.

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/app/v1/_lib/proxy/response-handler.ts (1)

1186-1202: ⚠️ Potential issue | 🟠 Major

Gemini 的 promptTokenCount 可能覆盖 Claude 的 input_tokens，优先级未被真正执行。

当前逻辑在存在 usage.input_tokens 时仍会被 promptTokenCount 覆盖，和注释中“Claude > Gemini > OpenAI”的优先级不一致，可能导致混合/转发场景下计费偏差。建议仅在 input_tokens 未设置时才使用 Gemini 的计数。

Proposed fix

-  if (typeof usage.promptTokenCount === "number") {
+  if (result.input_tokens === undefined && typeof usage.promptTokenCount === "number") {
     const cachedTokens =
       typeof usage.cachedContentTokenCount === "number" ? usage.cachedContentTokenCount : 0;
     result.input_tokens = Math.max(usage.promptTokenCount - cachedTokens, 0);
     hasAny = true;
   }

[github-actions]

Code Review Summary

No significant issues identified in this PR. Review performed via static inspection of the diff; automated test/build execution was not run in this runner.

PR Size: S

• Lines changed: 96
• Files changed: 2

Review Coverage

• Logic and correctness - Clean
• Security (OWASP Top 10) - Clean
• Error handling - Clean
• Type safety - Clean
• Documentation accuracy - Clean
• Test coverage - Adequate
• Code clarity - Good

---

Automated review by Codex AI

[github-actions]

• PR #716: applied label size/S (96 changed lines across 2 files).
• Reviewed diff in src/app/v1/_lib/proxy/response-handler.ts and tests/unit/proxy/extract-usage-metrics.test.ts; no issues in modified/added lines met the reporting threshold, so no inline comments were posted.
• Submitted a summary review to the PR via gh pr review --comment.
• Note: automated test/build execution wasn’t run in this runner (bun not present and node_modules missing); if you want a verification run, use bun install && bun run test.