Community : Add OpenAI prompt caching and reasoning tokens tracking

[vignesh14052002]

Added Token tracking for OpenAI's prompt caching and reasoning tokens
Costs updated from https://openai.com/api/pricing/

usage example

from langchain_community.callbacks import get_openai_callback
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model_name="o1-mini",temperature=1)

with get_openai_callback() as cb:
    response = llm.invoke("hi "*1500)
    print(cb)

Output

Tokens Used: 1720
	Prompt Tokens: 1508
		Prompt Tokens Cached: 1408
	Completion Tokens: 212
		Reasoning Tokens: 192
Successful Requests: 1
Total Cost (USD): $0.0049559999999999995

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Dec 18, 2024 4:24pm

[vignesh14052002]

Its been a week, can you review this? @baskaryan @efriis @eyurtsev

[efriis]

unfortunately this is a breaking change due to the removal of is_completion. Will wait for @baskaryan to return from vacation to think about how we want to handle.

[vignesh14052002]

unfortunately this is a breaking change due to the removal of is_completion. Will wait for @baskaryan to return from vacation to think about how we want to handle.

@efriis will it be okay, if I revert it and use another Boolean is_cached_prompt ?

[zep-hyr]

@efriis @baskaryan I am waiting for this PR to be processed. Could you provide an update on its progress?
cc. @vignesh14052002

[ccurme]

@baskaryan @efriis I made some modifications here such that:

• Token type (prompt, completion, cached) is represented with an enum.
• is_completion is deprecated in favor of this enum. Passing is_completion=True will generate a warning.

Warnings will only be generated for users if they are using get_openai_token_cost_for_model or standardize_model_name with is_completion=True directly. These functions are not in our docs. Using the callback as documented won't generate a deprecation warning.

lmk if any thoughts.