Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add token usage #121

Merged
merged 1 commit into from
Sep 18, 2024

Conversation

desaxce
Copy link
Contributor

@desaxce desaxce commented Sep 17, 2024

Changes:

  • add token usage for LangChain

To test:

  • Run the following snippet against a Literal AI project and notice token counts:
import os
from literalai import LiteralClient

from langchain_anthropic import ChatAnthropic
from langchain.schema.runnable.config import RunnableConfig
from langchain.schema import StrOutputParser
from langchain.prompts import ChatPromptTemplate

# Set ANTHROPIC_API_KEY

literal_client = LiteralClient(api_key="lsk-***")

cb = literal_client.langchain_callback()

prompt = ChatPromptTemplate.from_messages(
    ['human', 'Tell me a short joke about {topic}']
)

model = ChatAnthropic(model_name="claude-3-5-sonnet-20240620")
runnable = prompt | model

res = runnable.invoke(
    {"topic": "ice cream"},
    config=RunnableConfig(callbacks=[cb], run_name="joke")
)

@desaxce
Copy link
Contributor Author

desaxce commented Sep 18, 2024

It works with ChatAnthropic(model_name="claude-3-5-sonnet-20240620", streaming=True) too.

@desaxce desaxce merged commit e8ce2f9 into main Sep 18, 2024
2 checks passed
@desaxce desaxce deleted the hugues/eng-1890-claude-sonnet-token-count-not-showing branch September 18, 2024 16:15
@kumar781
Copy link

will it work with azureopenai gpt 4o?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants