[Question]: Unexpected Token Usage with Anthropic API

[PogChamp-Peter]

What is your question?

I'm using Anthropic's API to access Claude Sonnet 20240620 through LibreChat [Running via npm], and I've run into a weird issue with token usage. I made a simple query just to check how many tokens I'd consume:

When I checked the Anthropic dashboard, the input token count was way too high.

This is the first conversation I’ve ever had with the API, and I’m sure the API key isn’t being used anywhere else. The timestamps match exactly when I sent the query to Sonnet3.5, so it’s not a coincidence.

What’s even stranger is that the dashboard shows token usage for Haiku as well, something I never accessed or even attempted to use. And also the weird HTTP and Streaming diff btw Sonnet and Haiku.

To rule out caching as the culprit, I disabled it and ran another query in Chat 2. Same result, absurdly high input tokens.

I’m not using any prompts or custom instructions.

Everything is running on the default settings in LibreChat. Any idea what could be causing this massive token discrepancy or how to fix it? I'd appreciate any insight because this behavior doesn't make sense based on my usage.

Thanks in advance!

More Details

No response

What is the main subject of your question?

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Haiku is used for creating titles at the start of the conversation. There is a bit of prompting involved for the title, you can also disable titling.

[PogChamp-Peter]

Oh okay, that makes sense for Haiku being called :)

But, I still don't get why Sonnet's using way too much input tokens, when my input roughly amounts to 10-20 tokens...

Any other option(s) that might affect the input tokens usage?

[PogChamp-Peter]

I deleted the librechat.yaml config file and tried again, and it's the same issue, except the request got sent twice:

It literally wasted 11k tokens in total for no reason...

I don't know who's at fault here. Is it Anthropic or some extra feature of LibreChat? Can somebody please help?

[fuegovic]

Look if you have Artifacts enabled. When active it will send a big prompt to the LLM with instructions about its usage

[PogChamp-Peter]

Is there a way to check the number of input tokens that would be sent before actual sending the input? I wanna absolutely make sure to not end up using another 2k, trying to test if it works or not;

And, on a related side note, not to sound rude, but why is this enabled by default? Atleast, I would have preferred a pop-up telling you what it is, and that it would cost u extra tokens, cause Anthropic tokens are expensive.

[PogChamp-Peter]

I tried the same prompt in big-agi.com and it worked just as expected:

The issue definitely seems to be with LibreChat; I'm not sure if it's a bug or some enabled-by-default feature...