Get Tokens for a given string input

[VpkPrasanna]

i wanted to know is there any way how to see the tokens for an given string or any api from anthropic for a given string .
Like how other LLM has their own tokenizer to which return back the tokens .

[ncoop57]

Not currently, but this would make an awesome PR if you'd be interested in adding it 🤓 @VpkPrasanna

[VpkPrasanna]

would love to do it

Not currently, but this would make an awesome PR if you'd be interested in adding it 🤓 @VpkPrasanna

[VpkPrasanna]

would love to do it

Not currently, but this would make an awesome PR if you'd be interested in adding it 🤓 @VpkPrasanna

need some help to do it, because till now i could not able to find any online implementation of the same . am telling this when i raised this issue

[ncoop57]

Hey sorry for the late response @VpkPrasanna. You can see how to get the count of the tokens from this api page: https://docs.anthropic.com/en/docs/build-with-claude/token-counting

[VpkPrasanna]

Hey sorry for the late response @VpkPrasanna. You can see how to get the count of the tokens from this api page: https://docs.anthropic.com/en/docs/build-with-claude/token-counting

Actually i am aware of this but the issue raised is how to get the tokens back for a given string ?
like how tiktoken is there to get the tokens and other things