usage in openai compatiable api' llm update from upstream

[guogeer]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.10.1

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

usage in openai compatiable api' llm count by tiktoken, but response data may include field usage

✔️ Expected Behavior

usage is same with response

❌ Actual Behavior

usage is not same with response

[dosubot]

To ensure that the usage in the OpenAI compatible API's LLM update matches the response data when using tiktoken in a self-hosted Dify version 0.10.1 setup, you need to accurately calculate the number of tokens used in the request and response. This involves using the _num_tokens_from_string and _num_tokens_from_messages methods in the DeepSeekLargeLanguageModel class, which is a subclass of OpenAILargeLanguageModel.

1. Token Calculation for Text Completion:
   Use the _num_tokens_from_string method to calculate the number of tokens for text completion models. This method uses the tiktoken package with the cl100k_base encoding to encode the text and count the tokens.

2. Token Calculation for Chat Models:
   Use the _num_tokens_from_messages method to calculate the number of tokens for chat models like GPT-3.5-turbo and GPT-4. This method also uses the cl100k_base encoding and accounts for additional tokens per message and per name.

By using these methods, you can ensure that the token usage in your API requests matches the response data, as both are calculated using the same encoding and logic ^[1][2].

To continue talking to Dosu, mention @dosu.

[crazywoola]

Sorry we do not have a plan for this, if you are interested, please open a pull request.