We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python 3.8
通过max_tokens来截断对话长度,是节省token消耗的有效手段,这是合理的。 在chatgpt.py中是这么调用save函数的:
def reply(self, context: Context) -> Reply: ... if response["completion_tokens"] > 0: Session.save_session( response["content"], session_id, response["total_tokens"] ) return Reply(ReplyType.TEXT, response["content"])
而在session.py里,save和discard对话的函数是这样的:
@staticmethod def save_session(answer, session_id, total_tokens): max_tokens = conf().get("max_tokens") session = Session.all_sessions.get(session_id) if session: # append conversation gpt_item = {"role": "assistant", "content": answer} session.append(gpt_item) # discard exceed limit conversation Session.discard_exceed_conversation(session, max_tokens, total_tokens) @staticmethod def discard_exceed_conversation(session, max_tokens, total_tokens): dec_tokens = int(total_tokens) while dec_tokens > max_tokens: # pop first conversation if len(session) > 3: session.pop(1) session.pop(1) else: break dec_tokens = dec_tokens - max_tokens
max_tokens是对话长度的截断阈值,total_tokens是问答消息的总长度,严谨的逻辑应该是,不断pop掉最前面的问答直到session的内容长度小于max_tokens。这样的话dec_tokens每次要减掉的不是max_tokens,而是被pop掉的问答的对话长度。不知道是否是因为难以估计被pop掉的对话长度而直接使用max_tokens。
No response
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Search for answers in existing issues
Python version
python 3.8
Issue description
通过max_tokens来截断对话长度,是节省token消耗的有效手段,这是合理的。
在chatgpt.py中是这么调用save函数的:
而在session.py里,save和discard对话的函数是这样的:
max_tokens是对话长度的截断阈值,total_tokens是问答消息的总长度,严谨的逻辑应该是,不断pop掉最前面的问答直到session的内容长度小于max_tokens。这样的话dec_tokens每次要减掉的不是max_tokens,而是被pop掉的问答的对话长度。不知道是否是因为难以估计被pop掉的对话长度而直接使用max_tokens。
Repro steps
No response
Relevant log output
No response
The text was updated successfully, but these errors were encountered: