chatglm-turbo support 3.2k token

[xutengfei]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to file this report (我已阅读并同意 Language Policy).

Dify version

0.3.32

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

✔️ Expected Behavior

❌ Actual Behavior

we can not change max token of chatglm-turbo

[crazywoola]

We will take a look into this issue. :)

---- Update----
The model did not provide a setting called max_token, so it shouldn't be able to display anyway. We will remove this setting later.

[xutengfei]

We will take a look into this issue. :)

---- Update---- The model did not provide a setting called max_token, so it shouldn't be able to display anyway. We will remove this setting later.

so we have to wait the version of 0.3.4?
now the request log show that dify did not submit the content to llm model when context larger than 512token.

[crazywoola]

Yes, until we fix this.

[crazywoola]

I have a question about the version, we are currently using v0.3.33, And you said you are using 0.3.2? Do you mean 0.3.32?

[xutengfei]

I have a question about the version, we are currently using v0.3.33, And you said you are using 0.3.2? Do you mean 0.3.32?

yes ,i mean 0.3.32. Thanks

[crazywoola]

This is a frontend bug, it won't affect what it will send to the llm. Because we do not have max token for chatglm_turbo. So it's a placeholder(default) value.
First image shows the 512 tokens, but actually it costs more than this one.

[WSDzju]

This is a frontend bug, it won't affect what it will send to the llm. Because we do not have max token for chatglm_turbo. So it's a placeholder(default) value. First image shows the 512 tokens, but actually it costs more than this one.

I also found the problem. In the case of gpt-3.5-turbo, the Max Token can be costomized. So the Max Token option will be added for Chatglm? I also try the lastest version 0.3.34, the option have not provided yet. BTW, the ouput length of chatglm-turbo is relatively short, although I have provided considerable context docs and prompted to give the answer as long as possible. So the situation is only related to LLM itself ?

[crazywoola]

We have update the UI, and there is no max_token in ChatGLM. This should be resolved in v0.4.x