File Support For Qwen-long

[TChengZ]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.11.0

Cloud or Self Hosted

Cloud

Steps to reproduce

1、add a llm node using Qwen-Long
2、passing a long text into system

3、trigger the node,but fail like

since qwen-long support 10000k tokens and like the doc

if the input text longer than 1m token，it will failed,and suggest using file-id mode to enable large tokens,but dify llm node seems not to support this.

✔️ Expected Behavior

using Qwen-Long with large tokens running successful

❌ Actual Behavior

fail with large tokesn

[dosubot]

I couldn't find specific information about Dify's support for the file-id mode with Qwen-Long for handling large token inputs. However, there is a related discussion about errors when using large language models, which might provide some insights or similar experiences from other users ^[1]. Additionally, there is a closed issue related to token limits that might be relevant ^[2].

If you're encountering issues with token limits, you might want to check the configuration settings in Dify that can be adjusted to support larger token inputs. Specifically, you can modify parameters like PROMPT_GENERATION_MAX_TOKENS and CODE_GENERATION_MAX_TOKENS in the configuration to increase the token limits ^[3].

To continue talking to Dosu, mention @dosu.

[crazywoola]

There are few things you might misunderstood.

• You don't need the tongyi model to understand file, because the doc extractor will produce things in Array[text].
• You reference the result of the doc extractor twice, which might exceed the length
• Please use /context to add things in system prompt. FYI, tongyi doesn't seem to support this role, you can change it to user instead.

[TChengZ]

There are few things you might misunderstood.

• You don't need the tongyi model to understand file, because the doc extractor will produce things in Array[text].
• You reference the result of the doc extractor twice, which might exceed the length
• Please use /context to add things in system prompt. FYI, tongyi doesn't seem to support this role, you can change it to user instead.

i can remove the doc extractor, and i don't understant the /context u said, how to use /context
in offical qwen-long website, i can just upload a file and ask like below

how can i simulate this process in dify

[TChengZ]

i use /context in the user, and i am sure the doc is under 10000k tokens，it still exceed the tokens