upload a file to a knowledge but the indexing_status always waiting and the progress is 0%

[dwq370]

Self Checks

• This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

Dify version

0.6.0-fix1 and later

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

1. model provider is xinference and the embedding model is bge-large-zh-v1.5
2. create a knowledge
3. upload an file to knowledge
4. choice the policy to index
5. click the button Save & Process

✔️ Expected Behavior

the indexing_status always complete and the progress is 100%

❌ Actual Behavior

version 0.6.0-fix1:

version 0.6.4:

[dosubot]

你好，@dwq370！我是Dosu，一名可以帮助你解决bug、回答问题以及成为贡献者的机器人。在等待人类维护者的同时，我在这里帮忙。我将会马上查看你的问题，并很快给你一个完整的回答。🤖

[JohnJyong]

https://docs.dify.ai/getting-started/install-self-hosted/local-source-code

pls check your worker service is started or not @dwq370

[dosubot]

It looks like the issue with the indexing status not progressing might be due to the Worker service not being active or properly configured. The Worker service is crucial for handling asynchronous tasks such as data file import and dataset document updates, which includes indexing uploaded files. If this service isn't running, the indexing process won't start, leading to the symptoms you're experiencing.

To address this, please ensure the Worker service is started. For a development environment, you can start the Worker service using Celery within your project directory. If you're using Docker, make sure to define and start the Worker service in your docker-compose.yaml file. After starting the service, verify it's running and check the logs for any errors or warnings that might indicate why tasks aren't being processed.

Additionally, ensure your network configuration allows Dify to communicate with the Xinference endpoint, especially if you're self-hosting. Use the host IP address instead of localhost or 127.0.0.1 for the Server URL in Dify's settings. Also, verify the Model UID for the embedding model is correctly set.

If the Worker service is running and the configuration looks correct but the problem persists, reviewing the application logs might reveal more details. These logs can provide insights into the indexing process and help identify any errors or issues that need to be addressed.

For more detailed instructions on configuring and troubleshooting the Worker service, you might want to refer to the source code and documentation available in the Dify repository.

Details

Sources

• A huge of fail request indexing-status when upload knowledge file
• Embedding模型不生效
• en/tutorials/model-configuration/xinference.md
• en/tutorials/model-configuration/README.md
• zh_CN/guides/model-configuration/README.md
• en/tutorials/model-configuration/hugging-face.md
• zh_CN/guides/model-configuration/customizable-model.md
• docker/docker-compose.yaml
• api/docker/entrypoint.sh
• api/.vscode/launch.json
• api/tasks/recover_document_indexing_task.py
• api/tasks/document_indexing_task.py
• api/core/indexing_runner.py
• en/features/logs.md
• zh_CN/guides/logs.md
• api/tasks/retry_document_indexing_task.py
• en/getting-started/install-self-hosted/install-faq.md
• api/core/app/app_config/easy_ui_based_app/model_config/converter.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[dwq370]

i start dify with the flask cmd only
flask run --host 0.0.0.0 --port 5001 --debug

should i start worker service with celery cmd?

[JohnJyong]

both flask and celery are need, the celery worker is used for deal async task
pls follow the doc setp by step : https://docs.dify.ai/getting-started/install-self-hosted/local-source-code

[dwq370]

OK ,i will try it
tks!!!

[dwq370]

i start both api and woker sevice, there is an error when i upload the file to the knowledge

dify version is 0.6.0-fix1

[dwq370]

an error whit version 0.6.4

[dwq370]

i start weaviate with dify version 0.6.4, it's work

[dwq370]

i start weaviate with dify version 0.6.1-fix1, it's work too

thank you for your help

the issue can be closed