Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

导入文本报错 #209

Closed
chanjetsdp opened this issue May 25, 2023 · 5 comments
Closed

导入文本报错 #209

chanjetsdp opened this issue May 25, 2023 · 5 comments

Comments

@chanjetsdp
Copy link

导入文本文件,报错
docker-worker-1 | [2023-05-25 13:08:12,771: INFO/MainProcess] Task tasks.document_indexing_task.document_indexing_task[d5d942ff-834e-4ae1-a707-fa5e398d97a2] received
docker-worker-1 | [2023-05-25 13:08:12,772: INFO/MainProcess] Start process document: d1a75d51-1024-499f-82c3-7a3f673791a5
docker-worker-1 | [2023-05-25 13:08:12,778: ERROR/MainProcess] consume document failed
docker-worker-1 | Traceback (most recent call last):
docker-worker-1 | File "/app/api/tasks/document_indexing_task.py", line 41, in document_indexing_task
docker-worker-1 | indexing_runner.run(document)
docker-worker-1 | File "/app/api/core/indexing_runner.py", line 48, in run
docker-worker-1 | text_docs = self._load_data(document)
docker-worker-1 | File "/app/api/core/indexing_runner.py", line 217, in _load_data
docker-worker-1 | text_docs = self._load_data_from_file(file_detail)
docker-worker-1 | File "/app/api/core/indexing_runner.py", line 246, in _load_data_from_file
docker-worker-1 | self.storage.download(upload_file.key, filepath)
docker-worker-1 | File "/app/api/extensions/ext_storage.py", line 83, in download
docker-worker-1 | raise FileNotFoundError("File not found")
docker-worker-1 | FileNotFoundError: File not found

Dify version: Self Host

Steps To Reproduce

  1. 0.2.1

  2. docker-compose ps
    docker-compose ps
    NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
    docker-api-1 langgenius/dify-api:latest "/entrypoint.sh" api About an hour ago Up About an hour 80/tcp, 5001/tcp
    docker-db-1 postgres:15-alpine "docker-entrypoint.s…" db About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
    docker-nginx-1 nginx:latest "/docker-entrypoint.…" nginx About an hour ago Up About an hour 0.0.0.0:80->80/tcp, :::80->80/tcp
    docker-redis-1 redis:6-alpine "docker-entrypoint.s…" redis About an hour ago Up About an hour 6379/tcp
    docker-weaviate-1 semitechnologies/weaviate:1.18.4 "/bin/weaviate --hos…" weaviate About an hour ago Up About an hour
    docker-web-1 langgenius/dify-web:latest "/entrypoint.sh" web About an hour ago Up About an hour 80/tcp, 3000/tcp
    docker-worker-1 langgenius/dify-api:latest "/entrypoint.sh" worker About an hour ago Up About an hour 80/tcp, 5001/tcp

The current behavior

导入文本
image
image

The expected behavior

@takatost
Copy link
Collaborator

Have you pulled the latest image yet? You can try execute docker-compose pull in the docker directory of the project, then redeploy.
Or have you modified the volumes config in docker-compose.yaml? The storage volumes of service api and worker must be the same.

@chanjetsdp
Copy link
Author

docker-compose.yaml

yes, yesterday I download dify-0.2.1,
I user the default docker-compose.yaml, and only change URL , COOKIE , CORS envs.

@chanjetsdp
Copy link
Author

chanjetsdp commented May 26, 2023

after docker-compose pull
the error still here

langgenius/dify-web latest d38379a2304f 9 hours ago 1.93GB
langgenius/dify-api latest 9df74addc611 9 hours ago 1.56GB


grep -A 3  volumes docker-compose.yaml 
    volumes:
      # Mount the storage directory to the container, for storing user files.
      - ./volumes/app/storage:/app/storage

  # worker service
  # The Celery worker for processing the queue.
--
    volumes:
      # Mount the storage directory to the container, for storing user files.
      - ./volumes/app/storage:/app/storage

  # Frontend web application.
  web:
--
    volumes:
      - ./volumes/db/data:/var/lib/postgresql/data
      - ./volumes/db/scripts:/docker-entrypoint-initdb.d/
    ports:
      - "5432:5432"

--
    volumes:
      # Mount the redis data directory to the container.
      - ./volumes/redis/data:/data
    # Set the redis password when startup redis server.
    command: redis-server --requirepass difyai123456

--
    volumes:
      # Mount the Weaviate data directory to the container.
      - ./volumes/weaviate:/var/lib/weaviate
    environment:
      # The Weaviate configurations
      # You can refer to the [Weaviate](https://weaviate.io/developers/weaviate/config-refs/env-vars) documentation for more information.
--
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/proxy.conf:/etc/nginx/proxy.conf
      - ./nginx/conf.d:/etc/nginx/conf.d





logs


docker-nginx-1     | 10.1.174.10 - - [26/May/2023:01:08:58 +0000] "GET /console/api/datasets/process-rule HTTP/1.1" 200 208 "http://172.16.150.9/datasets/a4fe2e4a-4fad-402f-bc91-7e4895ec6685/documents/create" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" "-"
docker-nginx-1     | 10.1.174.10 - - [26/May/2023:01:08:58 +0000] "POST /console/api/datasets/file-indexing-estimate HTTP/1.1" 200 1643 "http://172.16.150.9/datasets/a4fe2e4a-4fad-402f-bc91-7e4895ec6685/documents/create" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" "-"
docker-nginx-1     | 10.1.174.10 - - [26/May/2023:01:08:58 +0000] "POST /console/api/datasets/file-indexing-estimate HTTP/1.1" 200 1643 "http://172.16.150.9/datasets/a4fe2e4a-4fad-402f-bc91-7e4895ec6685/documents/create" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" "-"


。。。。。


docker-nginx-1     | 10.1.174.10 - - [26/May/2023:01:10:26 +0000] "GET /console/api/datasets/a4fe2e4a-4fad-402f-bc91-7e4895ec6685/related-apps HTTP/1.1" 200 25 "http://172.16.150.9/datasets/a4fe2e4a-4fad-402f-bc91-7e4895ec6685/documents/create" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" "-"
docker-nginx-1     | 10.1.174.10 - - [26/May/2023:01:10:27 +0000] "GET /console/api/version?current_version=0.3.0 HTTP/1.1" 200 116 "http://172.16.150.9/datasets/a4fe2e4a-4fad-402f-bc91-7e4895ec6685/documents/create" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" "-"
docker-worker-1    | [2023-05-26 01:10:28,162: INFO/MainProcess] Task tasks.document_indexing_task.document_indexing_task[40343c29-de97-4a85-95bd-2d70a4626fcd] received
docker-worker-1    | [2023-05-26 01:10:28,162: INFO/MainProcess] Start process document: 226abe15-7ecf-4f97-bea7-0eb5b7b3bfc1
docker-worker-1    | [2023-05-26 01:10:28,169: ERROR/MainProcess] consume document failed
docker-worker-1    | Traceback (most recent call last):
docker-worker-1    |   File "/app/api/tasks/document_indexing_task.py", line 41, in document_indexing_task
docker-worker-1    |     indexing_runner.run(document)
docker-worker-1    |   File "/app/api/core/indexing_runner.py", line 48, in run
docker-worker-1    |     text_docs = self._load_data(document)
docker-worker-1    |   File "/app/api/core/indexing_runner.py", line 217, in _load_data
docker-worker-1    |     text_docs = self._load_data_from_file(file_detail)
docker-worker-1    |   File "/app/api/core/indexing_runner.py", line 246, in _load_data_from_file
docker-worker-1    |     self.storage.download(upload_file.key, filepath)
docker-worker-1    |   File "/app/api/extensions/ext_storage.py", line 83, in download
docker-worker-1    |     raise FileNotFoundError("File not found")
docker-worker-1    | FileNotFoundError: File not found


@chanjetsdp
Copy link
Author

chanjetsdp commented May 26, 2023

image

image

upload files in api container :
image

@chanjetsdp
Copy link
Author

chanjetsdp commented May 26, 2023

this problem. docker-compose.yaml is in dify-0.2.1

change volumes path
- ./volumes/app/storage:/app/storage

to

- ./volumes/app/storage:/app/api/storage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants