Skip to content

[Bug]: ChromaDB 0.5.0 ERROR:sqlite3.OperationalError: unable to open database file #4039

Open
@Novakkkk

Description

@Novakkkk

What happened?

The service is deployed on a Linux system. Initially, the service ran without any issues. However, after we increased the scale of the database, problems began to occur. The service now frequently crashes with the same error(sqlite3.OperationalError: unable to open database file). The current situation is that the service can start normally, but after some time (erratically), it crashes.
Some actions have been done to reproduce the error,but we failed:

  1. changed the relative database file path to the absolute ones.
  2. the database connection can be closed.
  3. permission issue is not existed

BTW:

  1. we observed that after a quering action, it seems that the program does not actually close the connection of DB.
  2. sometimes, before the sqlite3.OperationalError, OSError: [Errno 24] Too many open files [Bug]: OSError: [Errno 24] Too many open files:  #1379 occured first. BUT it would not occur every time before the sqliteOperational Error.

Addition:We hope fix it without any upgrading of chroma. We noticed that a former upgrade have fixed a connection bug(mentioned in OSError: [Errno 24] Too many open files issue).On the one hand, we can not make sure that this connection bug is actually the key to fix our problem. On the other hand, upgrading the version costs too much for us.

Thanks for help

Versions

ChromaDB: v0.5.0
python:v3.11.0

Relevant log output

File "/data/lib/python3.11/site-packages/chromadb/api/models/Collection.py", line 195, in query
  File "/data/lib/python3.11/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 146, in wrapper
  File "/data//lib/python3.11/site-packages/chromadb/rate_limiting/__init__.py", line 47, in wrapper
  File "/data/lib/python3.11/site-packages/chromadb/api/segment.py", line 754, in _query
  File "/data/lib/python3.11/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 146, in wrapper
  File "/data/lib/python3.11/site-packages/chromadb/segment/impl/metadata/sqlite.py", line 214, in get_metadata
  File "/data/lib/python3.11/site-packages/chromadb/db/impl/sqlite.py", line 132, in tx
  File "/data/lib/python3.11/site-packages/chromadb/db/impl/sqlite.py", line 31, in __init__
  File "/data/lib/python3.11/site-packages/chromadb/db/impl/sqlite_pool.py", line 141, in connect
  File "/data/lib/python3.11/site-packages/chromadb/db/impl/sqlite_pool.py", line 20, in __init__
sqlite3.OperationalError: unable to open database file

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions