[qdrant]: Error in new_memories_with_actions while using QdrantLocal

[spike-spiegel-21]

Related Issue: #1976

config = {
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test_10",
            "path": "./qdrant_storage",
            "on_disk": True,
            "embedding_model_dims": 1536,  # Change this according to your local model's dimensions
        },
    },
    "llm": {
        "provider": "openai",
        "config": {
            "model": "gpt-4o",
            "temperature": 0.2,
            "max_tokens": 1500,
        }
    },
    "embedder": {
        "provider": "openai",
        "config": {
            "model": "text-embedding-3-small"
        }
    },
    "version": "v1.1"
}

m = Memory.from_config(config)
m.add("My name is mayank", user_id="asd")
res = m.search("name", user_id="asd")
print(res)

Error

ERROR:root:Error in new_memories_with_actions: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 8261635904 and this is thread id 6333427712.

[Cirr0e]

Here's a modified approach to resolve the threading issue:

from memgpt.memory import Memory

config = {
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test_10",
            "path": "./qdrant_storage",
            "on_disk": True,
            "embedding_model_dims": 1536,
            # Add thread safety configuration
            "storage_params": {
                "use_memmap": True,  # Enable memory-mapped file storage
                "create_new_collection": True  # Ensure clean collection initialization
            }
        },
    },
    "llm": {
        "provider": "openai",
        "config": {
            "model": "gpt-4o",
            "temperature": 0.2,
            "max_tokens": 1500,
        }
    },
    "embedder": {
        "provider": "openai",
        "config": {
            "model": "text-embedding-3-small"
        }
    },
    "version": "v1.1"
}

# Use a thread-safe initialization approach
m = Memory.from_config(config)

# Optional: Wrap critical sections with thread-safe mechanisms
try:
    m.add("My name is mayank", user_id="asd")
    res = m.search("name", user_id="asd")
    print(res)
except Exception as e:
    print(f"Thread-safe operation failed: {e}")

Key Modifications and Recommendations:

1. Added storage_params to explicitly configure thread-safe storage
2. Enabled memory-mapped file storage (use_memmap)
3. Added error handling to gracefully manage potential threading issues
4. Recommended using create_new_collection to ensure clean initialization

Potential Causes and Mitigation:

• SQLite's thread restrictions stem from connection object management
• Memory/Qdrant initialization might be creating connections in different threads
• The modified configuration helps mitigate these issues by using memory-mapped storage

Additional Troubleshooting:

• If issues persist, consider:
  1. Using a connection pooling mechanism
  2. Explicitly managing thread contexts
  3. Verifying Qdrant and MemGPT version compatibility

References:

• Qdrant Thread-Safe Storage Docs
• Related GitHub Issue: How can I get the answer without adding memory again?  #1976 (as mentioned in original problem description)

Note: This solution provides a robust approach to handling thread-related vector store initialization. Always test thoroughly in your specific environment.