[BUG] Cannot create 'Knowledge': Failed to create or get collection

[AlemaoEc]

Description

I am following the documentation here: https://docs.crewai.com/concepts/knowledge#what-is-knowledge

Steps to Reproduce

crew = Crew(
            tasks=[task],
            process=Process.sequential,
            verbose=True,
            knowledge_sources: [
              PDFKnowledgeSource(
                  file_paths=[local_path],
                  chunk_size=1000,  # Tamanho dos chunks para processamento
                  chunk_overlap=100,  # Sobreposição entre chunks para preservar contexto
                  metadata={
                    'source': 'Informações sobre a clínica',
                    'description': 'Documento com processos e procedimentos da clínica'
                  }
              )
            ],
            embedder={
              "provider": "openai",
              "config": {
                "model": "text-embedding-3-small",
                "dimensions": 256
              }
            }
        )

When I create a Crew or Agent that has a parameter for knowledge_sources and then run the agent I get the error.

  File "<path>/crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/agent.py", line 140, in post_init_setup
    self._set_knowledge()
  File "<path>/crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/agent.py", line 246, in _set_knowledge
    self._knowledge = Knowledge(
                      ^^^^^^^^^^
  File "<path>crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/knowledge/knowledge.py", line 43, in __init__
    self.storage.initialize_knowledge_storage()
  File "<path>/crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/knowledge/storage/knowledge_storage.py", line 107, in initialize_knowledge_storage
    raise Exception("Failed to create or get collection")
Exception: Failed to create or get collection

Expected behavior

That the knowledge_source was created and could be accessed by the agent or Crew.

Screenshots/Code snippets

See above.

Operating System

macOS Monterey

Python Version

3.11

crewAI Version

0.100.1

crewAI Tools Version

None

Virtual Environment

Venv

Evidence

File "/crewai_support_agent/.venv/lib/python3.12/site-packages/pydantic/main.py", line 214, in init
validated_self = self.pydantic_validator.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/agent.py", line 140, in post_init_setup
self._set_knowledge()
File "/crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/agent.py", line 246, in _set_knowledge
self._knowledge = Knowledge(
^^^^^^^^^^
File "crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/knowledge/knowledge.py", line 43, in init
self.storage.initialize_knowledge_storage()
File "/crewai_support_agent/.venv/lib/python3.12/site-packages/crewai/knowledge/storage/knowledge_storage.py", line 107, in initialize_knowledge_storage
raise Exception("Failed to create or get collection")
Exception: Failed to create or get collection

Possible Solution

I don't know.

Additional context

Same error: #1859 (comment)

[Vidit-Ostwal]

I guess this has been fixed, can you try updating the crew version.
Check out this PR #2055

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[Vidit-Ostwal]

Hi @AlemaoEc, did this worked for you?
Can you check once!
Thanks

[AlemaoEc]

Hi @AlemaoEc, did this worked for you? Can you check once! Thanks

Hi! It didn't work. I migrated the solution to another library.

[GabrielBoninUnity]

Same issue here. When I put the knowledge_sources inside the agent, it gives me the same error, but when I put the knowledge_sources inside the crew, I get no error.

[Vidit-Ostwal]

Hi @GabrielBoninUnity,
Can you confirm you are using the latest version of crewai?

Also possible please share your code, will try to recreate the bug, thanks.

[GabrielBoninUnity]

Hey @Vidit-Ostwal , looks like i got it fix like this here

`import os
from crewai import Agent, Crew, Process, Task, LLM
from crewai.project import CrewBase, agent, crew, task
from crewai.knowledge.source.csv_knowledge_source import CSVKnowledgeSource
from dotenv import load_dotenv

Load environment variables

load_dotenv()

@crewbase
class newsletter:
"""newsletter crew"""

agents_config = "config/agents.yaml"
tasks_config = "config/tasks.yaml"

GOOGLE_CLOUD_PROJECT = os.getenv("GOOGLE_CLOUD_PROJECT")
GOOGLE_CLOUD_LOCATION = os.getenv("GOOGLE_CLOUD_LOCATION")
CLAUDE_MODEL = os.getenv("CLAUDE_MODEL")

# Set up LLM using CrewAI's built-in Vertex AI integration
llm = LLM(
    model=f"vertex_ai/{CLAUDE_MODEL}",
    vertex_ai_project=GOOGLE_CLOUD_PROJECT,
    vertex_ai_location=GOOGLE_CLOUD_LOCATION,
    temperature=0.7,
    credentials=None  # Uses default Google Cloud credentials
)

csv_source = CSVKnowledgeSource(
    file_paths=["data.csv"],
)

@agent
def researcher(self) -> Agent:
    return Agent(
        config=self.agents_config["researcher"],
        verbose=True,
        llm=self.llm,
        knowledge_sources = [self.csv_source],
    )

@task
def research_task(self) -> Task:
    return Task(
        config=self.tasks_config["research_task"],
        output_file="outputs/research_results.md"
    )

@crew
def crew(self) -> Crew:
    """Creates the newsletter crew"""

    return Crew(
        agents=self.agents,
        tasks=self.tasks,
        process=Process.sequential,
        verbose=True,
        llm=self.llm
    )

`

[Vidit-Ostwal]

Hi @GabrielBoninUnity, can you explain what exactly you changed.

[GabrielBoninUnity]

I added the llm code like this

llm = LLM(
model=f"vertex_ai/{CLAUDE_MODEL}",
vertex_ai_project=GOOGLE_CLOUD_PROJECT,
vertex_ai_location=GOOGLE_CLOUD_LOCATION,
temperature=0.7,
credentials=None # Uses default Google Cloud credentials
)

and also tagged the agent with it

like this
@agent
def researcher(self) -> Agent:
return Agent(
config=self.agents_config["researcher"],
verbose=True,
llm=self.llm,
knowledge_sources = [self.csv_source],