[Bug]: Knowledge fetching is very slow

[StanGirard]

What happened?

/knowledge/{knowledge_id}

https://quivr-brain.sentry.io/performance/python-fastapi:088c06d18e5c4cec88815e95b960f44a/?project=4505455607808000&query=http.method:DELETE&statsPeriod=24h&transaction=/knowledge/%7Bknowledge_id%7D&unselectedSeries=p100()&unselectedSeries=avg()

Relevant log output

Look at knowledge in a brain with multiple files

Twitter / LinkedIn details

No response

[dosubot]

🤖

Hello @StanGirard, it's good to see you back on our repository. I hope you've been doing well since our last interaction.

I understand that you're experiencing slow performance when fetching knowledge from a brain with multiple files. This seems to be a performance issue that might be related to the method used for fetching knowledge in the Quivr codebase.

Currently, the method used for fetching knowledge is through the use of the React hook useContext with KnowledgeToFeedContext as its argument. This context is then used to fetch the knowledge data. The useKnowledgeToFeedContext function returns the context along with two additional functions addKnowledgeToFeed and removeKnowledgeToFeed which are used to manipulate the knowledge data. You can find this in the frontend/lib/context/KnowledgeToFeedProvider/hooks/useKnowledgeToFeedContext.tsx file.

There was a similar issue reported earlier, Takes too long to index files, which was updated on August 27, 2023. However, I couldn't find the specific changes made to resolve this issue in the repository.

To better assist you, could you provide more details about the issue? Specifically, it would be helpful to know:

• The size and number of files in the brain where you're experiencing this issue.
• The specific steps you're taking when the issue occurs.
• Any error messages you're seeing in your console or logs.

This information will help us understand the issue better and find a suitable solution.

Sources

• Takes too long to index files
• frontend/lib/context/KnowledgeToFeedProvider/hooks/useKnowledgeToFeedContext.tsx