Remove sentence-transformers as a hard requirement

[jeffchuber]

Currently we use sentence-transformers as the default embedding model. However this means that it, and a lot of it's deps are included in the project. Additionally it downloads the model on start-up, which hurts startup time. Furthermore it makes Chroma not installable on certain envs, like Python 3.11.

Will close

• pip install chromadb fails with python 3.11.0 #163

[Tanzengeist]

Is there any workaround for using chromadb with python 3.11x? I have a VScode environment that is working well and I don't want to mess with it (still a newbie). I have been writing text-based AI code using chromadb in Colab but there are local modes like the microphone and speaker that I need to use.

[jeffchuber]

@Tanzengeist we are prioritizing this and will followup later today

[ayush-vibrant]

@jeffchuber Eagerly waiting for the solution. In the meantime, what alternative you recommend so I can use chromadb in my codebase?

[Tanzengeist]

Jeff, I’m sure your all working hard on this. When you have a workaround, please send up a flare.

[HammadB]

#267 removes sentence-transformers, but unfortunately will still not unblock 3.11 as onnxruntime does not yet support it. With major packages like onnx and pytorch not supporting 3.11, it is hard for us to deliver models to users and support 3.11 until these dependencies do :(

[kotakcloud]

Works fine with: ARCHFLAGS="-arch x86_64" pip install chromadb
See if that's any useful.

Reference: Yale-LILY/SummerTime#116 (comment)

[RiccardoGrin]

Any updates on removing sentence-transformers as a hard requirement?

[DiegoPiloni]

Hi ! I'm interested in this solution. Do we have a workaround before this is released ?

[specter119]

Hi, the project seems not hard dependent on sentence-transformers, will this dependency be removed in the requirements?

[jeffchuber]

@specter119 yes in two ways.

1. the default bundling will be switched to the trimmed down ONNX model Switch to ONNX model for default embedding model #267
2. we will ship a client-only build of chroma as a separate pypi project

both very soon

[specter119]

@jeffchuber thx, sentence-transformers brings a heavy dependency, which causes the Conda build not pass.

• [bot-automerge] chromadb v0.3.23 conda-forge/chromadb-feedstock#6
• Failed build

BTW, will the vector storage related features in LangChain are dependent on both server and client of chroma?

[jeffchuber]

Good to know, im glad we are removing that.

Langchain by default uses the in-memory version of chroma which is more of a library than a client or a server.

[jeffchuber]

chroma-client fixed this. https://pypi.org/project/chromadb-client/ I think for most users