community: upgrade openai>=1.30 for LocalAIEmbedings

[mkhludnev]

Thank you for contributing to LangChain!

• PR title: "package: description"

• Description: migrate LocalAIEmbeddings code to OpenAI 1.30

• Issue: LocalAIEmbeddings fails due to old OpenAI 0.28 code openai.error see module 'openai' has no attribute 'error' #13368

• Dependencies: OpenAI is already bumped 1.30

• Add tests and docs: integration test with vcr

• Lint and test:

Additional guidelines:

• Make sure optional dependencies are imported within a function.
• Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
• Most PRs should not touch more than one package.
• Changes should be backwards compatible.
• If you are adding something to community, do not re-import it in langchain.

If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Sep 22, 2024 2:46pm

[baskaryan]

shouldn't need to pass in client directly

[mkhludnev]

@baskaryan I think it should be
openai/openai-python#825 (comment)
https://github.com/openai/openai-python#configuring-the-http-client

[mkhludnev]

eg. I don't see actual proxy usage in https://github.com/langchain-ai/langchain/blob/master/libs/partners/openai/langchain_openai/embeddings/base.py

[mkhludnev]

shouldn't need to pass in client directly

I copied openai_proxy handling from OpenAIEmbedding

[mkhludnev]

I got back to the problem.
Let me ask very basic question: which version of openai we expect for langchain_community 1.x or 0.x?
We can assume that 90% of users have both langchain_community and langchain-openai in their dependencies.
Although I don't know whether langchain-openai is able to work with openai 0.x.

[mkhludnev]

@baskaryan kindly asking to check this PR out. I resolved the question regarding proxy. Note, after this PR is merged. I want to improve #22666 localai embeddings performance.

[mkhludnev]

ahhhh it's a bummer

       we can conclude that friendli-client>=1.2.4,<2 depends on
      httpx>=0.24.1,<0.25.0.
      And because you require friendli-client>=1.2.4,<2 and httpx>=0.27.0, we
      can conclude that your requirements are unsatisfiable.

UPD managed to keep up with old httpx

[mkhludnev]

ping @efriis pleaase

[dawu415]

Curious if there's any update on this or alternative solutions in the meantime?

[mkhludnev]

Curious if there's any update on this or alternative solutions in the meantime?

I can only rebase it from time to time. Let me know if it's necessary. I'll refresh.

[efriis]

Unfortunately there isn't a great way to update these old community integrations that rely on openai sdk v0 in a way that's both backwards compatible + supports v1 without being a huge mess to maintain. This is one of the main motivations behind moving towards dedicated integration packages, which have replaced the community package as the right path forward. I'll close this PR, and would recommend reopening with just docs updates, as well as registering your package in libs/packages.yml! We'll be able to review simple PRs that only modify these two things much faster :)

Would also love to @deprecated this integration pointing to your new integration package, which can even lock to openai v1 sdk to avoid support for v0!

Here's the guide, and if you have questions, feel free to leave them in the comments on those pages so others can see them! https://python.langchain.com/docs/contributing/how_to/integrations/

[mkhludnev]

Thanks @efriis for checking stale PRs during the holiday season!
I read it as a need to copy LocalAI Embeddings to the separate repo that I did https://github.com/mkhludnev/langchain-localai/

The first question is, which account this integration package will belong to:

• can you adopt it into https://github.com/langchain-ai/langchain-localai ?
• or @mudler you can kindly keep it under https://github.com/mudler/langchain-localai ?
• I think langchain/libs/partners/langchain-localai isn't a preferable option anymore?

UPD: I've fixed the build.

PS. The next (real target) is a performance via #22666

[efriis]

we won't create one under langchain-ai, and one under mudler would probably be best as an integration package.

However, that being said, because it relies on the openai sdk anyways, I would strongly recommend just rewriting the Local AI docs page in langchain to use the ChatOpenAI integration instead. That way it benefits from all fixes folks make against that package, and we can deprecate ChatLocalAI in favor of just using that.

[mkhludnev]

However, that being said, because it relies on the openai sdk anyways, I would strongly recommend just rewriting the Local AI docs page

aaaghh I've got your point after all. Let me check if embeddings work this way!

[mkhludnev]

@efriis
I tried to invoke LocalAI embeddings via OpenAIEmbeddings as far as I've got your suggestion.

embeddings = OpenAIEmbeddings(
    model="bge-m3",
    openai_api_base="https://localai.my/v1",
    api_key="foo bar")
vectors = embeddings.embed_documents(["lorem ipsum","dolor sit amet"])

I noticed that underneath OpenAIEmbeddings tokenizes text by cl100k_base and submits int array per the given string. LocalAI handles it somehow and yields vectors. Probably they are meaningful, but their values are different from what I've got from LocalAIEmbeddings, which submits texts as-is.
So, even if we got confirmation that "intput":[[1,2,3,..],[4,5,6]] a valid way to call https://localai.io/features/embeddings/ I don't think users were encouraged by need to reindex all vectors just due to client upgrade.
So, wdyt? Is there a country for LocalAIEmbeddings?

[mkhludnev]

ChatLocalAI

Let me clarify. There's no chat model for LocalAI here, there's only LocalAIEmbeddings

[efriis]

I see makes sense. Feel free to publish a langchain-localai then!