LocalAI embeddings shouldn't require OpenAI

[jamesbraza]

System Info

macOS Ventura 13.5.2, M1

Who can help?

@mudler

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

https://github.com/langchain-ai/langchain/blob/v0.0.298/libs/langchain/langchain/embeddings/localai.py#L197

Expected behavior

Why does LocalAI embeddings require OpenAI? I think LocalAI's embeddings has no need for OpenAI, it has a whole embeddings suite: https://localai.io/features/embeddings/

I think it should be directly usable with its /embeddings endpoint

[dosubot]

🤖

Hello,

Thank you for bringing up this question. The LocalAI embeddings in LangChain do indeed require OpenAI, but not for the reasons you might think. The OpenAI package is used for error handling and retrying requests, and the OpenAI's embedding endpoint is directly used for embedding both search documents and query texts. This is evident in the embed_documents, aembed_documents, embed_query, and aembed_query methods in the openai.py file. These methods call out to OpenAI's embedding endpoint to get the embeddings for the provided texts.

However, you're correct that LocalAI has its own embeddings suite. LangChain supports this through the LocalAIEmbeddings class, which can be used to embed documents and queries directly using LocalAI's /embeddings endpoint. This class requires the LocalAI service to be hosted somewhere and the embedding models to be configured.

If you're interested in using LocalAI's embeddings suite directly, you can do so by using the LocalAIEmbeddings class. Here's an example of how you can do this:

from langchain.embeddings.localai import LocalAIEmbeddings
...
embeddings = LocalAIEmbeddings()

You can find more information about this in the LangChain documentation.

I hope this answers your question. If you have any more questions or need further clarification, feel free to ask.

Best,
Dosu

Sources

• Is it possible to use open source embedding methods rather than OpenAIEmbeddings?
• libs/langchain/langchain/embeddings/localai.py
• libs/langchain/langchain/embeddings/openai.py
• docs/extras/integrations/text_embedding/localai.ipynb
• libs/langchain/langchain/chat_models/openai.py
• libs/langchain/langchain/llms/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jamesbraza]

Thank you @dosu-beta. The docstring in LocalAIEmbeddings is clear about openai being required, with OpenAI keys configured. I think it should be opt-in for added behaviors, but not required by default

[dosubot]

🤖

Hello,

Thank you for bringing this to our attention. You're correct that the LocalAI embeddings should not require OpenAI by default. We can make OpenAI an opt-in requirement by making the import of the openai module optional. This can be done by moving the import statements inside the methods that use them and catching the ImportError exception. If the openai module is not found, we can then decide how to handle this situation, for example by raising a custom exception or logging a warning message.

Here is how you can modify the validate_environment method in the BaseOpenAI, AzureOpenAI, and OpenAIChat classes:

@root_validator()
def validate_environment(cls, values: Dict) -> Dict:
    """Validate that api key and python package exists in environment."""
    values["openai_api_key"] = get_from_dict_or_env(
        values, "openai_api_key", "OPENAI_API_KEY"
    )
    values["openai_api_base"] = get_from_dict_or_env(
        values,
        "openai_api_base",
        "OPENAI_API_BASE",
        default="",
    )
    values["openai_proxy"] = get_from_dict_or_env(
        values,
        "openai_proxy",
        "OPENAI_PROXY",
        default="",
    )
    values["openai_organization"] = get_from_dict_or_env(
        values,
        "openai_organization",
        "OPENAI_ORGANIZATION",
        default="",
    )
    try:
        import openai
        openai.api_key = values["openai_api_key"]
        if values["openai_api_base"]:
            openai.api_base = values["openai_api_base"]
        if values["openai_organization"]:
            openai.organization = values["openai_organization"]
        if values["openai_proxy"]:
            openai.proxy = {"http": values["openai_proxy"], "https": values["openai_proxy"]}  # type: ignore[assignment]  # noqa: E501
    except ImportError:
        raise ImportError(
            "OpenAI is not installed. Please install it with `pip install openai` to use this feature."
        )
    try:
        values["client"] = openai.ChatCompletion
    except AttributeError:
        raise ValueError(
            "`openai` has no `ChatCompletion` attribute, this is likely "
            "due to an old version of the openai package. Try upgrading it "
            "with `pip install --upgrade openai`."
        )
    warnings.warn(
        "You are trying to use a chat model. This way of initializing it is "
        "no longer supported. Instead, please use: "
        "`from langchain.chat_models import ChatOpenAI`"
    )
    return values

In this way, the openai module becomes an opt-in requirement. If it is not installed, the user will be notified when they try to use a feature that requires it. The same approach can be applied to the _generate, _astream, and other methods in the BaseOpenAI, AzureOpenAI, and OpenAIChat classes that use the openai module.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best,
Dosu

Sources

• Is it possible to use open source embedding methods rather than OpenAIEmbeddings?
• libs/langchain/langchain/embeddings/localai.py
• libs/langchain/langchain/embeddings/openai.py
• libs/langchain/langchain/llms/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[mudler]

The LocalAI embeddings are using the OpenAI library as there is 1:1 compatibility with their API. There is no need of an API key, what's the issue here?

[jamesbraza]

Ha you're right, I was definitely reading too quickly. I saw use of openai package, the need to set OPENAI_API_KEY, and all the validations around OpenAI env vars, and I didn't keep reading to see OPENAI_API_KEY was being set to a random string.

I opened #10946 to better document this design choice.

[mkhludnev]

If it's required, let's bump it #22399