[Feature]: Support for OpenAIEmbeddings with Langchain

[yuhon0528]

🚀 The feature, motivation and pitch

I have hosted e5-mistral-7b-instruct with vllm OpenAI compatible APIs and it can be accessed by Posting to http://localhost:8000/v1/embeddings with:

{
    "model": "e5-mistral-7b-instruct",
    "input":["A sentence to encode."]
}

However, it seems cannot be accessed through Langchain with:

from langchain_openai import OpenAIEmbeddings

emb_model = OpenAIEmbeddings(
    model="e5-mistral-7b-instruct",
    openai_api_base="http://localhost:8000/v1",
    openai_api_key="EMPTY")

emb_model.embed_query("A sentence to encode.")

Error received:

openai.BadRequestError: Error code: 400 - {'object': 'error', 'message': 'base64 encoding is not currently supported', 'type': 'BadRequestError', 'param': None, 'code': 400}

Will it be supported in Langchain? Or have I done anything wrong?

Alternatives

No response

Additional context

No response

[avnerlipszyc]

Having the same issue this morning...

[damienw34]

same since a few minutes too.

[suanmiao]

Hey guys, I just found the cause for the issue:

• OpenAI SDK's embedding function supports encoding_format in base64 or float. Here is their interface: https://github.com/openai/openai-python/blob/f3e6e634a86d5789ab1274ae27f43adc842f4ba8/src/openai/types/embedding_create_params.py#L39
• Seems the vLLM embedding API doesn't support base64 encoding yet. So if you add the parameter encoding_format="float" to the embedding creation, it will work.

Here is the complete code:

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

# Sample prompts.
prompts = [
    "Hello, my name is",
]

responses = client.embeddings.create(input=prompts,model=model, encoding_format="float")

for data in responses.data:
    print(data.embedding)  

[damienw34]

thanks a lot ! it solved the problem for me.

[avnerlipszyc]

Thank you!

[mgoin]

Appreciate the quick work in debugging! :) @suanmiao Would you mind opening an issue for vLLM to support the default encoding_float="base64"? We should implement this

[simon-mo]

@mgoin it's going to be a good first issue!

[Etelis]

I'm on this :)

[ShantanuVichare]

I'm on this :)

Hi @Etelis , I'm new to open source development and found this issue as a good starting point. Do you think we can work on this together if you're interested?

[Etelis]

I'm on this :)

Hi @Etelis , I'm new to open source development and found this issue as a good starting point. Do you think we can work on this together if you're interested?

Of course! Go ahead and start working on it. We can merge both of our modifications once you're done. By the way, I think the modification needed here is slight, so there might not be too much to do.

[llmpros]

Hi folks @Etelis @DarkLight1337
based on my tests (on latest version 0.5.0post1), it seems to me that we just need to remove the "blocker message" (without additional code changes) and both float and base64 will work smoothly.

#5935

Please let me know your test results :) thanks

[DarkLight1337]

Closed as completed by #5935

[mujhenahiata]

Both these methods generate different embedding for the same text

[DarkLight1337]

Both these methods generate different embedding for the same text

Please open a new issue and provide more details, such as a script showing both methods.