How to configure RAG for Azure OpenAI?

[dareima]

Hi everyone,

today we've noticed that RAG only worked since we've added an OpenAI API Key to our .env file.

As our main endpoints are in Microsoft Azure OpenAI, we'd like to utilize our embeddings model there.
We gave the information for other providers provided in https://www.librechat.ai/docs/configuration/rag_api a go but weren't succesful.

Can someone describe which configurations we need to put in place to switch RAG to Azure OpenAI?

Any help would be much appreciated, thanks!

[dareima]

All good. Figured out how to do it, thanks to the documentation in the rag project: https://github.com/danny-avila/rag_api

Setting these params fixed it for us:

RAG_AZURE_OPENAI_API_KEY=nnn
RAG_AZURE_OPENAI_ENDPOINT=https://ourresrouce.openai.azure.com
EMBEDDINGS_PROVIDER=azure
EMBEDDINGS_MODEL=text-embedding-ada-002

In love with LibreChat @danny-avila 💯

[jmariquit-tvlk]

I tried this but am getting this error (note am not using OPENAI but using Azure OpenAI instead):

Value error, Must provide either the api_version argument or the OPENAI_API_VERSION environment variable [type=value_error, input_value={'azure_deployment': 'tex...one, 'model_kwargs': {}}, input_type=dict]

[dareima]

Hard to tell the cause without knowing more about the rest of your Azure OpenAI setup in LibreChat.

According to this error LibreChat is missing an API version setting. As API versions for the models are no longer set in .env but should be set in librechat.yaml, it will be good to see how your model setup there looks like.

Also, are you using modelSpecs or are you going without them?

[jmariquit-tvlk]

thanks for the input. this is what I have in my librechat.yaml (masked the values of the names and actual keys):

endpoints:
  azureOpenAI:
    titleModel: "current_model"
    plugins: true
    assistants: true
    titleConvo: true
    groups:
      - group: "xxx-yyy"
        apiKey: "xxx...yyy"
        instanceName: "xxx-yyy"
        version: "2024-05-01-preview"
        assistants: true
        models:
          gpt4o-2024-05-01:
            deploymentName: "chatgpt"

as for modelSpecs, I don't have anything defined. the endpoint definition does work though (i.e. I can chat with the model no issues except if I attach a file for the rag)

taking a look at the link I again, I will try to specify the RAG_AZURE_OPENAI_API_VERSION= var. will also play around with the modelSpecs.

[jmariquit-tvlk]

it worked! I was missing the RAG_AZURE_OPENAI_API_VERSION

I made no changes to my librechat.yaml (as above) and I did not need to look at modelSpecs either.
after i added the RAG_AZURE_OPENAI_API_VERSION to my .env it was able to connect properly.

here's my .env entry for those trying to get the RAG to work with Azure

# text-embedding-ada-002 and text-embedding-3-small also works (as long as you have them deployed as well)
EMBEDDINGS_PROVIDER=azure
EMBEDDINGS_MODEL=text-embedding-3-large

# replace with your key and endpoint of course
RAG_AZURE_OPENAI_API_KEY=xxx....yyy
RAG_AZURE_OPENAI_ENDPOINT=https://???.openai.azure.com

# endpoint had 2023-05-15 as a default but mine worked as well
RAG_AZURE_OPENAI_API_VERSION=2024-05-01-preview

thanks again.

[dareima]

it worked! I was missing the RAG_AZURE_OPENAI_API_VERSION

I made no changes to my librechat.yaml (as above) and I did not need to look at modelSpecs either. after i added the RAG_AZURE_OPENAI_API_VERSION to my .env it was able to connect properly.

here's my .env entry for those trying to get the RAG to work with Azure

# text-embedding-ada-002 and text-embedding-3-small also works (as long as you have them deployed as well)
EMBEDDINGS_PROVIDER=azure
EMBEDDINGS_MODEL=text-embedding-3-large

# replace with your key and endpoint of course
RAG_AZURE_OPENAI_API_KEY=xxx....yyy
RAG_AZURE_OPENAI_ENDPOINT=https://???.openai.azure.com

# endpoint had 2023-05-15 as a default but mine worked as well
RAG_AZURE_OPENAI_API_VERSION=2024-05-01-preview

thanks again.

Very nice, thanks for the update.
Interesting that we don't have to use RAG_AZURE_OPENAI_API_VERSION=2024-05-01-preview in our .env.

The only difference I can spot at a first glance is that we're also defining each model API version directly in librechat.yaml.

Like this:

endpoints:
  azureOpenAI:
    titleModel: "current_model"
    plugins: true
    assistants: true
    groups:
      - group: "swe-central" # arbitrary name
        apiKey: "${AZURE_OPEN_AI_API_KEY}"
        assistants: true
        instanceName: "aihubswe1234567890"
        version: "2024-05-01-preview"
        models:
          swe-gpt-4o:
            deploymentName: swe-gpt-4o
            version: "2024-02-15-preview"
          swe-gpt-4o-mini:
            deploymentName: swe-gpt-4o-mini
            version: "2024-02-15-preview"

I wouldn't have thought this might make a difference in regards to RAG.
Anyway, good to see it's working for you.