[Bug] Retriever microservice with ChatQnA failing on Oracle/IBM/AWS Cloud

[arun-gupta]

Priority

Undecided

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source

Deploy method

• Docker compose
• Docker
• Kubernetes
• Helm

Running nodes

Single Node

What's the version?

latest

Description

Set up ChatQnA on Oracle Cloud following the instructions at https://gist.github.com/arun-gupta/b4c13de906e5fe66024018ba48e3f6c3. Retriever microservice is giving the error:

ubuntu@opea-demo:~$ curl http://${host_ip}:7000/v1/retrieval   -X POST   -d "{\"text\":\"test\",\"embedding\":${your_embedding}}"   -H 'Content-Type: application/json'
curl: (7) Failed to connect to localhost port 7000 after 0 ms: Couldn't connect to server

Reproduce steps

https://gist.github.com/arun-gupta/b4c13de906e5fe66024018ba48e3f6c3

Raw log

ubuntu@opea-demo:~$ sudo docker logs retriever-redis-server
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name" in HuggingFaceInferenceAPIEmbeddings has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name_or_path" in Audio2TextDoc has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[2024-10-14 23:01:17,847] [    INFO] - Base service - CORS is enabled.
[2024-10-14 23:01:17,849] [    INFO] - Base service - Setting up HTTP server
[2024-10-14 23:01:17,850] [    INFO] - Base service - Uvicorn server setup on port 7000
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7000 (Press CTRL+C to quit)
[2024-10-14 23:01:17,857] [    INFO] - Base service - HTTP server setup successful
/home/user/comps/retrievers/redis/langchain/retriever_redis.py:106: LangChainDeprecationWarning: The class `HuggingFaceHubEmbeddings` was deprecated in LangChain 0.2.2 and will be removed in 1.0. An updated version of the class exists in the langchain-huggingface package and should be used instead. To use it run `pip install -U langchain-huggingface` and import as `from langchain_huggingface import HuggingFaceEndpointEmbeddings`.
  embeddings = HuggingFaceHubEmbeddings(model=tei_embedding_endpoint)
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 277, in connect
    sock = self.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/retry.py", line 62, in call_with_retry
    return do()
           ^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 278, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
            ^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 639, in _connect
    raise err
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 627, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/comps/retrievers/redis/langchain/retriever_redis.py", line 111, in <module>
    vector_db = Redis(embedding=embeddings, index_name=INDEX_NAME, redis_url=REDIS_URL)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/langchain_community/vectorstores/redis/base.py", line 309, in __init__
    check_redis_module_exist(redis_client, REDIS_REQUIRED_MODULES)
  File "/home/user/.local/lib/python3.11/site-packages/langchain_community/utilities/redis.py", line 55, in check_redis_module_exist
    installed_modules = client.module_list()
                        ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/commands/core.py", line 6250, in module_list
    return self.execute_command("MODULE LIST")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/client.py", line 545, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 1074, in get_connection
    connection.connect()
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 283, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name" in HuggingFaceInferenceAPIEmbeddings has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name_or_path" in Audio2TextDoc has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[2024-10-14 23:01:27,140] [    INFO] - Base service - CORS is enabled.
[2024-10-14 23:01:27,141] [    INFO] - Base service - Setting up HTTP server
[2024-10-14 23:01:27,142] [    INFO] - Base service - Uvicorn server setup on port 7000
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7000 (Press CTRL+C to quit)

[arun-gupta]

I got the exact same error on IBM Cloud as well:

ubuntu@opea-demo:~$ export your_embedding=$(python3 -c "import random; embedding = [random.uniform(-1, 1) for _ in range(768)]; print(embedding)")
  curl http://${host_ip}:7000/v1/retrieval \
    -X POST \
    -d "{\"text\":\"test\",\"embedding\":${your_embedding}}" \
    -H 'Content-Type: application/json'
curl: (7) Failed to connect to localhost port 7000 after 0 ms: Couldn't connect to server

Here is the log:

ubuntu@opea-demo:~$ sudo docker logs retriever-redis-server
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name" in HuggingFaceInferenceAPIEmbeddings has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name_or_path" in Audio2TextDoc has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[2024-10-17 00:27:31,027] [    INFO] - Base service - CORS is enabled.
[2024-10-17 00:27:31,030] [    INFO] - Base service - Setting up HTTP server
[2024-10-17 00:27:31,031] [    INFO] - Base service - Uvicorn server setup on port 7000
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7000 (Press CTRL+C to quit)
[2024-10-17 00:27:31,044] [    INFO] - Base service - HTTP server setup successful
/home/user/comps/retrievers/redis/langchain/retriever_redis.py:106: LangChainDeprecationWarning: The class `HuggingFaceHubEmbeddings` was deprecated in LangChain 0.2.2 and will be removed in 1.0. An updated version of the class exists in the langchain-huggingface package and should be used instead. To use it run `pip install -U langchain-huggingface` and import as `from langchain_huggingface import HuggingFaceEndpointEmbeddings`.
  embeddings = HuggingFaceHubEmbeddings(model=tei_embedding_endpoint)
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 277, in connect
    sock = self.retry.call_with_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/retry.py", line 62, in call_with_retry
    return do()
           ^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 278, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
            ^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 639, in _connect
    raise err
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 627, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/comps/retrievers/redis/langchain/retriever_redis.py", line 111, in <module>
    vector_db = Redis(embedding=embeddings, index_name=INDEX_NAME, redis_url=REDIS_URL)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/langchain_community/vectorstores/redis/base.py", line 309, in __init__
    check_redis_module_exist(redis_client, REDIS_REQUIRED_MODULES)
  File "/home/user/.local/lib/python3.11/site-packages/langchain_community/utilities/redis.py", line 55, in check_redis_module_exist
    installed_modules = client.module_list()
                        ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/commands/core.py", line 6250, in module_list
    return self.execute_command("MODULE LIST")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/client.py", line 545, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 1074, in get_connection
    connection.connect()
  File "/home/user/.local/lib/python3.11/site-packages/redis/connection.py", line 283, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.

[arun-gupta]

Same error on AWS as well. This is definitely a regression as this was working fine on AWS.

[ZailiWang]

Hi @arun-gupta would you help verify as the comment instructed? thanks. #972 (comment)

[arun-gupta]

This issue is now resolved