Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to Get OpenHands to Work with AWS Bedrock Using Docker on macOS? #6647

Open
jop8281 opened this issue Nov 7, 2024 · 0 comments
Open

How to Get OpenHands to Work with AWS Bedrock Using Docker on macOS? #6647

jop8281 opened this issue Nov 7, 2024 · 0 comments

Comments

@jop8281
Copy link

jop8281 commented Nov 7, 2024

I'm setting up OpenHands with AWS Bedrock on macOS using Docker, but encountering connection issues related to the Docker client and server API version. While some commands inside the container work, the main application fails with the following errors:

Docker Run Command:

docker run -it --pull=always \
        -e SANDBOX_RUNTIME_CONTAINER_IMAGE="docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik" \
        -e AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID" \
        -e AWS_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY" \
        -e AWS_REGION="us-east-1" \
        -e LLM_PROVIDER="bedrock" \
        -e MODEL_ID="anthropic.claude-v2" \
        -e BASE_URL="https://bedrock.us-east-1.amazonaws.com" \
        -p 3000:3000 \
        --add-host=host.docker.internal:host-gateway \
        --name openhands-app-bedrock \
        docker.all-hands.dev/all-hands-ai/openhands:latest

Successfully ran commands inside the container:

docker exec -it openhands-app-bedrock /bin/bash and inside ipython:

from litellm import completion
response = completion(model="anthropic.claude-v2", messages=[{"role": "user", "content": "Hello, how are you?"}])
print(response.choices[0].message.content)
# Output: I'm doing well, thanks for asking!

Error Details:

Starting OpenHands...
Running OpenHands as root
INFO:     Started server process [9]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
INFO:     192.168.65.1:57134 - "GET / HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /locales/en/translation.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57135 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57135 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57135 - "GET /favicon.ico HTTP/1.1" 200 OK
18:19:26 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:26 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:26 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:26 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO:     ('192.168.65.1', 57136) - "WebSocket /ws" [accepted]
18:19:26 - openhands:ERROR: auth.py:27 - Invalid token
INFO:     192.168.65.1:57135 - "GET /api/options/models HTTP/1.1" 200 OK
INFO:     connection open
INFO:     192.168.65.1:57134 - "GET /api/options/agents HTTP/1.1" 200 OK
INFO:     192.168.65.1:57137 - "GET /api/options/security-analyzers HTTP/1.1" 200 OK
INFO:     connection closed
INFO:     192.168.65.1:57137 - "GET /assets/end-session-CpBsKLW_.js HTTP/1.1" 200 OK
INFO:     192.168.65.1:57137 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57134 - "GET /favicon.ico HTTP/1.1" 200 OK
18:19:28 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:28 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:28 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:28 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO:     ('192.168.65.1', 57138) - "WebSocket /ws" [accepted]
18:19:28 - openhands:INFO: listen.py:336 - New session: aee02a3d-97c4-4c23-9df5-61094a47a0bf
INFO:     connection open
18:19:28 - openhands:WARNING: codeact_agent.py:101 - Function calling not supported for model anthropic.claude-v2. Disabling function calling.
INFO:     192.168.65.1:57134 - "GET /config.json HTTP/1.1" 200 OK
18:19:28 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Starting runtime with image: docker.all-hands.dev/all-hands-ai/runtime:0.12-nikolaik
18:19:28 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Container started: openhands-runtime-aee02a3d-97c4-4c23-9df5-61094a47a0bf
18:19:28 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Waiting for client to become ready at http://host.docker.internal:36410...
18:19:42 - openhands:INFO: base.py:98 - [runtime aee02a3d-97c4-4c23-9df5-61094a47a0bf] Runtime is ready.
18:19:42 - openhands:WARNING: state.py:118 - Failed to restore state from session: sessions/aee02a3d-97c4-4c23-9df5-61094a47a0bf/agent_state.pkl
18:19:42 - openhands:INFO: agent_controller.py:135 - [Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] Starting step loop...
18:19:42 - openhands:INFO: agent_controller.py:135 - [Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] Setting agent(CodeActAgent) state from AgentState.LOADING to AgentState.INIT
18:19:42 - openhands:INFO: agent_controller.py:135 - [Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] Setting agent(CodeActAgent) state from AgentState.INIT to AgentState.RUNNING
18:19:42 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:42 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:42 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:42 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO:     192.168.65.1:57331 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57337 - "GET /config.json HTTP/1.1" 200 OK
INFO:     192.168.65.1:57330 - "GET /api/list-files HTTP/1.1" 200 OK
18:19:42 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:42 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:42 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:42 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO:     192.168.65.1:57337 - "GET /api/list-files HTTP/1.1" 200 OK
18:19:43 - openhands:INFO: github.py:14 - Initializing UserVerifier
18:19:43 - openhands:INFO: github.py:27 - GITHUB_USER_LIST_FILE not configured
18:19:43 - openhands:INFO: github.py:48 - GITHUB_USERS_SHEET_ID not configured
18:19:43 - openhands:INFO: github.py:85 - No user verification sources configured - allowing all users
INFO:     192.168.65.1:57330 - "GET /api/list-files HTTP/1.1" 200 OK


==============
[Agent Controller aee02a3d-97c4-4c23-9df5-61094a47a0bf] LEVEL 0 LOCAL STEP 0 GLOBAL STEP 0


Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

18:19:44 - openhands:ERROR: retry_mixin.py:47 - litellm.APIConnectionError: 'output'
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2564, in completion
    response = bedrock_converse_chat_completion.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 421, in completion
    return litellm.AmazonConverseConfig()._transform_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 390, in _transform_response
    message: Optional[MessageBlock] = completion_response["output"]["message"]
                                      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'output'
. Attempt #1 | You can customize retry values in the configuration.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

18:19:59 - openhands:ERROR: retry_mixin.py:47 - litellm.APIConnectionError: 'output'
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2564, in completion
    response = bedrock_converse_chat_completion.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 421, in completion
    return litellm.AmazonConverseConfig()._transform_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 390, in _transform_response
    message: Optional[MessageBlock] = completion_response["output"]["message"]
                                      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'output'
. Attempt #2 | You can customize retry values in the configuration.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

18:20:14 - openhands:ERROR: retry_mixin.py:47 - litellm.APIConnectionError: 'output'
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2564, in completion
    response = bedrock_converse_chat_completion.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 421, in completion
    return litellm.AmazonConverseConfig()._transform_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_transformation.py", line 390, in _transform_response
    message: Optional[MessageBlock] = completion_response["output"]["message"]
                                      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'output'
. Attempt #3 | You can customize retry values in the configuration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jop8281