Connection attempts keep failing with asio.ssl error

[dbouras]

Describe the bug

Platform: Apple M2 Pro running MacOS Sequoia (15.0.1)

Two (seemingly) identical systems (at least, as of this writing, I am not able to pinpoint some difference that may be the root cause) behave quite differently: one connects without issues, the other gets stuck in a loop retrying and always failing with:
[error] Could not perform SSL handshake with proxy server: asio.ssl error

To Reproduce

I have not found a way to reproduce it unfortunately; localproxy works flawlessly on all but one system.

Expected behavior

A successful connection.

Actual behavior

The connection attempt fails; localroxy then goes into a loop of unsuccessful retries.

Logs

An excerpt from the debug log is as follows:

[2024-10-23 10:44:23.854539] (0x00000001e5d8f240) [info] setting source protocol to V1
[2024-10-23 10:44:23.855331] (0x00000001e5d8f240) [debug] v2 local proxy starts with v1 local proxy format
[2024-10-23 10:44:23.855397] (0x00000001e5d8f240) [debug] /Users/______________/bin/config does not exist!
[2024-10-23 10:44:23.855419] (0x00000001e5d8f240) [info] Starting proxy in source mode
[2024-10-23 10:44:23.855448] (0x00000001e5d8f240) [trace] Setting up web socket...
[2024-10-23 10:44:23.864030] (0x00000001e5d8f240) [trace] Calling control_callback with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:23.864237] (0x00000001e5d8f240) [info] Attempting to establish web socket connection with endpoint wss://data.tunneling.iot.eu-west-1.amazonaws.com:443
[2024-10-23 10:44:23.864260] (0x00000001e5d8f240) [trace] Resolving proxy server host: data.tunneling.iot.eu-west-1.amazonaws.com
[2024-10-23 10:44:23.886413] (0x00000001e5d8f240) [debug] Resolved proxy server IP: 52.31.213.74
[2024-10-23 10:44:23.886483] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:23.954914] (0x00000001e5d8f240) [debug] Connected successfully with proxy server
[2024-10-23 10:44:23.954968] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:23.954988] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:23.955004] (0x00000001e5d8f240) [trace] Performing SSL handshake with proxy server
[2024-10-23 10:44:23.955020] (0x00000001e5d8f240) [trace] Calling set_verify_mode with type: single_ssl_stream
[2024-10-23 10:44:23.955034] (0x00000001e5d8f240) [trace] Calling set_verify_callback with type: single_ssl_stream
[2024-10-23 10:44:23.955094] (0x00000001e5d8f240) [trace] Calling next_layer().async_handshake with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:23.955118] (0x00000001e5d8f240) [trace] SSL next_layer() SNI is set : data.tunneling.iot.eu-west-1.amazonaws.com
[2024-10-23 10:44:24.024944] (0x00000001e5d8f240) [error] Could not perform SSL handshake with proxy server: asio.ssl error
[2024-10-23 10:44:26.526264] (0x00000001e5d8f240) [trace] Calling is_open with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.526514] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.526578] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.527936] (0x00000001e5d8f240) [trace] Calling control_callback with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.528051] (0x00000001e5d8f240) [info] Attempting to establish web socket connection with endpoint wss://data.tunneling.iot.eu-west-1.amazonaws.com:443
[2024-10-23 10:44:26.528123] (0x00000001e5d8f240) [trace] Resolving proxy server host: data.tunneling.iot.eu-west-1.amazonaws.com
[2024-10-23 10:44:26.530954] (0x00000001e5d8f240) [debug] Resolved proxy server IP: 52.31.213.74
[2024-10-23 10:44:26.531047] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.603017] (0x00000001e5d8f240) [debug] Connected successfully with proxy server
[2024-10-23 10:44:26.603227] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.603285] (0x00000001e5d8f240) [trace] Calling lowest_layer with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.603325] (0x00000001e5d8f240) [trace] Performing SSL handshake with proxy server
[2024-10-23 10:44:26.603358] (0x00000001e5d8f240) [trace] Calling set_verify_mode with type: single_ssl_stream
[2024-10-23 10:44:26.603396] (0x00000001e5d8f240) [trace] Calling set_verify_callback with type: single_ssl_stream
[2024-10-23 10:44:26.603439] (0x00000001e5d8f240) [trace] Calling next_layer().async_handshake with type: websocket_stream_single_ssl_type
[2024-10-23 10:44:26.603478] (0x00000001e5d8f240) [trace] SSL next_layer() SNI is set : data.tunneling.iot.eu-west-1.amazonaws.com
[2024-10-23 10:44:26.682584] (0x00000001e5d8f240) [error] Could not perform SSL handshake with proxy server: asio.ssl error

Environment (please complete the following information):

• MacOS Sequoia
• Version 15.0.1
• Architecture: arm64
• Localproxy commit: b9d706f using macos binary downloaded from https://github.com/aws-samples/aws-iot-securetunneling-localproxy/actions/runs/11434364820; first experienced the issue with a locally built binary from commit a2423da.

Additional context

N/A

[ig15]

Hello @dbouras . Thanks for reaching out to us. I think the issue is specific to your system environment, since I am able to successfully execute the localproxy binary from https://github.com/aws-samples/aws-iot-securetunneling-localproxy/actions/runs/11434364820 on my machine. Can you share more details about both the source and destination machines and which one is having the concerned error. Also please share the command you are using to execute the binary for reference.

[dbouras]

Hi @ig15 ,
Details about the source machine you already have (see "Environment" info above). The destination is a Greengrass core device running aws.greengrass.SecureTunneling v1.0.19. The localproxy command line is as follows:

localproxy --destination-client-type ${DESTVER} -v ${LOGLEVEL} -s ${LOCALPORT}

with the following defined in the environment:

AWSregion="eu-west-1"
LOCALPORT="8940"
LOGLEVEL=6
DESTVER="V1"

I fully realize that not being able to replicate consistently is a huge blocker but wanted to report it anyway just in case someone had come across this issue in the past and had some ideas on what to try...