openssl wrapper gets stuck if client is started before network is ready

[larsonmpdx]

in my client I set up a connection using code copied from the pubsub sample, which uses the openssl wrapper:

https://github.com/aws/aws-iot-device-sdk-cpp/tree/master/samples/PubSub

if I start my program before the system's network is available then it gets stuck forever on the iot_client->connect() call (see this line for example):

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/samples/PubSub/PubSub.cpp#L181

the error returned is -300, NETWORK_TCP_CONNECT_ERROR, with a log line "TCP Connection error" which means it came from this bit of code:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/network/OpenSSL/OpenSSLConnection.cpp#L290

and this specific system call:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/network/OpenSSL/OpenSSLConnection.cpp#L158

I'm going to instrument the system call to get its error code. What happens is the connection fails, my program waits N seconds and retries (with a fresh network::OpenSSLConnection object), and continues to get the same error even though my computer can ping and the network is now up. If I restart my client after the network is up it works fine.

[larsonmpdx]

error from connect() is ECONNREFUSED

[larsonmpdx]

figured this out. glibc caches /etc/resolv.conf from the program's start and you need a call to res_init() to update it. gethostbyname() was returning a local IP for my aws endpoint instead of failing, and will do so forever because res_init() is never called. I am working on a patch that I'll submit as a PR. I suggest a review of all the C system calls being used because I found a few other details wrong (not calling close() after connect() failed, for example, and some deprecated syscalls).

[chaurah]

Hi @larsonmpdx,
Thanks for figuring this out. We have decided we need to simulate this in our release pipeline somehow and make sure we cover this issue for our future releases. I will also be spending some time working only on the network code to make sure your C system call issues are addressed. I think the wrappers can be improved substantially for each of the platforms. Please do let us know if you have any further suggestions.

Rahul