(maint) - Add connect-timeout to transport

[jordanbreen28]

This PR adds a connect-timeout flag to each provisioned node in the litmus_inventory, to help prevent transient timeout errors during testing.

Initially, this defaulted to 10s which caused numerous false errors due to timeouts, this increases the default to 2 minutes.

note - machines provisioned with the provision_service will be updated in a separate PR in provision_service

[GSPatton]

this is great! I'm wondering if 2 mins is a bit long though with one retry?

I was thinking something like a maximum of 3 retries but each retry has a longer timeout.

E.g. 1st retry has timeout of 30s, 2nd of 1min and 3rd of 2mins? What do you think?

[jordanbreen28]

this is great! I'm wondering if 2 mins is a bit long though with one retry?

I was thinking something like a maximum of 3 retries but each retry has a longer timeout.

E.g. 1st retry has timeout of 30s, 2nd of 1min and 3rd of 2mins? What do you think?

I get ya and this would be niche!
But this isn’t a if connection drops retry, but more of a maximum number of seconds that the client will wait for a response from the node before dropping the connection.

So before, the node had a maximum of 10 seconds to complete the process and return the result to the client, where as now it has two minutes.

So this essentially means, the client will wait up to two minutes for a response from the node before terminating the process and dropping the connection (thus causing the test suite to fail).
Hope that clears it up :-)