Add an option to disable retries on UnknownHostException #4739
Description
Describe the bug
See #4738
The S3A connector doesn't want to retry on unknown host exception on the following basis: the host is unknown. While from the AWS perspective this may seem a transient problem, it's generally isn't.
- third-party stores with custom endpoints usually hit this when the endpoint what is URL is misconfigured. This is the configuration problem which will not be recovered from.
- region-based derivation of endpoints is brittle to the region being misconfigured. Again, retries achieve nothing here.
- Attempting to use an s3express bucket with a region which doesn't support it will trigger retries of the crate session API. And this will not resolved until AWS rolls out of that storage class that region.
Expected Behavior
attempts to connect to an endpoint which trigger UnknownHostException to fail fast and let our application logic decide what to do (for us: fail)
Current Behavior
- Api Call attempt is retried
- If too many retries are attempted for the api call timeout: failure with loss of inner cause
- S3AFS currently treats ApiCallTimeoutException as recoverable; we may need to revisit this.
Reproduction Steps
See HADOOP-19000 for this surfacing connecting to S3Express buckets.
Possible Solution
client builder to allow us to disable treating unknownHostException as something SDK should retry on.
Additional Information/Context
No response
AWS Java SDK version used
2.21.33
JDK version used
openjdk version "1.8.0_362" OpenJDK Runtime Environment (Zulu 8.68.0.21-CA-macos-aarch64) (build 1.8.0_362-b09) OpenJDK 64-Bit Server VM (Zulu 8.68.0.21-CA-macos-aarch64) (build 25.362-b09, mixed mode
Operating System and version
macos 13.4.1