Allow setting the maximum number of retry attempts for boto3

[jamin-chen]

🚀 Feature Request

Currently the retries in S3Downloader are configured like so:

streaming/streaming/base/storage/download.py

Lines 249 to 251 in a18648b

	retries = {
	'mode': 'adaptive',
	}

boto3 supports passing in a max_attempts key to this dictionary which sets the maximum number of request retries. The default value is 3 but we'd like to increase this value because we are seeing flakey behavior from some object storage providers.

[Optional] Implementation

You could allow users to set the envvar S3_MAX_ATTEMPTS.

[ricor07]

Hello, I'd be glad to contribute to the project by fixing the issue. Can you assign it to me? Thanks

[snarayan21]

Hey @ricor07, as mentioned in the comment to the attached PR, you can already specify the download_retry argument to StreamingDataset to get download retry functionality across any cloud provider.

@jamin-chen going to close out this issue as per our discussion on #853