Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure S3 redirects use TLS #1762

Merged
merged 1 commit into from
Mar 27, 2024
Merged

Conversation

daviesrob
Copy link
Member

When following a 3xx redirection from AWS, redirect_endpoint_callback() wasn't putting https:// on the new URL. The redirection worked, but dropped back to http. Obviously, this is not desirable.

Fix by prepending https:// to ensure it uses TLS, and adding a bit of error checking to ensure all parts of the new url have been included.

Fixes #1760

When following a 3xx redirection from AWS,
redirect_endpoint_callback() wasn't putting 'https://' on the
new URL.  The redirection worked, but dropped back to http.
Fix by prepending 'https://' to ensure it uses TLS, and
adding a bit of error checking to ensure all parts of the
new url have been included.
@daviesrob
Copy link
Member Author

To reproduce the problem, try:

HTS_S3_ADDRESS_STYLE=path AWS_DEFAULT_REGION=us-east-2 ./test/test_view -vvvvvvvvv -p /dev/null 's3://1000genomes/1000G_2504_high_coverage/data/ERR3243158/HG01785.final.cram' chr2:178523221-178523221

and look for Connected to .* port in the debugging output.

@whitwham whitwham self-assigned this Mar 26, 2024
@whitwham whitwham merged commit c1247f9 into samtools:develop Mar 27, 2024
9 checks passed
@daviesrob daviesrob deleted the s3_redirect branch March 28, 2024 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

S3 plugin does not correctly handle 307 redirects for newly created buckets
2 participants