Skip to content

HADOOP-19286: S3A: Support cross region access when S3 region/endpoint is set (ADDENDUM) #7098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 4, 2024
+8 −1
Merged

HADOOP-19286: S3A: Support cross region access when S3 region/endpoint is set (ADDENDUM) #7098

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,8 +271,10 @@ protected ClientOverrideConfiguration.Builder createClientOverrideConfiguration(
* <li>If endpoint is configured via via fs.s3a.endpoint, set it.
* If no region is configured, try to parse region from endpoint. </li>
* <li> If no region is configured, and it could not be parsed from the endpoint,
* set the default region as US_EAST_2 and enable cross region access. </li>
* set the default region as US_EAST_2</li>
* <li> If configured region is empty, fallback to SDK resolution chain. </li>
* <li> S3 cross region is enabled by default irrespective of region or endpoint
* is set or not.</li>
* </ol>
*
* @param builder S3 client builder.
Expand Down
5 changes: 5 additions & 0 deletions hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,10 @@ public void testWithOutCrossRegionAccess() throws Exception {
// skip the test if the region is sa-east-1
skipCrossRegionTest();
final Configuration newConf = new Configuration(getConfiguration());
removeBaseAndBucketOverrides(newConf,
ENDPOINT,
AWS_S3_CROSS_REGION_ACCESS_ENABLED,
AWS_REGION);
// disable cross region access
newConf.setBoolean(AWS_S3_CROSS_REGION_ACCESS_ENABLED, false);
newConf.set(AWS_REGION, SA_EAST_1);
Expand All @@ -374,6 +378,7 @@ public void testWithCrossRegionAccess() throws Exception {
skipCrossRegionTest();
final Configuration newConf = new Configuration(getConfiguration());
removeBaseAndBucketOverrides(newConf,
ENDPOINT,
AWS_S3_CROSS_REGION_ACCESS_ENABLED,
AWS_REGION);
// enable cross region access
Expand Down