Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow cross region PrivateLink connections #30897

Conversation

bobbyiliev
Copy link
Contributor

Motivation

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

if !connection_az
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-')
|| !connection_az.contains("-az")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like we could tighten this up to be more precise. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#az-ids has more details on the structure

the most legit solution would be to automatically pull every AZ ID from the partition we're operating in during our Pulumi run and then plumbing it 20 layers deep to here... in practice I think just ensuring we're following the structure in the doc above is sufficient

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and if we pull out some logic to validate the ids, we should add a quick unit test)

@bobbyiliev bobbyiliev force-pushed the privatelink-cross-region-support branch from a5f7d10 to a1bf87b Compare January 7, 2025 12:14
@bobbyiliev bobbyiliev changed the title [WIP] Allow cross region PrivateLink connections Allow cross region PrivateLink connections Jan 7, 2025
@bobbyiliev bobbyiliev marked this pull request as ready for review January 7, 2025 19:06
@bobbyiliev bobbyiliev requested a review from a team as a code owner January 7, 2025 19:06
fn is_valid_az_format(az: &str) -> bool {
let az_pattern = Regex::new(r"^[a-z]{3,4}\d-az[1-6]$").unwrap();
az_pattern.is_match(az)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worries me a bit injecting a pattern when we don't know what rules AWS uses internally for naming these, but this is probably as good as we can do without injecting the complete list.

@bobbyiliev bobbyiliev requested a review from pH14 January 8, 2025 11:48
Copy link
Member

@ParkMyCar ParkMyCar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code itself looks good to me, not super familiar though with cross region PrivateLink so would defer to Cloud folks on that

@bobbyiliev
Copy link
Contributor Author

Looking into this further I think that this PR might not even be necessary! Still validating this but will keep the threat updated.

@bobbyiliev
Copy link
Contributor Author

Closing this PR in favor ofhttps://github.com/MaterializeInc/cloud/pull/10617

No changes will be needed on the database side nor the AZ id validation on the controller side.

AWS handles all this on their end. All that we would need to do to support this is to just pass the service_region to the create request. We are extracting that information from the service name itself as it contains the region already.

You must now select the VPC and subnets you want to create the VPC endpoint in. Cross-region connectivity does not require any AZ alignment, unlike in-region PrivateLink where the consumer can only create an endpoint in the AZs supported by the service provider. You can choose as many AZs as your resiliency posture requires and choose the subnets that best fit your needs.

I will open just a follow-up docs PR for this clarification.

@bobbyiliev bobbyiliev closed this Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants