Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(efs): replicating file systems #29347

Merged
merged 61 commits into from
Apr 24, 2024
Merged

Conversation

badmintoncryer
Copy link
Contributor

@badmintoncryer badmintoncryer commented Mar 3, 2024

Issue # (if applicable)

Closes #21455.

Reason for this change

EFS supports replicating file systems but AWS CDK cannot configure it.

Description of changes

Add replicationConfiguration to FileSystemProps

declare const vpc: ec2.Vpc;
declare const kmsKey: kms.Key;

// auto generate a replication destination file system
new efs.FileSystem(this, 'ReplicationSourceFileSystem1', {
  vpc,
  replicationConfiguration: {
    kmsKey, // optional
    region: 'us-east-1', // optional
    availabilityZone: 'us-east-1a', // optional, Specifing the AZ means creating a One Zone file system as the replication destination
  }
});

// specify the replication destination file system
const destinationFileSystem = new efs.FileSystem(this, 'DestinationFileSystem', {
  vpc,
  // set as the read-only file system for use as a replication destination
  replicationOverwriteProtection: efs.ReplicationOverwriteProtection.DISABLED,
});

new efs.FileSystem(this, 'ReplicationSourceFileSystem2', {
  vpc,
  replicationConfiguration: {
    destinationFileSystem,
    // cannot configure other properties when destinationFileSystem is specified
  }
});

Description of how you validated changes

I have added both unit and integ tests.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added feature-request A feature should be added or improved. p2 labels Mar 3, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team March 3, 2024 17:14
@github-actions github-actions bot added the star-contributor [Pilot] contributed between 25-49 PRs to the CDK label Mar 3, 2024
@badmintoncryer badmintoncryer changed the title feat(efs): replicationg file systems feat(efs): replicating file systems Mar 3, 2024
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Mar 4, 2024
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍 I left some initial comments for some refactoring.
Please let me know if you have suggestions for further improvements.

*
* @default - no replication
*/
readonly replicationConfiguration?: ReplicationConfiguration;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why aren't we allowing multiple destinations? (CFN declaration)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CloudFormation supports an array of destinations, but the maximum number of elements is restricted to 1. Therefore, I did not implement support for multiple destinations.

Should I add support for it to accommodate future expansion?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tricky question.
I guess we can keep it like this for now and deprecate it in the future in favor of an array of configurations if necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated to receive replicationConfiguration as array.

Copy link
Contributor Author

@badmintoncryer badmintoncryer Mar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I apologize for misunderstanding your opinion.. Which implementation will you adopt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what I would do here is actually make this an enum like class to future proof us against if/when the service team does allow for multiple regions. My suggestion would be to make ReplicationConfiguration a class with static functions like the Schedule class. For now, we'll only have one function that takes in one replicationConfiguration, but in the future we can add more functions for multiple.

packages/aws-cdk-lib/aws-efs/lib/efs-file-system.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-efs/lib/efs-file-system.ts Outdated Show resolved Hide resolved
@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Mar 5, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@aws-cdk-automation aws-cdk-automation dismissed their stale review March 5, 2024 19:57

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@badmintoncryer
Copy link
Contributor Author

@gracelu0 I've updated my implementation and have a question. Could you please confirm it again?

Copy link
Contributor

@TheRealAmazonKendra TheRealAmazonKendra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well shit. I have a bunch of review comments in pending. They're from a hot minute ago so some may be outdated.

*
* @default - create regional file system for the replication destination
*/
readonly availabilityZone?: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the availability zone is only a valid field if the region is provided, these two should fields should be a single object.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further down I'm also seeing a lot of checking for whether certain properties are compatible with one another. This tells me that the contract here is too loose. I get that you are working from a construct that uses a lot of optional props when they really shouldn't be but I'd like further changes to not follow that pattern.

Can we structure this in a way that enforces the dependent props in the contract and disallows the combinations you're checking for below?

*
* @default - no replication
*/
readonly replicationConfiguration?: ReplicationConfiguration;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what I would do here is actually make this an enum like class to future proof us against if/when the service team does allow for multiple regions. My suggestion would be to make ReplicationConfiguration a class with static functions like the Schedule class. For now, we'll only have one function that takes in one replicationConfiguration, but in the future we can add more functions for multiple.

Copy link
Contributor

@gracelu0 gracelu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final tiny docstring comments and then I'm good to approve :)

packages/aws-cdk-lib/aws-efs/lib/efs-file-system.ts Outdated Show resolved Hide resolved
Comment on lines 366 to 367
*
* @default - create a new file system for the replication
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*
* @default - create a new file system for the replication

Copy link
Contributor Author

@badmintoncryer badmintoncryer Apr 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gracelu0
I believe the @ default description is essential because the destinationFileSystem is optional. Removing this description could cause the CI to fail.

What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh right, thanks for pointing that out! perhaps “None” or “No destinationFileSystem set” instead then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've updated it.

badmintoncryer and others added 2 commits April 24, 2024 10:32
Co-authored-by: Grace Luo <54298030+gracelu0@users.noreply.github.com>
Copy link
Contributor

@gracelu0 gracelu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for contributing!

Copy link
Contributor

mergify bot commented Apr 24, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Apr 24, 2024
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: ecfd8c0
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit a15dc93 into aws:main Apr 24, 2024
12 checks passed
Copy link
Contributor

mergify bot commented Apr 24, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@badmintoncryer badmintoncryer deleted the 21455-efsReplication branch April 24, 2024 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request A feature should be added or improved. p2 star-contributor [Pilot] contributed between 25-49 PRs to the CDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EFS: Create replication configuration
5 participants