Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(kinesisfirehose): add support for backing up source records to S3 #15725

Merged
merged 218 commits into from
Jul 30, 2021
Merged

feat(kinesisfirehose): add support for backing up source records to S3 #15725

merged 218 commits into from
Jul 30, 2021

Conversation

madeline-k
Copy link
Contributor

Closes #15724

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

madeline-k and others added 30 commits July 14, 2021 10:04
@madeline-k
feat(aws-kinesisfirehose): DeliveryStream API and basic S3 destination
8401fb4
@madeline-k
fix typo in aws-kinesisfirehose-destinations README
dca12f2
@madeline-k
remove unnecessary deps
6fceebf
@madeline-k
feat(aws-kinesisfirehose): specific metrics functions for DeliveryStream
58b3fe1
@madeline-k
feat(aws-kinesisfirehose): Add support for server-side encryption on …
55cb8d6 
…DeliveryStream
@madeline-k
feat(aws-kinesisfirehose): support for compression on S3 delivery str…
cfc1427 
…eam destinations
@madeline-k
feat(aws-kinesisfirehose-destinations): support prefixes for S3 desti…
b5b48ed 
…nation
@madeline-k
Merge branch 'master' into madeline-k/feat/kinesisfirehose-deliveryst…
7892f58 
…ream
@madeline-k
feat(aws-kinesisfirehose): add support for BufferingHints
be0bf89
@madeline-k
feat(aws-kinesisfirehose): support for S3 destination encryption on D…
c764118 
…eliveryStream
@madeline-k
remove awslint exception for IConnectable.connections
78e6a9c
@madeline-k
Change name of S3 destination to S3Bucket
5f7a15c
@madeline-k
add Role to DestinationBindOptions, update S3 to S3Bucket in tests
efe2fe0
@madeline-k
remove helper function to enforce ExtendedS3DestinationConfigurationP…
4308f44 
…roperty type and use local var instead
@madeline-k
remove lambda dependency
172ab43
@madeline-k
update README
c0d85f5
@madeline-k
remove tagsand ITaggable interface
3d34c46
@madeline-k
use splitArn instead of parseArn
5ceff5c
@madeline-k
update error message for bad input ARN
ce4dde1
@madeline-k
add ArnFormat to formatArn() call
0ed658f
@madeline-k
update error message in tests for bad input ARN
1925560
@madeline-k
pass deliveryStreamName as physicalName through to super constructor
e46c595
@madeline-k
move validation to beginning of constructor
d1463a2
@madeline-k
simplify createLoggingOptions
1623199
@madeline-k
remove awslint exclude
16a54e5
@madeline-k
feat(aws-kinesisfirehose): specific metrics functions for DeliveryStr…
b011392 
…eam (#15545)

Closes #15543 

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@BenChaimberg
change name of s3 destination props to S3BucketProps
56aa16a
@BenChaimberg
remove basic s3 integration test
f70fc31
@BenChaimberg
rename integration tests
73f1777
@BenChaimberg
de-exports DeliveryStreamBase
314c4ca
skinny85
skinny85 approved these changes Jul 29, 2021
View reviewed changes
packages/@aws-cdk/aws-kinesisfirehose-destinations/lib/s3-bucket.ts
private getS3BackupMode(): string | undefined {
return this.props.s3Backup?.bucket || this.props.s3Backup?.mode === BackupMode.ALL
? 'Enabled'
: undefined;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably return undefined here only if this.props. s3Backup was undefined. But that's a super minor nitpick.

@skinny85 skinny85 changed the title feat(aws-kinesisfirehose): Add support for backing up source records to S3 feat(kinesisfirehose): add support for backing up source records to S3 Jul 29, 2021
Base automatically changed from madeline-k/feat/kinesisfirehose-dataprocessors to master July 29, 2021 17:48
@mergify mergify bot dismissed stale reviews from skinny85, BenChaimberg, and otaviomacedo via d1a6905 July 29, 2021 17:48
@madeline-k madeline-k removed the pr/do-not-merge This PR should not be merged at this time. label Jul 30, 2021
BenChaimberg
BenChaimberg reviewed Jul 30, 2021
View reviewed changes
Copy link
Contributor

@BenChaimberg BenChaimberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I'll let Adam take a final look

@mergify
Copy link
Contributor

mergify bot commented Jul 30, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
  • Commit ID: 36dc4b8
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit b86062f into master Jul 30, 2021
@mergify mergify bot deleted the madeline-k/feat/kinesisfirehose-backup branch July 30, 2021 18:22
@mergify
Copy link
Contributor

mergify bot commented Jul 30, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

TikiTDO pushed a commit to TikiTDO/aws-cdk that referenced this pull request Aug 3, 2021
@madeline-k @TikiTDO
feat(kinesisfirehose): add support for backing up source records to S3 (
18f5be6 
aws#15725)

Closes aws#15724 


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
hollanddd pushed a commit to hollanddd/aws-cdk that referenced this pull request Aug 26, 2021
@madeline-k @hollanddd
feat(kinesisfirehose): add support for backing up source records to S3 (
2c7f2eb 
aws#15725)

Closes aws#15724 


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
david-doyle-as24 pushed a commit to david-doyle-as24/aws-cdk that referenced this pull request Sep 7, 2021
@madeline-k @david-doyle-as24
feat(kinesisfirehose): add support for backing up source records to S3 (
1961e11 
aws#15725)

Closes aws#15724 


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BenChaimberg BenChaimberg BenChaimberg left review comments

@skinny85 skinny85 skinny85 approved these changes

@otaviomacedo otaviomacedo Awaiting requested review from otaviomacedo

Assignees
No one assigned
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

(aws-kinesisfirehose): Support for backing up source records to S3
5 participants
@madeline-k @skinny85 @aws-cdk-automation @otaviomacedo @BenChaimberg