Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bucket Notifications #201

Merged
merged 3 commits into from
Aug 13, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions packages/@aws-cdk/aws-s3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,3 +143,40 @@ new Consumer(app, 'consume', {

process.stdout.write(app.run());
```

### Bucket Notifications

The Amazon S3 notification feature enables you to receive notifications when
certain events happen in your bucket as described under [S3 Bucket
Notifications] of the S3 Developer Guide.

To subscribe for bucket notifications, use the `bucket.onEvent` method. The
`bucket.onObjectCreated` and `bucket.onObjectRemoved` can also be used for these
common use cases.

The following example will subscribe an SNS topic to be notified of all
``s3:ObjectCreated:*` events:

```ts
const myTopic = new sns.Topic(this, 'MyTopic');
bucket.onEvent(s3.EventType.ObjectCreated, myTopic);
```

This call will also ensure that the topic policy can accept notifications for
this specific bucket.

The following destinations are currently supported:

* `sns.Topic`
* `sqs.Queue`
* `lambda.Function`

It is also possible to specify S3 object key filters when subscribing. The
following example will notify `myQueue` when objects prefixed with `foo/` are
deleted from the bucket.

```ts
bucket.onEvent(s3.EventType.ObjectRemoved, myQueue, 'foo/*')
```

[S3 Bucket Notifications]: https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
156 changes: 156 additions & 0 deletions packages/@aws-cdk/aws-s3/lib/bucket.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import iam = require('@aws-cdk/aws-iam');
import kms = require('@aws-cdk/aws-kms');
import cdk = require('@aws-cdk/cdk');
import { BucketPolicy } from './bucket-policy';
import { INotificationDestination } from './notification-dest';
import { BucketNotifications } from './notifications-resource';
import perms = require('./perms');
import { LifecycleRule } from './rule';
import { BucketArn, BucketDomainName, BucketDualStackDomainName, cloudformation } from './s3.generated';
Expand Down Expand Up @@ -289,6 +291,7 @@ export class Bucket extends BucketRef {
protected autoCreatePolicy = true;
private readonly lifecycleRules: LifecycleRule[] = [];
private readonly versioned?: boolean;
private readonly notifications: BucketNotifications;

constructor(parent: cdk.Construct, name: string, props: BucketProps = {}) {
super(parent, name);
Expand Down Expand Up @@ -316,6 +319,10 @@ export class Bucket extends BucketRef {

// Add all lifecycle rules
(props.lifecycleRules || []).forEach(this.addLifecycleRule.bind(this));

// defines a BucketNotifications construct. Notice that an actual resource will only
// be added if there are notifications added, so we don't need to condition this.
this.notifications = new BucketNotifications(this, 'Notifications', { bucket: this });
}

/**
Expand All @@ -333,6 +340,47 @@ export class Bucket extends BucketRef {
this.lifecycleRules.push(rule);
}

/**
* Adds a bucket notification event target.
* @param event The event to trigger the notification
* @param dest The notification destination (Lambda, SNS Topic or SQS Queue)
*
* @param s3KeyFilters S3 filter rules to determine which objects trigger
* this event. Rules must include either a prefix asterisk ("*foo/bar") or
* suffix asterisk ("foo/bar*") to indicate if this is a prefix or a suffix
* rule.
*
* @example
*
* bucket.onEvent(EventType.OnObjectCreated, myLambda, 'home/myusername/*')
*
* @see
* https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
*/
public onEvent(event: EventType, dest: INotificationDestination, ...s3KeyFilters: string[]) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change filter to be strongly typed. Prefix can only be specified once apparently for example

this.notifications.addNotification(event, dest, ...s3KeyFilters);
}

/**
* Subscribes a target to receive notificatins when an object is created in the bucket.
* This is identical to calling `onEvent(EventType.ObjectCreated)`.
* @param dest The notification destination (see onEvent)
* @param s3KeyFilters Filters (see onEvent)
*/
public onObjectCreated(dest: INotificationDestination, ...s3KeyFilters: string[]) {
return this.onEvent(EventType.ObjectCreated, dest, ...s3KeyFilters);
}

/**
* Subscribes a target to receive notificatins when an object is removed from the bucket.
* This is identical to calling `onEvent(EventType.ObjectRemoved)`.
* @param dest The notification destination (see onEvent)
* @param s3KeyFilters Filters (see onEvent)
*/
public onObjectRemoved(dest: INotificationDestination, ...s3KeyFilters: string[]) {
return this.onEvent(EventType.ObjectRemoved, dest, ...s3KeyFilters);
}

/**
* Set up key properties and return the Bucket encryption property from the
* user's configuration.
Expand Down Expand Up @@ -485,6 +533,114 @@ export class S3Url extends cdk.Token {

}

/**
* Notification event types.
*/
export enum EventType {
/**
* Amazon S3 APIs such as PUT, POST, and COPY can create an object. Using
* these event types, you can enable notification when an object is created
* using a specific API, or you can use the s3:ObjectCreated:* event type to
* request notification regardless of the API that was used to create an
* object.
*/
ObjectCreated = 's3:ObjectCreated:*',

/**
* Amazon S3 APIs such as PUT, POST, and COPY can create an object. Using
* these event types, you can enable notification when an object is created
* using a specific API, or you can use the s3:ObjectCreated:* event type to
* request notification regardless of the API that was used to create an
* object.
*/
ObjectCreatedPut = 's3:ObjectCreated:Put',

/**
* Amazon S3 APIs such as PUT, POST, and COPY can create an object. Using
* these event types, you can enable notification when an object is created
* using a specific API, or you can use the s3:ObjectCreated:* event type to
* request notification regardless of the API that was used to create an
* object.
*/
ObjectCreatedPost = 's3:ObjectCreated:Post',

/**
* Amazon S3 APIs such as PUT, POST, and COPY can create an object. Using
* these event types, you can enable notification when an object is created
* using a specific API, or you can use the s3:ObjectCreated:* event type to
* request notification regardless of the API that was used to create an
* object.
*/
ObjectCreatedCopy = 's3:ObjectCreated:Copy',

/**
* Amazon S3 APIs such as PUT, POST, and COPY can create an object. Using
* these event types, you can enable notification when an object is created
* using a specific API, or you can use the s3:ObjectCreated:* event type to
* request notification regardless of the API that was used to create an
* object.
*/
ObjectCreatedCompleteMultipartUpload = 's3:ObjectCreated:CompleteMultipartUpload',

/**
* By using the ObjectRemoved event types, you can enable notification when
* an object or a batch of objects is removed from a bucket.
*
* You can request notification when an object is deleted or a versioned
* object is permanently deleted by using the s3:ObjectRemoved:Delete event
* type. Or you can request notification when a delete marker is created for
* a versioned object by using s3:ObjectRemoved:DeleteMarkerCreated. For
* information about deleting versioned objects, see Deleting Object
* Versions. You can also use a wildcard s3:ObjectRemoved:* to request
* notification anytime an object is deleted.
*
* You will not receive event notifications from automatic deletes from
* lifecycle policies or from failed operations.
*/
ObjectRemoved = 's3:ObjectRemoved:*',

/**
* By using the ObjectRemoved event types, you can enable notification when
* an object or a batch of objects is removed from a bucket.
*
* You can request notification when an object is deleted or a versioned
* object is permanently deleted by using the s3:ObjectRemoved:Delete event
* type. Or you can request notification when a delete marker is created for
* a versioned object by using s3:ObjectRemoved:DeleteMarkerCreated. For
* information about deleting versioned objects, see Deleting Object
* Versions. You can also use a wildcard s3:ObjectRemoved:* to request
* notification anytime an object is deleted.
*
* You will not receive event notifications from automatic deletes from
* lifecycle policies or from failed operations.
*/
ObjectRemovedDelete = 's3:ObjectRemoved:Delete',

/**
* By using the ObjectRemoved event types, you can enable notification when
* an object or a batch of objects is removed from a bucket.
*
* You can request notification when an object is deleted or a versioned
* object is permanently deleted by using the s3:ObjectRemoved:Delete event
* type. Or you can request notification when a delete marker is created for
* a versioned object by using s3:ObjectRemoved:DeleteMarkerCreated. For
* information about deleting versioned objects, see Deleting Object
* Versions. You can also use a wildcard s3:ObjectRemoved:* to request
* notification anytime an object is deleted.
*
* You will not receive event notifications from automatic deletes from
* lifecycle policies or from failed operations.
*/
ObjectRemovedDeleteMarkerCreated = 's3:ObjectRemoved:DeleteMarkerCreated',

/**
* You can use this event type to request Amazon S3 to send a notification
* message when Amazon S3 detects that an object of the RRS storage class is
* lost.
*/
ReducedRedundancyLostObject = 's3:ReducedRedundancyLostObject',
}

class ImportedBucketRef extends BucketRef {
public readonly bucketArn: BucketArn;
public readonly bucketName: BucketName;
Expand Down
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-s3/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export * from './bucket';
export * from './bucket-policy';
export * from './rule';
export * from './notification-dest';

// AWS::S3 CloudFormation Resources:
export * from './s3.generated';
37 changes: 37 additions & 0 deletions packages/@aws-cdk/aws-s3/lib/notification-dest.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import cdk = require('@aws-cdk/cdk');
import { Bucket } from './bucket';

/**
* Implemented by constructs that can be used as bucket notification targets.
*/
export interface INotificationDestination {
/**
* Registers this resource to receive notifications for the specified bucket.
* @param bucket The bucket. Use the `path` of the bucket as a unique ID.
*/
bucketNotificationDestination(bucket: Bucket): NotificationDestinationProps;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we said the standard name for this function was going to be notificationDestination (based off the name of the interface and the props type).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did, but because of namespacing the interface name lost some of the context. However as a method of an L2 construct “notificationDestination” is not sufficient. We must have some delinataion as for where is this coming from. Otherwise it will look very odd. Maybe ‘s3NotificationDestination’?

Otherwise, I can rename the interface to ‘BucketNotificationDestination’. What do you think?

I am also wondering - ideally users shouldn’t care about these “inverted control” methods whatsoever (we now have a few of them). I wonder if we should find some convention that makes it intuitively clear that you don’t need to call this yourself.

We could prefix with something like an underscore... or something like “asBucketNotificationDestination”.

I kinda like the “as” prefix.

Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, changed to asBucketNotificationDestination. It's a mouthful, but not really intended to be used by users, and I feel the as prefix works.

}

/**
* Represents the properties of a notification target.
*/
export interface NotificationDestinationProps {
/**
* The notification type.
*/
readonly type: NotificationDestinationType;

/**
* The ARN of the target.
*/
readonly arn: cdk.Arn;
}

/**
* Supported types of notificaiton targets.
*/
export enum NotificationDestinationType {
Lambda,
Queue,
Topic
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from './notifications-resource';
Loading