Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(iot): add the TopicRule L2 construct #16681

Merged
merged 23 commits into from
Oct 21, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
e6c26f5
feat(iot): support the TopicRule L2 construct
yamatatsu Sep 28, 2021
259ca2e
Merge branch 'master' into aws-iot
yamatatsu Oct 6, 2021
61c9d70
Merge branch 'master' into aws-iot
yamatatsu Oct 8, 2021
5b3f786
Update packages/@aws-cdk/aws-iot/lib/action.ts
yamatatsu Oct 13, 2021
84001cd
Update packages/@aws-cdk/aws-iot/lib/topic-rule.ts
yamatatsu Oct 13, 2021
94dcbbe
fix(iot): be a small PR
yamatatsu Oct 13, 2021
f969810
Merge branch 'master' into aws-iot
yamatatsu Oct 13, 2021
e2b78f5
Update packages/@aws-cdk/aws-iot/lib/topic-rule.ts
yamatatsu Oct 15, 2021
4023a08
Update packages/@aws-cdk/aws-iot/lib/topic-rule.ts
yamatatsu Oct 15, 2021
912fdaf
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 15, 2021
1562ac6
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 15, 2021
1ada690
fix(iot): about feedbacks
yamatatsu Oct 16, 2021
9e7256b
feat(iot): modeling AWS IoT SQL
yamatatsu Oct 18, 2021
59815ad
Update packages/@aws-cdk/aws-iot/test/iot-sql.test.ts
yamatatsu Oct 19, 2021
89a8571
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 19, 2021
8a94620
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 19, 2021
a252d75
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 19, 2021
30d0fa6
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 19, 2021
f21567e
Update packages/@aws-cdk/aws-iot/test/topic-rule.test.ts
yamatatsu Oct 19, 2021
e150499
Update packages/@aws-cdk/aws-iot/lib/topic-rule.ts
yamatatsu Oct 19, 2021
13b4701
fix(iot): about feedbacks
yamatatsu Oct 19, 2021
1bcb870
feat(iot): remove IAction
yamatatsu Oct 21, 2021
f2c0a38
Merge branch 'master' into aws-iot
mergify[bot] Oct 21, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 77 additions & 1 deletion packages/@aws-cdk/aws-iot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,84 @@
>
> [CFN Resources]: https://docs.aws.amazon.com/cdk/latest/guide/constructs.html#constructs_lib

![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)

> The APIs of higher level constructs in this module are experimental and under active development.
> They are subject to non-backward compatible changes or removal in any future version. These are
> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be
> announced in the release notes. This means that while you may use them, you may need to update
> your source code when upgrading to a newer version of this package.

---

<!--END STABILITY BANNER-->

This module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.
The `TopicRule` construct defined Rules that give your devices the ability to interact with AWS services.
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
Rules are analyzed and actions are performed based on the MQTT topic stream.
The `TopicRule` construct can use actions like these:

- Write data received from a device to an Amazon DynamoDB database.
- Save a file to Amazon S3.
- Send a push notification to all users using Amazon SNS.
- Publish data to an Amazon SQS queue.
- Invoke a Lambda function to extract data.
- Process messages from a large number of devices using Amazon Kinesis. (*To be developed*)
- Send data to the Amazon OpenSearch Service. (*To be developed*)
- Capture a CloudWatch metric.
- Change a CloudWatch alarm.
- Send message data to an AWS IoT Analytics channel. (*To be developed*)
- Start execution of a Step Functions state machine. (*To be developed*)
- Send message data to an AWS IoT Events input. (*To be developed*)
- Send message data an asset property in AWS IoT SiteWise. (*To be developed*)
- Send message data to a web application or service. (*To be developed*)

## Installation

Install the module:

```console
$ npm i @aws-cdk/aws-iot
```

Import it into your code:

```ts
import * as iot from '@aws-cdk/aws-iot';
```

The `iot.Project` construct represents a build project resource. See the reference documentation for a comprehensive list of initialization properties, methods and attributes.
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved

## `TopicRule`

For example, to define a rule that triggers to put to a S3 bucket:

```ts
new iot.TopicRule(stack, 'MyTopicRule', {
topicRuleName: 'MyRuleName', // optional property
sql: "SELECT topic(2) as device_id, temperature FROM 'device/+/data'",
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
acctions: [
{
configration: {
s3: {
bucketName: 'your-bucket-name',
key: 'your-bucket-key',
roleArn: 'your-role-arn',
}
}
}
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
],
});
```

Or you can add action after constructing instance as following:

```ts
const topicRule = new TopicRule(stack, 'MyTopicRule', {
sql: "SELECT topic(2) as device_id, temperature FROM 'device/+/data'",
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
});
topicRule.addAction({
configration: {
functionArn: 'your-lambda-function-arn',
}
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
});
```
skinny85 marked this conversation as resolved.
Show resolved Hide resolved
24 changes: 24 additions & 0 deletions packages/@aws-cdk/aws-iot/lib/action.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { CfnTopicRule } from './iot.generated';
import { ITopicRule } from './topic-rule';

/**
* An abstract action for TopicRule.
*/
export interface IAction {
/**
* Returns the topic rule action specification.
*
* @param rule The TopicRule that would trigger this action.
*/
bind(rule: ITopicRule): ActionConfig;
}

/**
* Properties for an topic rule action
*/
export interface ActionConfig {
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
/**
* The configuration for this action.
*/
readonly configuration: CfnTopicRule.ActionProperty;
}
3 changes: 3 additions & 0 deletions packages/@aws-cdk/aws-iot/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
export * from './action';
export * from './topic-rule';

// AWS::IoT CloudFormation Resources:
export * from './iot.generated';
137 changes: 137 additions & 0 deletions packages/@aws-cdk/aws-iot/lib/topic-rule.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
import { ArnFormat, Lazy, Resource, Stack, IResource } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { IAction } from './action';
import { CfnTopicRule } from './iot.generated';

/**
* Properties for defining an AWS IoT Rule
*/
export interface TopicRuleProps {
/**
* The name of the rule.
* @default None
*/
readonly topicRuleName?: string;

/**
* The actions associated with the rule.
*
* @default No actions will be perform
*/
readonly actions?: Array<IAction>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this should be required, no? It's required in the CfnTopicRule, at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, no. I think it is better that this is not required.
Because

  • addAction() is provided
  • Either way, we need to validate that one or more of the actions are set. Because these are the possibilities that empty actions are defined in props.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Do we have validation somewhere that at least one Action was provided? Perhaps you want to override the validate() method here? Similar like we do in CodePipeline, for example?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry, I was mistaken about AWS IoT specification. AWS IoT can be defined with empty actions as following image:
image

So I'll not implement validate(), but it is very useful knowledge for me. Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm.

Given providing IActions is not required to deploy a TopicRule - is there any way we can get rid of IAction completely from this PR? And introduce it later?

Copy link
Contributor Author

@yamatatsu yamatatsu Oct 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skinny85
OK! I'd like to remove actions: from sample codes in the README but not remove IAction from implementation.
Because Actions property of CloudFormation is required (though we can use empty array). So if we remove it, here's code will be as following:

topicRulePayload: {
  actions: [],
  awsIotSqlVersion: sqlConfig.awsIotSqlVersion,
  sql: sqlConfig.sql,
},

I think this is a bit confusing.
What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is confusing at all 🙂. This is an implementation detail, the user of TopicRule will never see that 🙂.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skinny85
Thank you for your opinion! I think it is better to follow your opinion for simplicity of this PR. I will commit removing!

If I create a PR to implement IAction after this PR, is it should includes to implement aws-iot-actions? Or only IAction?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the next PR should include:

  1. The IAction interface in @aws-cdk/aws-iot.
  2. Using the IAction in TopicRule (specifying it in TopicRuleProps, TopicRule.addAction() method, calling IAction.bind(), passing actions to the CfnTopicRule, etc.).
  3. The new @aws-cdk/aws-iot-actions module (all of the files CDK requires, like package.json, NOTICE, LICENSE, etc.).
  4. A single implementation of IAction in @aws-cdk/aws-iot-actions (it's your choice which one you want to do first 🙂).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And of course, ReadMe files updates 🙂.


/**
* A simplified SQL syntax to filter messages received on an MQTT topic and push the data elsewhere.
*
* @see https://docs.aws.amazon.com/iot/latest/developerguide/iot-sql-reference.html
*/
readonly sql: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's model this a little bit differently.

I have a feeling we might want to add a nice API around constructing the SQL string too. Let's "future proof" this a little bit.

I know you had an enum here before that allowed you to choose the version used. Let's combine the version with this concept to make it what CDK calls a union-like class.

So, we'll have:

export interface TopicRuleProps {
  // ...

  readonly sql: IotSql;
}

export abstract class IotSql {
  public static ver_2015_10_08_fromString(sql: string): IotSql { ... }

  // ...

  public abstract bind(scope: Construct): IotSqlConfig;
}

Take a look at how these are implemented in the AppMesh library, which uses this pattern often; here's a good example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I will implement as lambda.Code and appmesh.HttpRoutePathMatch.

But I'd like to understand more clearly.
I wonder why this implementation is "future proof"?
Is it because the user's code will explicit the SQL version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant "future proof" because you removed the the SQL version enum in this iteration of the PR (which was the correct thing to do, BTW), and I'm kind of bringing it back with this vision for the IotSql class.

}

/**
* Represents an AWS IoT Rule
*/
export interface ITopicRule extends IResource {
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
/**
* The value of the topic rule Amazon Resource Name (ARN), such as
* arn:aws:iot:us-east-2:123456789012:rule/rule_name
*
* @attribute
*/
readonly topicRuleArn: string;

/**
* The name topic rule
*
* @attribute
*/
readonly topicRuleName: string;
}

/**
* Defines an AWS IoT Rule in this stack.
*/
export class TopicRule extends Resource implements ITopicRule {
/**
* Import an existing AWS IoT Rule provided an ARN
*
* @param scope The parent creating construct (usually `this`).
* @param id The construct's name.
* @param topicRuleArn AWS IoT Rule ARN (i.e. arn:aws:iot:<region>:<account-id>:rule/MyRule).
*/
public static fromTopicRuleArn(scope: Construct, id: string, topicRuleArn: string): ITopicRule {
const parts = Stack.of(scope).splitArn(topicRuleArn, ArnFormat.SLASH_RESOURCE_NAME);
if (!parts.resourceName) {
throw new Error('Invalid topic rule arn: topicRuleArn has no resource name.');
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
}
const resourceName = parts.resourceName;

class Import extends Resource implements ITopicRule {
public readonly topicRuleArn = topicRuleArn;
public readonly topicRuleName = resourceName;
}
return new Import(scope, id);
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Arn of this rule
* @attribute
*/
public readonly topicRuleArn: string;

/**
* Name of this rule
* @attribute
*/
public readonly topicRuleName: string;

private readonly actions = new Array<CfnTopicRule.ActionProperty>();

constructor(scope: Construct, id: string, props: TopicRuleProps) {
super(scope, id, {
physicalName: props.topicRuleName,
});

if (props.sql === '') {
throw new Error('sql cannot be empty.');
}

const resource = new CfnTopicRule(this, 'Resource', {
ruleName: this.physicalName,
topicRulePayload: {
actions: Lazy.any({ produce: () => this.actions }),
sql: props.sql,
},
});

this.topicRuleArn = this.getResourceArnAttribute(resource.attrArn, {
service: 'iot',
resource: 'rule',
resourceName: this.physicalName,
});
this.topicRuleName = this.getResourceNameAttribute(resource.ref);

props.actions?.forEach(action => {
this.addAction(action);
});
}

/**
* Add a action to the rule.
*
* @param action the action to associate with the rule.
*/
public addAction(action: IAction): void {
const { configuration } = action.bind(this);

const keys = Object.keys(configuration);
if (keys.length === 0) {
throw new Error('Empty actions are not allowed. Please define one type of action');
}
if (keys.length >= 2) {
throw new Error(`Each object in the actions list can only have one action defined. keys: ${keys}`);
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
}

this.actions.push(configuration);
}
}
6 changes: 4 additions & 2 deletions packages/@aws-cdk/aws-iot/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,11 @@
"devDependencies": {
"@aws-cdk/assertions": "0.0.0",
"@aws-cdk/cdk-build-tools": "0.0.0",
"@aws-cdk/cdk-integ-tools": "0.0.0",
"@aws-cdk/cfn2ts": "0.0.0",
"@aws-cdk/pkglint": "0.0.0",
"@types/jest": "^26.0.24"
"@types/jest": "^26.0.24",
"jest": "^26.6.3"
},
"dependencies": {
"@aws-cdk/core": "0.0.0",
Expand All @@ -91,7 +93,7 @@
"node": ">= 10.13.0 <13 || >=13.7.0"
},
"stability": "experimental",
"maturity": "cfn-only",
"maturity": "experimental",
"awscdkio": {
"announce": false
},
Expand Down
19 changes: 19 additions & 0 deletions packages/@aws-cdk/aws-iot/test/integ.topic-rule.expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"Resources": {
"TopicRule40A4EA44": {
"Type": "AWS::IoT::TopicRule",
"Properties": {
"TopicRulePayload": {
"Actions": [
{
"Lambda": {
"FunctionArn": "test-arn"
}
}
],
"Sql": "SELECT topic(2) as device_id FROM 'device/+/data'"
}
}
}
}
}
28 changes: 28 additions & 0 deletions packages/@aws-cdk/aws-iot/test/integ.topic-rule.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/// !cdk-integ pragma:ignore-assets
import * as cdk from '@aws-cdk/core';
import * as iot from '../lib';

const app = new cdk.App();

class TestStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);

const topicRule = new iot.TopicRule(this, 'TopicRule', {
sql: "SELECT topic(2) as device_id FROM 'device/+/data'",
});

topicRule.addAction({
bind: () => ({
configuration: {
lambda: {
functionArn: 'test-arn',
yamatatsu marked this conversation as resolved.
Show resolved Hide resolved
},
},
}),
});
}
}

new TestStack(app, 'test-stack');
app.synth();
6 changes: 0 additions & 6 deletions packages/@aws-cdk/aws-iot/test/iot.test.ts

This file was deleted.

Loading