Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pipes-targets): add CloudWatch Logs #30665

Merged
merged 22 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 132 additions & 99 deletions packages/@aws-cdk/aws-pipes-targets-alpha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,118 +17,161 @@
<!--END STABILITY BANNER-->


EventBridge Pipes Targets let you create a target for a EventBridge Pipe.
EventBridge Pipes Targets let you create a target for an EventBridge Pipe.

For more details see the service documentation:

[Documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html)
For more details see the [service documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html).

## Targets

Pipe targets are the end point of a EventBridge Pipe.
Pipe targets are the end point of an EventBridge Pipe. The following targets are supported:

* `targets.ApiDestinationTarget`: [Send event source to an EventBridge API destination](#amazon-eventbridge-api-destination)
* `targets.CloudWatchLogsTarget`: [Send event source to a CloudWatch Logs log group](#amazon-cloudwatch-logs-log-group)
* `targets.EventBridgeTarget`: [Send event source to an EventBridge event bus](#amazon-eventbridge-event-bus)
* `targets.KinesisTarget`: [Send event source to a Kinesis data stream](#amazon-kinesis-data-stream)
* `targets.LambdaFunction`: [Send event source to a Lambda function](#aws-lambda-function)
* `targets.SfnStateMachine`: [Invoke a Step Functions state machine from an event source](#aws-step-functions-state-machine)
* `targets.SqsTarget`: [Send event source to an SQS queue](#amazon-sqs)

### Amazon EventBridge API Destination

The following targets are supported:
An EventBridge API destination can be used as a target for a pipe.
The API destination will receive the (enriched/filtered) source payload.

1. `targets.SqsTarget`: [Send event source to an SQS queue](#amazon-sqs)
2. `targets.SfnStateMachine`: [Invoke a State Machine from an event source](#aws-step-functions-state-machine)
3. `targets.LambdaFunction`: [Send event source to a Lambda function](#aws-lambda-function)
4. `targets.ApiDestinationTarget`: [Send event source to an EventBridge API destination](#amazon-eventbridge-api-destination)
5. `targets.KinesisTarget`: [Send event source to a Kinesis data stream](#amazon-kinesis-data-stream)
6. `targets.EventBridgeTarget`: [Send event source to an EventBridge event bus](#amazon-eventbridge-event-bus)
```ts
declare const sourceQueue: sqs.Queue;
declare const dest: events.ApiDestination;

const apiTarget = new targets.ApiDestinationTarget(dest);

### Amazon SQS
const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: apiTarget,
});
```

A SQS message queue can be used as a target for a pipe. Messages will be pushed to the queue.
The input to the target API destination can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetQueue: sqs.Queue;
declare const dest: events.ApiDestination;

const pipeTarget = new targets.SqsTarget(targetQueue);
const apiTarget = new targets.ApiDestinationTarget(dest, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: apiTarget,
});
```

The target input can be transformed:
### Amazon CloudWatch Logs Log Group

A CloudWatch Logs log group can be used as a target for a pipe.
The log group will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetQueue: sqs.Queue;
declare const targetLogGroup: logs.LogGroup;

const pipeTarget = new targets.SqsTarget(targetQueue,
{
inputTransformation: pipes.InputTransformation.fromObject(
{
"SomeKey": pipes.DynamicInput.fromEventPath('$.body')
})
}
);
const logGroupTarget = new targets.CloudWatchLogsTarget(targetLogGroup);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: logGroupTarget,
});
```

### AWS Step Functions State Machine
The input to the target log group can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetLogGroup: logs.LogGroup;

const logGroupTarget = new targets.CloudWatchLogsTarget(targetLogGroup, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

A State Machine can be used as a target for a pipe. The State Machine will be invoked with the (enriched) source payload.
const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: logGroupTarget,
});
```

### Amazon EventBridge Event Bus

An EventBridge event bus can be used as a target for a pipe.
The event bus will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStateMachine: sfn.IStateMachine;
declare const targetEventBus: events.EventBus;

const pipeTarget = new targets.SfnStateMachine(targetStateMachine,{});
const eventBusTarget = new targets.EventBridgeTarget(targetEventBus);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: eventBusTarget,
});
```

Specifying the Invocation Type when the target State Machine is invoked:
The input to the target event bus can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStateMachine: sfn.IStateMachine;
declare const targetEventBus: events.EventBus;

const pipeTarget = new targets.SfnStateMachine(targetStateMachine,
{
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
}
);
const eventBusTarget = new targets.EventBridgeTarget(targetEventBus, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: eventBusTarget,
});
```

The input to the target State Machine can be transformed:
### Amazon Kinesis Data Stream

A Kinesis data stream can be used as a target for a pipe.
The data stream will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStateMachine: sfn.IStateMachine;
declare const targetStream: kinesis.Stream;

const pipeTarget = new targets.SfnStateMachine(targetStateMachine,
{
inputTransformation: pipes.InputTransformation.fromObject({ body: '<$.body>' }),
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
}
);
const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: streamTarget,
});
```

The input to the target data stream can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStream: kinesis.Stream;

const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: streamTarget,
});
```

### AWS Lambda Function

A Lambda Function can be used as a target for a pipe. The Lambda Function will be invoked with the (enriched) source payload.
A Lambda function can be used as a target for a pipe.
The Lambda function will be invoked with the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
Expand All @@ -142,7 +185,7 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
});
```

The target Lambda Function is invoked synchronously by default. You can also choose to invoke the Lambda Function asynchronously by setting `invocationType` property to `FIRE_AND_FORGET`.
The target Lambda function is invoked synchronously by default. You can also choose to invoke the Lambda Function asynchronously by setting `invocationType` property to `FIRE_AND_FORGET`.

```ts
declare const sourceQueue: sqs.Queue;
Expand Down Expand Up @@ -174,102 +217,92 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
});
```

### Amazon EventBridge API Destination
### AWS Step Functions State Machine

An EventBridge API destination can be used as a target for a pipe.
The API destination will receive the (enriched/filtered) source payload.
A Step Functions state machine can be used as a target for a pipe.
The state machine will be invoked with the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const dest: events.ApiDestination;
declare const targetStateMachine: sfn.IStateMachine;

const apiTarget = new targets.ApiDestinationTarget(dest);
const pipeTarget = new targets.SfnStateMachine(targetStateMachine,{});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: apiTarget,
target: pipeTarget
});
```

The input to the target API destination can be transformed:
You can specify the invocation type when the target state machine is invoked:

```ts
declare const sourceQueue: sqs.Queue;
declare const dest: events.ApiDestination;
declare const targetStateMachine: sfn.IStateMachine;

const apiTarget = new targets.ApiDestinationTarget(dest, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
const pipeTarget = new targets.SfnStateMachine(targetStateMachine, {
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: apiTarget,
});
```

### Amazon Kinesis Data Stream

A data stream can be used as a target for a pipe. The data stream will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStream: kinesis.Stream;

const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: streamTarget,
target: pipeTarget
});
```

The input to the target data stream can be transformed:
The input to the target state machine can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStream: kinesis.Stream;
declare const targetStateMachine: sfn.IStateMachine;

const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});
const pipeTarget = new targets.SfnStateMachine(targetStateMachine,
{
inputTransformation: pipes.InputTransformation.fromObject({ body: '<$.body>' }),
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
}
);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: streamTarget,
target: pipeTarget
});
```

### Amazon EventBridge Event Bus
### Amazon SQS Queue

An event bus can be used as a target for a pipe. The event bus will receive the (enriched/filtered) source payload.
An SQS queue can be used as a target for a pipe.
The queue will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetEventBus: events.EventBus;
declare const targetQueue: sqs.Queue;

const eventBusTarget = new targets.EventBridgeTarget(targetEventBus);
const pipeTarget = new targets.SqsTarget(targetQueue);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: eventBusTarget,
target: pipeTarget
});
```

The input to the target event bus can be transformed:
The target input can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetEventBus: events.EventBus;
declare const targetQueue: sqs.Queue;

const eventBusTarget = new targets.EventBridgeTarget(targetEventBus, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});
const pipeTarget = new targets.SqsTarget(targetQueue,
{
inputTransformation: pipes.InputTransformation.fromObject(
{
"SomeKey": pipes.DynamicInput.fromEventPath('$.body')
})
}
);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: eventBusTarget,
target: pipeTarget
});
```
Loading