Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): support for APIGW API: Invoke #11565

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ This module is part of the [AWS Cloud Development Kit](https://github.com/aws/aw
- [ResultPath](#resultpath)
- [Parameters](#task-parameters-from-the-state-json)
- [Evaluate Expression](#evaluate-expression)
- [API Gateway](#api-gateway)
- [Invoke](#invoke)
- [Athena](#athena)
- [StartQueryExecution](#startQueryExecution)
- [GetQueryExecution](#getQueryExecution)
Expand Down Expand Up @@ -211,6 +213,25 @@ runtime to use to evaluate the expression. Currently, the only runtime
supported is `lambda.Runtime.NODEJS_10_X`.


## API Gateway

Step Functions supports [API Gateway](https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html) through the service integration pattern.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just confirming: this is not the same as apigatewayv2 correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not the same.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this works to invoke functions defined via apigatewayv2, correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haven't played with it myself yet, can you clarify @Sumeet-Badyal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From this page - https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html

You use Amazon API Gateway to create, publish, maintain, and monitor HTTP and REST APIs. To integrate with API Gateway, you define a Task state in Step Functions that directly calls an API Gateway HTTP or API Gateway REST endpoint, without writing code or relying on other infrastructure.

This task supports invoking Rest APIs (v1) and HTTP APIs (v2) but not websocket APIs (v2).

Given our modeling of higher level constructs, we'll need to implement this as two tasks (not necessarily in the same PR). One that takes IRestApi from the @aws-cdk/aws-apigateway module and another that takes IHttpApi from the @aws-cdk/aws-apigatewayv2 module.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah; looks like someone still needs to add IHttpApi too

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sumeet-Badyal are you still working on this? Any plans to add the second task for the IHttpApi case? Else when my aws-cdk dev container finishes spinning up I could have a go -- imagine there'd be common code between the two we'd want to abstract into a shared place in the end


### Invoke

The [Invoke](https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html) API calls the API endpoint.

```ts
import * as sfn from '@aws-cdk/aws-stepfunctions';
import * as tasks from `@aws-cdk/aws-stepfunctions-tasks`;

const invokeJob = new tasks.ApiGatewayInvoke(stack, 'Invoke APIGW', {
apiEndpoint: 'APIID.execute-api.REGION.amazonaws.com',
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
stage: 'prod',
method: ApiGatewayMethodType.GET,
});
```

## Athena

Step Functions supports [Athena](https://docs.aws.amazon.com/step-functions/latest/dg/connect-athena.html) through the service integration pattern.
Expand Down
277 changes: 277 additions & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/lib/apigateway/invoke.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,277 @@
import * as iam from '@aws-cdk/aws-iam';
import * as sfn from '@aws-cdk/aws-stepfunctions';
import { Stack } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { integrationResourceArn, validatePatternSupported } from '../private/task-utils';

/**
* Properties for invoking an API Endpoint with ApiGatewayInvoke
*/
export interface ApiGatewayInvokeProps extends sfn.TaskStateBaseProps {

/**
* hostname of an API Gateway URL
* @example {ApiId}.execute-api.{region}.amazonaws.com
*/
readonly apiEndpoint: string;
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should not be needed anymore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the endpoint can be inferred from the api so you shouldn't need to make this an input anymore


/**
* Http method for the API
*/
readonly method: HttpMethod;

/**
* HTTP headers string to list of strings
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can provide a more informative doc string here. maybe throw in an example too?

* @default - No headers
*/
readonly headers?: sfn.TaskInput;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be a TaskInput? - is that because they can be passed through the state input as too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a json. I can change the type.


/**
* Name of the stage where the API is deployed to in API Gateway
* @default - Required for REST and $default for HTTP
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: be sure to describe what $default means as the user might find that information useful in determining whether they need to set it or not.

*/
readonly stage?: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is called stageName in the @aws-cdk/aws-apigateway module - we should align rather than maintain the naming from the Api


/**
* Path parameters appended after API endpoint
* @default - No path
*/
readonly path?: string;

/**
* Query strings string to list of strings
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an example might communicate this better

* @default - No query parameters
*/
readonly queryParameters?: sfn.TaskInput;

/**
* HTTP Request body
* @default - No requestBody
*/
readonly requestBody?: sfn.TaskInput;
/**
* Authentication methods
*
* NO_AUTH: call the API direclty with no authorization method
*
* IAM_ROLE: Use the IAM role associated with the current state machine for authorization
*
* RESOURCE_POLICY: Use the resource policy of the API for authorization
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
*
* @default - NO_AUTH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @default - NO_AUTH
* @default AuthType.NO_AUTH

*/
readonly authType?: sfn.TaskInput;
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved

}
/**
* Invoke an API endpoint as a Task
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html
*/
export class ApiGatewayInvoke extends sfn.TaskStateBase {

private static readonly SUPPORTED_INTEGRATION_PATTERNS: sfn.IntegrationPattern[] = [
sfn.IntegrationPattern.REQUEST_RESPONSE,
sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN,
];

protected readonly taskMetrics?: sfn.TaskMetricsConfig;
protected readonly taskPolicies?: iam.PolicyStatement[];

private readonly integrationPattern: sfn.IntegrationPattern;

constructor(scope: Construct, id: string, private readonly props: ApiGatewayInvokeProps) {
super(scope, id, props);
this.integrationPattern = props.integrationPattern ?? sfn.IntegrationPattern.REQUEST_RESPONSE;

validatePatternSupported(this.integrationPattern, ApiGatewayInvoke.SUPPORTED_INTEGRATION_PATTERNS);
const authType = this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value;
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
if (authType === 'IAM_ROLE') {
const resource = props.apiEndpoint.split('.', 1)[0] + '/' + (props.stage ? props.stage + '/' : '$default/') + props.method + '/' + (props.path ?? '');
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved

this.taskPolicies = [
new iam.PolicyStatement({
resources: [
Stack.of(this).formatArn({
service: 'execute-api',
resource: resource,
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
}),
],
actions: ['execute-api:Invoke'],
}),
];
} else if (authType === 'RESOURCE_POLICY') {
if (!sfn.FieldUtils.containsTaskToken(props.headers)) {
throw new Error('Task Token is required in `headers` Use JsonPath.taskToken to set the token.');
}
const resource = props.apiEndpoint.split('.', 1)[0] + '/' + (props.stage ? props.stage + '/' : '') + props.method + '/' + (props.path ? props.path + '/*' : '*');

this.taskPolicies = [
new iam.PolicyStatement({
resources: [
Stack.of(this).formatArn({
service: 'execute-api',
resource: resource,
}),
],
actions: ['execute-api:Invoke'],
conditions: {
StringEquals: {
'aws:SourceArn': '*',
},
},
}),
];
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
}
}

/**
* Provides the API Gateway Invoke service integration task configuration
*/
/**
* @internal
*/
protected _renderTask(): any {
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
if (this.props.headers && this.props.queryParameters && this.props.requestBody) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
Method: this.props.method,
Headers: this.props.headers ? this.props.headers.value : sfn.TaskInput.fromDataAt('$.Headers').value,
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
Stage: this.props.stage,
Path: this.props.path,
QueryParameters: this.props.queryParameters ? this.props.queryParameters.value : sfn.TaskInput.fromDataAt('$.QueryParameters').value,
RequestBody: this.props.requestBody ? this.props.requestBody.value : sfn.TaskInput.fromDataAt('$.RequestBody').value,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else if (this.props.headers && this.props.queryParameters) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Headers: this.props.headers ? this.props.headers.value : sfn.TaskInput.fromDataAt('$.Headers').value,
Stage: this.props.stage,
Path: this.props.path,
QueryParameters: this.props.queryParameters ? this.props.queryParameters.value : sfn.TaskInput.fromDataAt('$.QueryParameters').value,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else if (this.props.queryParameters && this.props.requestBody) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Stage: this.props.stage,
Path: this.props.path,
QueryParameters: this.props.queryParameters ? this.props.queryParameters.value : sfn.TaskInput.fromDataAt('$.QueryParameters').value,
RequestBody: this.props.requestBody ? this.props.requestBody.value : sfn.TaskInput.fromDataAt('$.RequestBody').value,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else if (this.props.headers && this.props.requestBody) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Headers: this.props.headers ? this.props.headers.value : sfn.TaskInput.fromDataAt('$.Headers').value,
Stage: this.props.stage,
Path: this.props.path,
RequestBody: this.props.requestBody ? this.props.requestBody.value : sfn.TaskInput.fromDataAt('$.RequestBody').value,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else if (this.props.headers) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Headers: this.props.headers ? this.props.headers.value : sfn.TaskInput.fromDataAt('$.Headers').value,
Stage: this.props.stage,
Path: this.props.path,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else if (this.props.queryParameters) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Stage: this.props.stage,
Path: this.props.path,
QueryParameters: this.props.queryParameters ? this.props.queryParameters.value : sfn.TaskInput.fromDataAt('$.QueryParameters').value,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else if (this.props.requestBody) {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Stage: this.props.stage,
Path: this.props.path,
RequestBody: this.props.requestBody ? this.props.requestBody.value : sfn.TaskInput.fromDataAt('$.RequestBody').value,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
} else {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.props.apiEndpoint,
Method: this.props.method,
Stage: this.props.stage,
Path: this.props.path,
AuthType: (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) === '$' ? (this.props.authType ? this.props.authType.value : sfn.TaskInput.fromDataAt('$.AuthType').value) : 'NO_AUTH',
}),
};
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a lot of duplicated code in this control flow chain from the if on down. I think we can simplify this to be a lot more succinct. let's refactor this please!

}

/**
* Http Methods that API Gateway supports
*/
export enum HttpMethod {
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
/**
* Retreive data from a server at the specified resource
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/**
* Retreive data from a server at the specified resource
*/
/** Retrieve data from a server at the specified resource */

GET = 'GET',

/**
* Send data to the API endpoint to create or udpate a resource
*/
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
POST = 'POST',

/**
* Send data to the API endpoint to update or create a resource
*/
PUT = 'PUT',

/**
* Delete the resource at the specified endpoint
*/
DELETE = 'DELETE',

/**
* Apply partial modifications to the resource
*/
PATCH = 'PATCH',

/**
* Retreive data from a server at the specified resource without the response body
*/
HEAD = 'HEAD',

/**
* Return data describing what other methods and operations the server supports
*/
OPTIONS = 'OPTIONS'
}
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,4 @@ export * from './athena/start-query-execution';
export * from './athena/stop-query-execution';
export * from './athena/get-query-execution';
export * from './athena/get-query-results';
export * from './apigateway/invoke';
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
{
"Resources": {
"StateMachineRoleB840431D": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": {
"Fn::Join": [
"",
[
"states.",
{
"Ref": "AWS::Region"
},
".amazonaws.com"
]
]
}
}
}
],
"Version": "2012-10-17"
}
}
},
"StateMachine2E01A3A5": {
"Type": "AWS::StepFunctions::StateMachine",
"Properties": {
"RoleArn": {
"Fn::GetAtt": [
"StateMachineRoleB840431D",
"Arn"
]
},
"DefinitionString": {
"Fn::Join": [
"",
[
"{\"StartAt\":\"Invoke APIGW\",\"States\":{\"Invoke APIGW\":{\"End\":true,\"Type\":\"Task\",\"Resource\":\"arn:",
{
"Ref": "AWS::Partition"
},
":states:::apigateway:invoke\",\"Parameters\":{\"ApiEndpoint\":\"apiid.execute-api.us-east-1.amazonaws.com\",\"Method\":\"GET\",\"Stage\":\"prod\",\"AuthType\":\"NO_AUTH\"}}},\"TimeoutSeconds\":30}"
]
]
}
},
"DependsOn": [
"StateMachineRoleB840431D"
]
}
},
"Outputs": {
"stateMachineArn": {
"Value": {
"Ref": "StateMachine2E01A3A5"
}
}
}
}
Loading