Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): support for APIGW API: Invoke #11565

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ This module is part of the [AWS Cloud Development Kit](https://github.com/aws/aw
- [ResultPath](#resultpath)
- [Parameters](#task-parameters-from-the-state-json)
- [Evaluate Expression](#evaluate-expression)
- [API Gateway](#api-gateway)
- [Invoke](#invoke)
- [Athena](#athena)
- [StartQueryExecution](#startQueryExecution)
- [GetQueryExecution](#getQueryExecution)
Expand Down Expand Up @@ -211,6 +213,28 @@ runtime to use to evaluate the expression. Currently, the only runtime
supported is `lambda.Runtime.NODEJS_10_X`.


## API Gateway

Step Functions supports [API Gateway](https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html) through the service integration pattern.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just confirming: this is not the same as apigatewayv2 correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not the same.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this works to invoke functions defined via apigatewayv2, correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haven't played with it myself yet, can you clarify @Sumeet-Badyal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From this page - https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html

You use Amazon API Gateway to create, publish, maintain, and monitor HTTP and REST APIs. To integrate with API Gateway, you define a Task state in Step Functions that directly calls an API Gateway HTTP or API Gateway REST endpoint, without writing code or relying on other infrastructure.

This task supports invoking Rest APIs (v1) and HTTP APIs (v2) but not websocket APIs (v2).

Given our modeling of higher level constructs, we'll need to implement this as two tasks (not necessarily in the same PR). One that takes IRestApi from the @aws-cdk/aws-apigateway module and another that takes IHttpApi from the @aws-cdk/aws-apigatewayv2 module.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah; looks like someone still needs to add IHttpApi too

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sumeet-Badyal are you still working on this? Any plans to add the second task for the IHttpApi case? Else when my aws-cdk dev container finishes spinning up I could have a go -- imagine there'd be common code between the two we'd want to abstract into a shared place in the end


### Invoke

The [Invoke](https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html) API calls the API endpoint.

```ts
import * as sfn from '@aws-cdk/aws-stepfunctions';
import * as tasks from `@aws-cdk/aws-stepfunctions-tasks`;

const restApi = new apigateway.RestApi(stack, 'MyRestApi');

const invokeJob = new tasks.ApiGatewayInvoke(stack, 'Invoke APIGW', {
api: restApi,
apiEndpoint: restApi.restApiId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does this need to be an input parameter as well? you're already accepting the restApi as input

stageName: 'prod',
method: ApiGatewayMethodType.GET,
});
```

## Athena

Step Functions supports [Athena](https://docs.aws.amazon.com/step-functions/latest/dg/connect-athena.html) through the service integration pattern.
Expand Down
218 changes: 218 additions & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/lib/apigateway/invoke.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
import * as apigateway from '@aws-cdk/aws-apigateway';
import * as iam from '@aws-cdk/aws-iam';
import * as sfn from '@aws-cdk/aws-stepfunctions';
import * as cdk from '@aws-cdk/core';
import { Construct } from 'constructs';
import { integrationResourceArn, validatePatternSupported } from '../private/task-utils';

/**
* Properties for invoking an API Endpoint with ApiGatewayInvoke
*/
export interface ApiGatewayInvokeProps extends sfn.TaskStateBaseProps {

/** API to call */
readonly api: apigateway.IRestApi;

/**
* hostname of an API Gateway URL
* @example {ApiId}.execute-api.{region}.amazonaws.com
*/
readonly apiEndpoint: string;
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should not be needed anymore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the endpoint can be inferred from the api so you shouldn't need to make this an input anymore


/** Http method for the API */
readonly method: HttpMethod;

/**
* HTTP request information that does not relate to contents of the request
* @default - No headers
* @example
* Headers: {
* type: 1,
* value:{
* 'TaskToken.$': 'States.Array($$.Task.Token)',
* }
* },
*/
readonly headers?: { [key: string]: any };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirming to ensure we have the right type here: "Headers.$" : "$.input" is not valid correct?
i.e. the entire headers field cannot be allocated to a field in the state input

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyone digging into this? Happy to try and help; looking forward to putting this functionality to use ;)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eikeon - happy to have your help here!! :)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this pretty much the sole remaining thing holding up this pull request? Can save some time to attempt to dig into this headers issue later today

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

headers here probably should be similar to input here: packages/@aws-cdk/aws-stepfunctions-tasks/test/start-execution.test.ts and packages/@aws-cdk/aws-stepfunctions-tasks/lib/start-execution.ts -- and they both have the same type. Gotta run for a few hours, but will trace how headers is being used in this pull request compared to other working examples.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

readonly headers?: { [key: string]: string[] }

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also see a string value in one of the examples on that doc page: readonly headers?: { [key: string]: string[] | string }


/**
* Name of the stage where the API is deployed to in API Gateway
* @default - Required for REST and $default for HTTP
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: be sure to describe what $default means as the user might find that information useful in determining whether they need to set it or not.

*/
readonly stageName?: string;

/**
* Path parameters appended after API endpoint
* @default - No path
*/
readonly path?: string;

/**
* Query strings attatched to end of request
* @default - No query parameters
* @example
* "QueryParameters": {
* "billId": ["123", "456"]
* },
*/
readonly queryParameters?: { [key: string]: any };

/**
* HTTP Request body
* @default - No requestBody
* @example
* "RequestBody": {
* "billId": ["my-new-bill"]
* },
*/
readonly requestBody?: sfn.TaskInput;

/**
* Authentication methods
* @default - NO_AUTH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @default - NO_AUTH
* @default AuthType.NO_AUTH

*/
readonly authType?: AuthType;

}

/**
* Invoke an API endpoint as a Task
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html
*/
export class ApiGatewayInvoke extends sfn.TaskStateBase {

private static readonly SUPPORTED_INTEGRATION_PATTERNS: sfn.IntegrationPattern[] = [
sfn.IntegrationPattern.REQUEST_RESPONSE,
sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN,
];
protected readonly taskMetrics?: sfn.TaskMetricsConfig;
protected readonly taskPolicies?: iam.PolicyStatement[];
protected readonly apiEndpoint: string;

private readonly integrationPattern: sfn.IntegrationPattern;

constructor(scope: Construct, id: string, private readonly props: ApiGatewayInvokeProps) {
super(scope, id, props);
this.integrationPattern = props.integrationPattern ?? sfn.IntegrationPattern.REQUEST_RESPONSE;

validatePatternSupported(this.integrationPattern, ApiGatewayInvoke.SUPPORTED_INTEGRATION_PATTERNS);

this.taskPolicies = this.createPolicyStatements();
this.apiEndpoint = this.createApiEndpoint();
}

/**
* Provides the API Gateway Invoke service integration task configuration
*/
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*/
/**

* @internal
*/
protected _renderTask(): any {
return {
Resource: integrationResourceArn('apigateway', 'invoke', this.integrationPattern),
Parameters: sfn.FieldUtils.renderObject({
ApiEndpoint: this.apiEndpoint,
Method: this.props.method,
Headers: this.props.headers,
Stage: this.props.stageName,
Path: this.props.path,
QueryParameters: this.props.queryParameters,
RequestBody: this.props.requestBody,
AuthType: this.props.authType ? this.props.authType : 'NO_AUTH',
}),
};
}

/**
* Gets the "execute-api" ARN
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think the doc string should be a little more informative than the method name. Perhaps describe what that ARN is / what it's used for, show an example of what it looks like, etc.

* @returns The "execute-api" ARN.
* @default "*" returns the execute API ARN for all methods/resources in
* this API.
* @param method The method (default `*`)
* @param path The resource path. Must start with '/' (default `*`)
* @param stage The stage (default `*`)
*/
get arnForExecuteApi() {
return this.props.api.arnForExecuteApi(this.props.method, this.props.path, this.props.stageName);
}

/**
* Generates the api endpoint
* @returns The api id
* @example {ApiId}.execute-api.{region}.amazonaws.com
*/
private createApiEndpoint(): string {
const apiStack = cdk.Stack.of(this.props.api);
return `${this.props.api.restApiId}.execute-api.${apiStack.region}.${apiStack.urlSuffix}`;
}

/**
* This generates the PolicyStatements required by the Task to call invoke.
*/
private createPolicyStatements(): iam.PolicyStatement[] {
if (this.props.authType === AuthType.IAM_ROLE) {
return [
new iam.PolicyStatement({
resources: [this.arnForExecuteApi],
actions: ['ExecuteAPI:Invoke'],
}),
];
} else if (this.props.authType === AuthType.RESOURCE_POLICY) {
if (!sfn.FieldUtils.containsTaskToken(this.props.headers)) {
throw new Error('Task Token is required in `headers` Use JsonPath.taskToken to set the token.');
}
return [
new iam.PolicyStatement({
resources: [this.arnForExecuteApi],
actions: ['ExecuteAPI:Invoke'],
conditions: {
StringEquals: {
'aws:SourceArn': '*',
},
},
}),
];
}
return [];
}
}

/** Http Methods that API Gateway supports */
export enum HttpMethod {
Sumeet-Badyal marked this conversation as resolved.
Show resolved Hide resolved
/** Retreive data from a server at the specified resource */
GET = 'GET',

/** Send data to the API endpoint to create or udpate a resource */
POST = 'POST',

/** Send data to the API endpoint to update or create a resource */
PUT = 'PUT',

/** Delete the resource at the specified endpoint */
DELETE = 'DELETE',

/** Apply partial modifications to the resource */
PATCH = 'PATCH',

/** Retreive data from a server at the specified resource without the response body */
HEAD = 'HEAD',

/** Return data describing what other methods and operations the server supports */
OPTIONS = 'OPTIONS'
}

/**
* The authentication method used to call the endpoint
* @default NO_AUTH
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enums don't have defaults, it's only where they're used that would

Suggested change
* @default NO_AUTH

*/
export enum AuthType {
/** Call the API direclty with no authorization method */
NO_AUTH = 'NO_AUTH',

/** * Use the IAM role associated with the current state machine for authorization */
IAM_ROLE = 'IAM_ROLE',

/** Use the resource policy of the API for authorization */
RESOURCE_POLICY = 'RESOURCE_POLICY',
}
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,4 @@ export * from './athena/start-query-execution';
export * from './athena/stop-query-execution';
export * from './athena/get-query-execution';
export * from './athena/get-query-results';
export * from './apigateway/invoke';
2 changes: 2 additions & 0 deletions packages/@aws-cdk/aws-stepfunctions-tasks/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
},
"dependencies": {
"@aws-cdk/assets": "0.0.0",
"@aws-cdk/aws-apigateway": "0.0.0",
"@aws-cdk/aws-batch": "0.0.0",
"@aws-cdk/aws-cloudwatch": "0.0.0",
"@aws-cdk/aws-codebuild": "0.0.0",
Expand All @@ -96,6 +97,7 @@
"homepage": "https://github.com/aws/aws-cdk",
"peerDependencies": {
"@aws-cdk/assets": "0.0.0",
"@aws-cdk/aws-apigateway": "0.0.0",
"@aws-cdk/aws-batch": "0.0.0",
"@aws-cdk/aws-cloudwatch": "0.0.0",
"@aws-cdk/aws-codebuild": "0.0.0",
Expand Down
Loading