(aws_stepfunctions_tasks): (Add support to start glue crawlers) #24188
Labels
@aws-cdk/aws-stepfunctions-tasks
effort/small
Small work item – less than a day of effort
feature-request
A feature should be added or improved.
p2
Comments
rodgjame
added
feature-request
|
We also need the same feature. up voting this. |
|
Hi @rodgjame , Thanks for reaching out. I am marking this issue as p2, the suggested request should adhere to the guidelines of existing Stepfunction task construct. We use +1s to help prioritize our work, and are happy to revaluate this issue based on community feedback. You can reach out to the cdk.dev community on Slack to solicit support for reprioritization. Sharing the contributing guide for reference. |
khushail
added
p2
effort/small
|
+1 |
|
+1 |
6 similar comments
|
+1 |
|
+1 |
|
+1 |
|
+1 |
|
+1 |
|
+1 |
1 task
GavinZZ
added a commit
that referenced
this issue
Mar 11, 2024
### Issue Closes #24188. ### Reason for this change AWS Step Functions supports the ability to start a Glue Crawler as a task within a state machine. However, this is not configurable. ### Description of changes I've implemented `GlueStartCrawlerRun` class in stepfunctions-tasks and we can create start crawler task easily: ```ts const task = new GlueStartCrawlerRun(stack, 'Task', { crawlerName: 'glue-crawler-name', }); ``` ### Description of how you validated changes I've added both unit and integ tests. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license* --------- Co-authored-by: Luca Pizzini <lpizzini7@gmail.com> Co-authored-by: GZ <yuanhaoz@amazon.com>
|
This was referenced May 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the feature
As a user of aws-cdk, I would like to create a step function with a step to start a Glue Crawler so I don't have invoke a crawler via a Lambda. Lambdas have a 15 min timeout which is too short for some crawlers of large datasets to be fully crawled.
Use Case
When I'm in the step function design studio, I do see an option for a
StartCrawlerstep. This step is not available in the CDK modules so I'm unable to create step functions that has this step in it.Proposed Solution
I would like a new task type that allows me to kick off a glue crawler. Something like:
Other Information
No response
Acknowledgements
CDK version used
"aws-cdk": "^2.58.1"
Environment details (OS name and version, etc.)
Amazon Linux 2
The text was updated successfully, but these errors were encountered: