Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(glue-alpha): Improve Glue version synth time checks #26620

Closed
1 of 2 tasks
dokeita opened this issue Aug 3, 2023 · 4 comments · Fixed by #26650
Closed
1 of 2 tasks

(glue-alpha): Improve Glue version synth time checks #26620

dokeita opened this issue Aug 3, 2023 · 4 comments · Fixed by #26650
Labels
@aws-cdk/aws-glue Related to AWS Glue effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. good first issue Related to contributions. See CONTRIBUTING.md p1

Comments

@dokeita
Copy link

dokeita commented Aug 3, 2023

Describe the feature

we can create PythonShell job with GlueVersion 3.0 using AWS Glue Studio.
I would like to be able to create it in glue-alpha.Job as well.

Use Case

When we create a PythonShell job in AWS Glue Studio, the GlueVersion is set to 3.0.
Therefore, it is better to be able to create PythonShell jobs with GlueVersion 3.0 using CDK.

Proposed Solution

We just need to fix this if statement.

https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-glue-alpha/lib/job-executable.ts#L357

      if ([GlueVersion.V0_9, GlueVersion.V3_0, GlueVersion.V4_0].includes(config.glueVersion)) {
        throw new Error(`Specified GlueVersion ${config.glueVersion.name} does not support Python Shell`);
      }

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.89.0

Environment details (OS name and version, etc.)

Ubuntu 20.04 on Windows10 ( WSL2 )
@dokeita dokeita added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Aug 3, 2023
@github-actions github-actions bot added the @aws-cdk/aws-glue Related to AWS Glue label Aug 3, 2023
@pahud
Copy link
Contributor

pahud commented Aug 3, 2023 •
edited
Loading

related to #26599

Making it a p1 and any PR is welcome and appreciated.

@pahud pahud added p1 effort/small Small work item – less than a day of effort and removed needs-triage This issue or PR still needs to be triaged. labels Aug 3, 2023
@peterwoodworth peterwoodworth added the good first issue Related to contributions. See CONTRIBUTING.md label Aug 3, 2023
@peterwoodworth
Copy link
Contributor

and related to #26616

@peterwoodworth peterwoodworth changed the title (glue-alpha): Enable GlueVersion 3.0 in PythonShell jobs (glue-alpha): Improve Glue version synth time checks Aug 3, 2023
This was referenced Aug 3, 2023
Closed
Closed
@peterwoodworth
Copy link
Contributor

Consolidating all these issues into this one

@lpizzinidev lpizzinidev mentioned this issue Aug 7, 2023
Merged
@mergify mergify bot closed this as completed in #26650 Aug 7, 2023
mergify bot pushed a commit that referenced this issue Aug 7, 2023
@lpizzinidev
fix(glue): synth time validation does not work in Python/Java/C#/Go (#…
dba8cf3 
…26650)

Fixes some issues on Glue Job synth time validation:

- `maxCapacity` can be specified for PythonShell jobs with Glue Version 2.0+ and should be either 0.0625 or 1 (defaults to 0.0625)
- Validation was not working when using `GlueVersion.of()` and `JobType.of()`
- PythonShell jobs supports GlueVersion 3.0

Closes #26620.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

github-actions bot commented Aug 7, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@github-actions github-actions bot mentioned this issue Sep 1, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/aws-glue Related to AWS Glue effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. good first issue Related to contributions. See CONTRIBUTING.md p1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(glue): synth time validation does not work in Python/Java/C#/Go lpizzinidev/aws-cdk
3 participants
@pahud @peterwoodworth @dokeita