Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(glue): Add support for Glue Job Python shell 3.9 #21568

Closed
1 of 2 tasks
tmokmss opened this issue Aug 12, 2022 · 5 comments · Fixed by #21670
Closed
1 of 2 tasks

(glue): Add support for Glue Job Python shell 3.9 #21568

tmokmss opened this issue Aug 12, 2022 · 5 comments · Fixed by #21670
Labels
@aws-cdk/aws-glue Related to AWS Glue effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2

Comments

@tmokmss
Copy link
Contributor

tmokmss commented Aug 12, 2022 •
edited
Loading

Describe the feature

Support Glue Job Python shell 3.9, which was released recently. (AWS blog)

Currently according to the doc, it seems CFn does not yet support Python shell with Python 3.9 though. We should wait for the doc to be updated.

Use Case

Python 3.6 is currently available for Python shell but 3.6 itself has already reached EOL. We need Python 3.9 support.

Proposed Solution

After CFn will support it, update glue.Job construct.

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.37.1

Environment details (OS name and version, etc.)

n/a
@tmokmss tmokmss added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Aug 12, 2022
@github-actions github-actions bot added the @aws-cdk/aws-glue Related to AWS Glue label Aug 12, 2022
@peterwoodworth peterwoodworth added p2 effort/small Small work item – less than a day of effort needs-cfn This issue is waiting on changes to CloudFormation before it can be addressed. and removed needs-triage This issue or PR still needs to be triaged. labels Aug 13, 2022
@peterwoodworth
Copy link
Contributor

Thanks for the request, hopefully Cfn support comes soon

@meve
Copy link
Contributor

meve commented Aug 18, 2022 •
edited
Loading

CloudFormation already supports it, it just seems that the documentation has not been updated properly yet. Even the API documentation itself [0] is not clear about it.

PythonVersion – UTF-8 string, matching the Custom string pattern #16.

The Python version being used to run a Python shell job. Allowed values are 2 or 3.

However aforementioned string pattern 16 reads:

Custom string pattern #16 – "^([2-3]|3[.]9)$"

I just tried to just create a new job using an escape hatch, and this seems to work just fine.

[0] https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-jobs-job.html#aws-glue-api-jobs-job-JobCommand

@tmokmss
Copy link
Contributor Author

tmokmss commented Aug 18, 2022

Hi @meve I also confirmed lt works. I'll still wait for the doc to be updated to properly design the construct API. (I'm expecting all we need to do will be just modifying the existing enum though:)

@peterwoodworth
Copy link
Contributor

@tmokmss the updating of documentation can be a pretty slow process (and can be forgotten about), if the functionality works then we can go ahead and implement it for CDK. Thanks for letting us know @meve

@peterwoodworth peterwoodworth removed the needs-cfn This issue is waiting on changes to CloudFormation before it can be addressed. label Aug 18, 2022
@meve meve mentioned this issue Aug 18, 2022
Merged
4 tasks
@mergify mergify bot closed this as completed in #21670 Aug 29, 2022
mergify bot pushed a commit that referenced this issue Aug 29, 2022
@meve
feat(glue): Added value to PythonVersion enum (#21670)
9774d4c 
PR to fix #21568. Extended the PythonVersion enum to include 3.9, as it already seems to be supported everywhere (CloudFormation, SDK). #21568 (comment)

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

josephedward pushed a commit to josephedward/aws-cdk that referenced this issue Aug 30, 2022
@meve @josephedward
feat(glue): Added value to PythonVersion enum (aws#21670)
b174247 
PR to fix aws#21568. Extended the PythonVersion enum to include 3.9, as it already seems to be supported everywhere (CloudFormation, SDK). aws#21568 (comment)

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
This was referenced Nov 20, 2022
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/aws-glue Related to AWS Glue effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(glue): Added value to PythonVersion enum meve/aws-cdk
4 participants
@meve @tmokmss @kaizencc @peterwoodworth