Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(glue): support Ray jobs with Runtime parameter #25867

Merged
merged 8 commits into from
Jun 29, 2023

Conversation

moomindani
Copy link
Contributor

AWS Glue recently changed API and CloudFormation spec to require Runtime parameter for Ray jobs.
To address the changes, I am submitting this PR.
This PR is for fixing the issue #25787.

Reference

CloudFormation doc has been also updated for Runtime parameter.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-job-jobcommand.html#cfn-glue-job-jobcommand-runtime


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Jun 6, 2023

@aws-cdk-automation aws-cdk-automation requested a review from a team June 6, 2023 10:44
@github-actions github-actions bot added p2 beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK labels Jun 6, 2023
/**
* Ray 2.4
*/
RAY_TWO_FOUR = 'Ray2.4',

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a repetitive operation, wondering if we want to move away from enums

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be an enum-like class:

export class Runtime {
  public static RAY_TWO_FOUR = new Runtime('Ray2.4');
  public constructor(public readonly runtime: string) {}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sharing the example. Reflected with minor change.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jun 6, 2023
Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting started on this @moomindani! Some minor comments

/**
* Ray 2.4
*/
RAY_TWO_FOUR = 'Ray2.4',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be an enum-like class:

export class Runtime {
  public static RAY_TWO_FOUR = new Runtime('Ray2.4');
  public constructor(public readonly runtime: string) {}
}

@@ -150,6 +160,12 @@ interface PythonExecutableProps {
}

interface SharedJobExecutableProps {
/**
* Runtime.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stronger docs needed. need to know when is runtime required, etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added bit more information.
Our public doc does not have much information about this property, but I tried to add as much as possible.

@@ -388,6 +407,13 @@ export interface JobExecutableConfig {
*/
readonly pythonVersion?: PythonVersion;

/**
* The Runtime to use.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better docs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our public doc does not have much information about this property, but I tried to add as much as possible.

@mergify mergify bot dismissed kaizencc’s stale review June 8, 2023 02:18

Pull request has been modified.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: cbaf88c
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Jun 29, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 8153237 into aws:main Jun 29, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants