Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(batch): new L2 Constructs #24775

Merged
merged 75 commits into from
Apr 7, 2023
Merged

feat(batch): new L2 Constructs #24775

merged 75 commits into from
Apr 7, 2023

Conversation

comcalvi
Copy link
Contributor

@comcalvi comcalvi commented Mar 23, 2023

Adds a new set of L2s for Batch. See https://github.com/aws/aws-cdk-rfcs/pull/484/files for details.

This contains breaking changes to the alpha module! To use the previous API, pin your version of aws-batch-alpha to 2.71.0.

BREAKING CHANGE: ComputeEnvironment has been removed and replaced by ManagedEc2EcsComputeEnvironment, ManagedEc2EksComputeEnvironment, and UnmanagedComputeEnvironment.

JobDefinition has been removed and replaced by EcsJobDefinition, EksJobDefinition, and MultiNodeJobDefinition


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team March 23, 2023 23:01
@comcalvi comcalvi removed the pr/do-not-merge This PR should not be merged at this time. label Apr 7, 2023
@mergify
Copy link
Contributor

mergify bot commented Apr 7, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 34b5ba5
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 92e6c67 into aws:main Apr 7, 2023
@mergify
Copy link
Contributor

mergify bot commented Apr 7, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@rjuengling-hf
Copy link

Prior to this change container secrets could be added to a JobDefinition this way:

db_secret = secretsmanager.Secret(self, "secret")

batch.JobDefinition(self, "batch-job-def-secrets",
    container=batch.JobDefinitionContainer(
        image=ecs.EcrImage.from_registry("docker/whalesay"),
        secrets={
            "PASSWORD": ecs.Secret.from_secrets_manager(db_secret, "password")
        }
    )
)

I suppose now the secrets need to be set via EcsEc2ContainerDefinitiont & co. I have tried a few things but have not found a way to do this. Could you please provide an example such as the one I quoted, for the new API?

@comcalvi
Copy link
Contributor Author

comcalvi commented Apr 19, 2023

Thanks for taking the new API for a spin! You should be able to do this now:

import * as cdk from 'aws-cdk-lib';
import * as secrets from 'aws-cdk-lib/aws-secretsmanager';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import { Construct } from 'constructs';
import * as batch from '@aws-cdk/aws-batch-alpha';

export class FooStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new batch.EcsJobDefinition(this, 'job', {
      container: new batch.EcsEc2ContainerDefinition(this, 'container', {
        cpu: 256,
        image: ecs.ContainerImage.fromRegistry('foo'),
        memory: cdk.Size.mebibytes(2048),
        secrets: [new secrets.Secret(this, 'mysecret')],
      }),
    });
  }
}
from aws_cdk import (
    Stack,
    aws_ecs as ecs,
    aws_secretsmanager as secrets,
    Size,
)
from constructs import Construct
from aws_cdk.aws_batch_alpha import EcsJobDefinition, EcsEc2ContainerDefinition

class FooPyStack(Stack):

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)
        job_defn = EcsJobDefinition(self, "JobDefn",
            container=EcsEc2ContainerDefinition(self, "containerDefn",
            image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
            memory=Size.mebibytes(2048),
            cpu=256,
            secrets=[secrets.Secret(self, 'Secret')]
        )
    )

@rjuengling-hf
Copy link

Thanks. What I am not seeing in your example is the mapping from a secret to an environment variable set in the Docker container. In the example I quoted a secret is made available in the Docker container per environment variable PASSWORD.

How is the secret to be accessed in the Docker container with the new API?

@comcalvi
Copy link
Contributor Author

comcalvi commented Apr 21, 2023

I think this is the feature you're talking about: #20871

This is not supported in the new APIs, so I can add support for it; should just be changing the type from Secret[] to { [key: string]: Secret }

spotaws referenced this pull request in aws-samples/aws-cdk-examples Apr 25, 2023
Pinning aws_batch_alpha to 2.71.0 to use the previous API. Current API has this failure:

Traceback (most recent call last):
  File "app.py", line 1, in <module>
    from aws_cdk import (aws_ec2 as ec2,
  File "/tmp/.venv/lib/python3.7/site-packages/aws_cdk/aws_batch_alpha/__init__.py", line 6505, in <module>
    typing_extensions.Protocol,
  File "/tmp/.venv/lib/python3.7/site-packages/aws_cdk/aws_batch_alpha/__init__.py", line 6606, in IManagedEc2EcsComputeEnvironment
    ) -> typing.Optional[_aws_cdk_aws_ec2_ceddda9d.IPlacementGroup]:
AttributeError: module 'aws_cdk.aws_ec2' has no attribute 'IPlacementGroup'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants