Skip to content

Commit

Permalink
feat(batch): ComputeEnvironment implements IConnectable (aws#21458)
Browse files Browse the repository at this point in the history
closes aws#20983

ComputeEnvironments have embedded security groups in them.  These are currently difficult to reach and modify in CDK stacks, which forces users into undesirable practices when integrating batch queues with other services such as as EFS filesystems or RDS instances.  Ideally, it should be possible to use compute environments as a target:

```ts
something.connections.allowDefaultPortFrom(computeEnvironment);
```

but this isn't currently possible, so the user may try to work around it by allowing from any IPv4, or by having to use an escape hatch, which is not simple.

This pull request adds ec2.IConnectable to batch.ComputeEnvironment.  It still seems to pass all the existing tests and integration tests, so I don't think it's a breaking change, but I suspect it could be done in a better way than I've done it, to make things even simpler for the user.

----

### All Submissions:

* [yes] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [no] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [yes] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [yes] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
tcutts authored and josephedward committed Aug 30, 2022
1 parent 509dd6f commit 113baa3
Show file tree
Hide file tree
Showing 11 changed files with 5,446 additions and 7 deletions.
16 changes: 16 additions & 0 deletions packages/@aws-cdk/aws-batch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,22 @@ const spotEnvironment = new batch.ComputeEnvironment(this, 'MySpotEnvironment',
});
```

### Compute Environments and Security Groups

Compute Environments implement the `IConnectable` interface, which means you can use
connections on other CDK resources to manipulate the security groups and allow access.

For example, allowing a Compute Environment to access an EFS filesystem:

```ts
import * as efs from '@aws-cdk/aws-efs';

declare const fileSystem: efs.FileSystem;
declare const computeEnvironment: batch.ComputeEnvironment;

fileSystem.connections.allowDefaultPortFrom(computeEnvironment);
```

### Fargate Compute Environment

It is possible to have AWS Batch submit jobs to be run on Fargate compute resources. Below is an example of how this can be done:
Expand Down
37 changes: 30 additions & 7 deletions packages/@aws-cdk/aws-batch/lib/compute-environment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,7 @@ export interface IComputeEnvironment extends IResource {
*
* Defines a batch compute environment to run batch jobs on.
*/
export class ComputeEnvironment extends Resource implements IComputeEnvironment {
export class ComputeEnvironment extends Resource implements IComputeEnvironment, ec2.IConnectable {
/**
* Fetches an existing batch compute environment by its amazon resource name.
*
Expand Down Expand Up @@ -357,6 +357,11 @@ export class ComputeEnvironment extends Resource implements IComputeEnvironment
*/
public readonly computeEnvironmentName: string;

/**
* Connections for this compute environment.
*/
public readonly connections: ec2.Connections;

constructor(scope: Construct, id: string, props: ComputeEnvironmentProps = { enabled: true, managed: true }) {
super(scope, id, {
physicalName: props.computeEnvironmentName,
Expand All @@ -370,8 +375,11 @@ export class ComputeEnvironment extends Resource implements IComputeEnvironment
const spotFleetRole = this.getSpotFleetRole(props);
let computeResources: CfnComputeEnvironment.ComputeResourcesProperty | undefined;

this.connections = this.buildConnections(props.computeResources?.vpc, props.computeResources?.securityGroups);

// Only allow compute resources to be set when using MANAGED type
if (props.computeResources && this.isManaged(props)) {

computeResources = {
bidPercentage: props.computeResources.bidPercentage,
desiredvCpus: props.computeResources.desiredvCpus,
Expand All @@ -380,7 +388,7 @@ export class ComputeEnvironment extends Resource implements IComputeEnvironment
launchTemplate: props.computeResources.launchTemplate,
maxvCpus: props.computeResources.maxvCpus || 256,
placementGroup: props.computeResources.placementGroup,
securityGroupIds: this.buildSecurityGroupIds(props.computeResources.vpc, props.computeResources.securityGroups),
securityGroupIds: this.getSecurityGroupIds(),
spotIamFleetRole: spotFleetRole?.roleArn,
subnets: props.computeResources.vpc.selectSubnets(props.computeResources.vpcSubnets).subnetIds,
tags: props.computeResources.computeResourcesTags,
Expand Down Expand Up @@ -576,14 +584,29 @@ export class ComputeEnvironment extends Resource implements IComputeEnvironment
return instanceTypes.map((type: ec2.InstanceType) => type.toString());
}

private buildSecurityGroupIds(vpc: ec2.IVpc, securityGroups?: ec2.ISecurityGroup[]): string[] | undefined {
private buildConnections(vpc?: ec2.IVpc, securityGroups?:ec2.ISecurityGroup[]): ec2.Connections {

if (vpc === undefined) {
return new ec2.Connections({});
}

if (securityGroups === undefined) {
return [
new ec2.SecurityGroup(this, 'Resource-Security-Group', { vpc }).securityGroupId,
];
return new ec2.Connections({
securityGroups: [
new ec2.SecurityGroup(this, 'Resource-Security-Group', { vpc }),
],
});
}

return new ec2.Connections({ securityGroups });
};

private getSecurityGroupIds(): string[] | undefined {
if (this.connections === undefined) {
return undefined;
}

return securityGroups.map((group: ec2.ISecurityGroup) => group.securityGroupId);
return this.connections.securityGroups.map((group: ec2.ISecurityGroup) => group.securityGroupId);
}

/**
Expand Down
2 changes: 2 additions & 0 deletions packages/@aws-cdk/aws-batch/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@
"@aws-cdk/aws-ec2": "0.0.0",
"@aws-cdk/aws-ecr": "0.0.0",
"@aws-cdk/aws-ecs": "0.0.0",
"@aws-cdk/aws-efs": "0.0.0",
"@aws-cdk/aws-iam": "0.0.0",
"@aws-cdk/aws-secretsmanager": "0.0.0",
"@aws-cdk/aws-ssm": "0.0.0",
Expand All @@ -104,6 +105,7 @@
"@aws-cdk/aws-ec2": "0.0.0",
"@aws-cdk/aws-ecr": "0.0.0",
"@aws-cdk/aws-ecs": "0.0.0",
"@aws-cdk/aws-efs": "0.0.0",
"@aws-cdk/aws-iam": "0.0.0",
"@aws-cdk/aws-secretsmanager": "0.0.0",
"@aws-cdk/aws-ssm": "0.0.0",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Loading

0 comments on commit 113baa3

Please sign in to comment.