Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: construct library for ECS #1041

Closed
wants to merge 113 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
939452c
Ignore swp files
SoManyHs Aug 23, 2018
7302a2f
Add ECS lib and example
SoManyHs Aug 23, 2018
4c7127a
Add service and task def constructs
SoManyHs Aug 24, 2018
98749ec
WIP adding VPC/ASG
SoManyHs Aug 24, 2018
6d0942f
Refactor ECS demo
Aug 24, 2018
bddfe56
Add more props on Service
SoManyHs Aug 24, 2018
3574611
Add TODO for AMI IDs
SoManyHs Sep 6, 2018
a63c377
Weird hack?
SoManyHs Sep 7, 2018
b00e0dd
Update example to instantiate service separately
SoManyHs Sep 7, 2018
80ba2e4
WIP Add Container Definitions
SoManyHs Sep 12, 2018
0980f52
WIP Add other Task Def properties
SoManyHs Sep 14, 2018
8648540
Merge remote-tracking branch 'official/master' into ecs-demo
Sep 28, 2018
2d53d89
Indentation 4 -> 2
Sep 28, 2018
dbdab86
Fix type errors
Sep 28, 2018
ee6b427
Use SSM parameter for ECS Optimized AMI
SoManyHs Sep 28, 2018
2b16dad
Move containerdefinition to its own file
Sep 28, 2018
16a92c0
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Sep 28, 2018
ae881c2
Turning ContainerDefinition into a construct
Sep 28, 2018
0672e6a
Add enum for placement constraints
SoManyHs Oct 1, 2018
4f93a5b
Rename fleet to autoScalingGroup
SoManyHs Oct 1, 2018
9b61b89
Qualify "PlacementConstraintType" usage with package name
Oct 1, 2018
98bf81b
ECR can return itself as container image
Oct 1, 2018
ae1d525
Automatically generate execution and task roles
Oct 1, 2018
23072a9
Make taskRole public
Oct 1, 2018
e5cc7f2
Remove shellCommand from healthCheck
SoManyHs Oct 1, 2018
047eed8
Rename toContainerDefinitionJson to renderContainerDefinition
SoManyHs Oct 1, 2018
65d3f79
Some work on Service
Oct 1, 2018
11541d3
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 1, 2018
fb17532
Separate Fargate and ECS clusters
SoManyHs Oct 1, 2018
be8cb86
Separate ECS and Fargate Service
SoManyHs Oct 1, 2018
b8cce3a
Separate ECS/Fargate TaskDefinition
SoManyHs Oct 1, 2018
2741608
Fix lint errors
SoManyHs Oct 1, 2018
a901912
Fix lint errors
SoManyHs Oct 1, 2018
a91bda7
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 2, 2018
51adff5
Fix bugs in service and task def
SoManyHs Oct 2, 2018
70ae682
Fix AMI ID getter for ECS clusters
SoManyHs Oct 2, 2018
298e0a7
Add Capabilities in Linux Parameters on ContainerDefs
SoManyHs Oct 2, 2018
1541edd
Add Capabilities in Linux Parameters on ContainerDefs
SoManyHs Oct 2, 2018
a154d09
Add default arg to SSMProvider.getString
SoManyHs Oct 2, 2018
4b46685
Add Devices to LinuxParameters
SoManyHs Oct 2, 2018
9875c6e
Add Tmpfs to LinuxParameters
SoManyHs Oct 2, 2018
1044d1c
Update ecs demo with full Linux Parameters example
SoManyHs Oct 2, 2018
e7f8000
Rename renderLogDriver
SoManyHs Oct 2, 2018
e7f7b7d
Do most of Service
Oct 2, 2018
dda869d
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 2, 2018
97083dc
Add Ulimits
SoManyHs Oct 2, 2018
a3e3615
Fix log driver
SoManyHs Oct 2, 2018
b913f0e
Make add* functions consistently plural
SoManyHs Oct 2, 2018
fd21c0d
Make load balancers respect network mode
Oct 2, 2018
3b4695c
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 2, 2018
187f62b
Make add* functions consistently plural
SoManyHs Oct 2, 2018
6912e3f
Add portMappings to ContainerDefinition
SoManyHs Oct 2, 2018
2bf2d57
Missing semicolons
SoManyHs Oct 2, 2018
f477d8d
Add MountPoints to ContainerDefinition
SoManyHs Oct 2, 2018
b947f51
Change 'addContainer' to take the props and return ContainerDefinition.
Oct 2, 2018
ab83d35
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 2, 2018
30cb925
Add VolumesFrom to ContainerDefinition
SoManyHs Oct 2, 2018
c64e184
Add fargate example for demo purposes
SoManyHs Oct 3, 2018
566aac3
Remove Service Role, fix DesiredCount default, make task memory defin…
Oct 3, 2018
96e3105
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 3, 2018
48cd8d4
Starting on port mapping defaults
Oct 3, 2018
e311293
Add validation on port mappings with tests
SoManyHs Oct 3, 2018
4ffed09
Implement ingressPort method
SoManyHs Oct 3, 2018
97d3db3
Make ALB listener ACTUALLY default to 'true'
Oct 4, 2018
4f9b548
Add dependency on listener
SoManyHs Oct 4, 2018
e8aa186
Integ test for Fargate service
SoManyHs Oct 4, 2018
eb4a562
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 4, 2018
e06a3ec
Add VolumesFrom to ContainerDefinition
SoManyHs Oct 2, 2018
95b8d84
Add fargate example for demo purposes
SoManyHs Oct 3, 2018
5595707
Starting on port mapping defaults
Oct 3, 2018
a1b78f4
Make ALB listener ACTUALLY default to 'true'
Oct 4, 2018
6f08406
Add validation on port mappings with tests
SoManyHs Oct 3, 2018
5312362
Implement ingressPort method
SoManyHs Oct 3, 2018
6f42026
Add dependency on listener
SoManyHs Oct 4, 2018
1983858
Integ test for Fargate service
SoManyHs Oct 4, 2018
4ea6b8b
hello-cdk-fargate example works
SoManyHs Oct 4, 2018
e34d515
Add lazy evaluation of network configuration property
SoManyHs Oct 4, 2018
bf90c27
Add LoadBalancedFargateService L3 construct
SoManyHs Oct 4, 2018
ffbad9f
Start LoadBalancedFargateServiceApplet
SoManyHs Oct 4, 2018
d1b22bd
Application AutoScaling WIP
Oct 4, 2018
7840634
Merge branch 'ecs-demo' of github.com:SoManyHs/aws-cdk into ecs-demo
Oct 4, 2018
d1ec640
Make declarative example work
Oct 4, 2018
36e1ba2
Add import exports to Cluster
SoManyHs Oct 5, 2018
178898f
Fix portMappings/ingressPort
SoManyHs Oct 5, 2018
b73bdad
Clean up fargate integ test
SoManyHs Oct 5, 2018
049557e
Add integ test for load balanced service in bridge mode
SoManyHs Oct 5, 2018
2295ef2
Add integ test for awsvpc network mode on EcsCluster
SoManyHs Oct 5, 2018
f34c23c
Fix ingress/containerPort on LBs
SoManyHs Oct 5, 2018
c491a1d
Cleanup example
SoManyHs Oct 5, 2018
430f81e
Add cloudwatch metrics
SoManyHs Oct 5, 2018
dca6972
Bridge mode works with host port specified
SoManyHs Oct 5, 2018
cf528d6
Merge remote-tracking branch 'origin/master' into ecs-demo
Oct 12, 2018
ef51786
Fix libs to be in line with latest API changes from master
Oct 12, 2018
c3b543a
Merge remote-tracking branch 'origin/master' into ecs-demo
Oct 15, 2018
64b6d06
Fix build errors
Oct 17, 2018
af5a2a4
Merge remote-tracking branch 'origin/master' into ecs-demo
Oct 17, 2018
51097fd
Fix ingressPort/containerPort tests
SoManyHs Oct 22, 2018
7b19dd2
Add LoadBalancedEcsService L3 Construct
SoManyHs Oct 22, 2018
9bea6be
Add ability to specify cluster size
SoManyHs Oct 24, 2018
4ab455c
Merge remote-tracking branch 'origin/master' into ecs-demo
Oct 26, 2018
be9f447
feat(aws-ecs): add Task AutoScaling to Service
Oct 26, 2018
2eab513
Add unit tests for EcsCluster
SoManyHs Oct 26, 2018
57857a9
Merge branch 'master' into ecs-demo
SoManyHs Oct 29, 2018
4cf23fc
Unit tests for Cluster
SoManyHs Oct 29, 2018
37d3c4e
Set volumes on TaskDefinition
SoManyHs Oct 30, 2018
1ad3669
Clarify doc string for memoryLimit and memoryReservation
SoManyHs Oct 30, 2018
78d2a49
WIP Unit tests on EcsTaskDefinition
SoManyHs Oct 30, 2018
ca61a74
WIP Unit tests for EcsService
SoManyHs Oct 30, 2018
98345b3
WIP Unit tests for FargateService
SoManyHs Oct 30, 2018
35ebd9c
WIP unit tests Fargate Task Definition
SoManyHs Oct 30, 2018
09a182a
Merge branch 'huijbers/autoscaling' into ecs-demo
Oct 30, 2018
2fd2e7c
Add docs and readme and a couple of tests
Oct 30, 2018
9825127
Unit test for error case on service creation
SoManyHs Oct 30, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ pack
coverage
.nyc_output
.LAST_BUILD
*.swp
./examples/cdk-examples-typescript/hello-cdk-ecs/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"app": "../node_modules/.bin/cdk-applet-js fargate-service.yml"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do I get this magical cdk-applet-js file, because it is not appearing for me locally even after a build

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# applet is loaded from the local ./test-applet.js file
applet: @aws-cdk/aws-ecs:LoadBalancedFargateServiceApplet
image: 'amazon/amazon-ecs-sample'
cpu: "2048"
memoryMiB: "1024"
42 changes: 42 additions & 0 deletions examples/cdk-examples-typescript/hello-cdk-ecs/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"app": "node index",
"context": {
"availability-zones:585695036304:us-east-1": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it normal practice in the CDK to check in account IDs? Normally we avoid publicly sharing account IDs...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. We should definitely delete

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this should not have been checked in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect customers to check this in to their repos, or should cdk.json generally be git'ignored? Should cdk init add that to a .gitignore file? Sounds like we need a separate issue to track that

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi this stuff is online for all to see

"us-east-1a",
"us-east-1b",
"us-east-1c",
"us-east-1d",
"us-east-1e",
"us-east-1f"
],
"ssm:585695036304:us-east-1:/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2": "ami-14c5486b",
"availability-zones:585695036304:eu-west-2": [
"eu-west-2a",
"eu-west-2b",
"eu-west-2c"
],
"ssm:585695036304:eu-west-2:/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2": "ami-a36f8dc4",
"availability-zones:585695036304:eu-west-1": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
],
"ssm:585695036304:eu-west-1:/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2": "ami-ca0135b3",
"availability-zones:794715269151:us-west-2": [
"us-west-2a",
"us-west-2b",
"us-west-2c"
],
"availability-zones:993655754359:us-west-2": [
"us-west-2a",
"us-west-2b",
"us-west-2c"
],
"availability-zones:993655754359:eu-west-1": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
],
"ssm:794715269151:us-west-2:/aws/service/ecs/optimized-ami/amazon-linux/recommended": "{\"schema_version\":1,\"image_name\":\"amzn-ami-2018.03.g-amazon-ecs-optimized\",\"image_id\":\"ami-00430184c7bb49914\",\"os\":\"Amazon Linux\",\"ecs_runtime_version\":\"Docker version 18.06.1-ce\",\"ecs_agent_version\":\"1.20.3\"}"
}
}
113 changes: 113 additions & 0 deletions examples/cdk-examples-typescript/hello-cdk-ecs/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
import ec2 = require('@aws-cdk/aws-ec2');
import { InstanceType } from '@aws-cdk/aws-ec2';
import ecs = require('@aws-cdk/aws-ecs');
import cdk = require('@aws-cdk/cdk');

class BonjourECS extends cdk.Stack {
constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
super(parent, name, props);

// For better iteration speed, it might make sense to put this VPC into
// a separate stack and import it here. We then have two stacks to
// deploy, but VPC creation is slow so we'll only have to do that once
// and can iterate quickly on consuming stacks. Not doing that for now.
const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
const cluster = new ecs.EcsCluster(this, 'EcsCluster', {
vpc,
size: 3,
instanceType: new InstanceType("t2.xlarge")
});

// Instantiate ECS Service with just cluster and image
const ecsService = new ecs.LoadBalancedEcsService(this, "EcsService", {
cluster,
memoryLimitMiB: 512,
image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
});

// Output the DNS where you can access your service
new cdk.Output(this, 'LoadBalancerDNS', { value: ecsService.loadBalancer.dnsName });
}
}

const app = new cdk.App();

new BonjourECS(app, 'Bonjour');

app.run();

// name, image, cpu, memory, port (with default)
//
// Include in constructs:
// - networking - include SD, ALB
// - logging - cloudwatch logs integration? talk to nathan about 3rd
// party integrations - aggregated logging across the service
// (instead of per task). Probably prometheus or elk?
// - tracing aws-xray-fargate - CNCF opentracing standard - jaeger,
// zipkin.
// - so x-ray is a container that is hooked up to sidecars that come
// with the application container itself
// - autoscaling - application autoscaling (Fargate focused?)

// const taskDefinition = new ecs.EcsTaskDefinition(this, "EcsTD", {
// family: "ecs-task-definition",
// });

// const container = taskDefinition.addContainer('web', {
// image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
// cpu: 1024,
// memoryLimitMiB: 512,
// essential: true
// });

// container.linuxParameters.addCapabilities(ecs.Capability.All);
// container.linuxParameters.dropCapabilities(ecs.Capability.Chown);

// container.linuxParameters.addDevices({
// containerPath: "/dev/pudding",
// hostPath: "/dev/clyde",
// permissions: [ecs.DevicePermission.Read]
// });

// container.linuxParameters.addTmpfs({
// containerPath: "/dev/sda",
// size: 12345,
// mountOptions: [ecs.TmpfsMountOption.Ro]
// });

// container.linuxParameters.sharedMemorySize = 65535;
// container.linuxParameters.initProcessEnabled = true;

// container.addUlimits({
// name: ecs.UlimitName.Core,
// softLimit: 1234,
// hardLimit: 1234,
// });

// container.addPortMappings({
// containerPort: 80,
// // hostPort: 80,
// protocol: ecs.Protocol.Tcp,
// });

// container.addMountPoints({
// containerPath: '/tmp/cache',
// sourceVolume: 'volume-1',
// readOnly: true,
// }, {
// containerPath: './cache',
// sourceVolume: 'volume-2',
// readOnly: true,
// });

// container.addVolumesFrom({
// sourceContainer: 'web',
// readOnly: true,
// });

// new ecs.EcsService(this, "EcsService", {
// cluster,
// taskDefinition,
// desiredCount: 1,
// });
// }
47 changes: 47 additions & 0 deletions examples/cdk-examples-typescript/hello-cdk-fargate/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"app": "node index",
"context": {
"availability-zones:585695036304:us-east-1": [
"us-east-1a",
"us-east-1b",
"us-east-1c",
"us-east-1d",
"us-east-1e",
"us-east-1f"
],
"ssm:585695036304:us-east-1:/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2": "ami-14c5486b",
"availability-zones:585695036304:eu-west-2": [
"eu-west-2a",
"eu-west-2b",
"eu-west-2c"
],
"ssm:585695036304:eu-west-2:/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2": "ami-a36f8dc4",
"availability-zones:585695036304:eu-west-1": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
],
"ssm:585695036304:eu-west-1:/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2": "ami-ca0135b3",
"availability-zones:794715269151:us-west-2": [
"us-west-2a",
"us-west-2b",
"us-west-2c"
],
"availability-zones:993655754359:us-west-2": [
"us-west-2a",
"us-west-2b",
"us-west-2c"
],
"availability-zones:993655754359:eu-west-1": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
],
"ssm:794715269151:us-west-2:/aws/service/ecs/optimized-ami/amazon-linux/recommended": "{\"schema_version\":1,\"image_name\":\"amzn-ami-2018.03.g-amazon-ecs-optimized\",\"image_id\":\"ami-00430184c7bb49914\",\"os\":\"Amazon Linux\",\"ecs_runtime_version\":\"Docker version 18.06.1-ce\",\"ecs_agent_version\":\"1.20.3\"}",
"availability-zones:794715269151:eu-west-1": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
]
}
}
29 changes: 29 additions & 0 deletions examples/cdk-examples-typescript/hello-cdk-fargate/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import ec2 = require('@aws-cdk/aws-ec2');
import ecs = require('@aws-cdk/aws-ecs');
import cdk = require('@aws-cdk/cdk');

class BonjourFargate extends cdk.Stack {
constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
super(parent, name, props);

// Create VPC and Fargate Cluster
// NOTE: Limit AZs to avoid reaching resource quotas
const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
const cluster = new ecs.FargateCluster(this, 'Cluster', { vpc });

// Instantiate Fargate Service with just cluster and image
const fargateService = new ecs.LoadBalancedFargateService(this, "FargateService", {
cluster,
image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
});

// Output the DNS where you can access your service
new cdk.Output(this, 'LoadBalancerDNS', { value: fargateService.loadBalancer.dnsName });
}
}

const app = new cdk.App();

new BonjourFargate(app, 'Bonjour');

app.run();
2 changes: 2 additions & 0 deletions examples/cdk-examples-typescript/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@
"@aws-cdk/aws-cognito": "^0.14.1",
"@aws-cdk/aws-dynamodb": "^0.14.1",
"@aws-cdk/aws-ec2": "^0.14.1",
"@aws-cdk/aws-ecs": "^0.14.1",
"@aws-cdk/aws-elasticloadbalancing": "^0.14.1",
"@aws-cdk/aws-elasticloadbalancingv2": "^0.14.1",
"@aws-cdk/aws-iam": "^0.14.1",
"@aws-cdk/aws-lambda": "^0.14.1",
"@aws-cdk/aws-neptune": "^0.14.1",
Expand Down
21 changes: 21 additions & 0 deletions packages/@aws-cdk/aws-ecr/lib/repository-ref.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import ecs = require('@aws-cdk/aws-ecs');
import iam = require('@aws-cdk/aws-iam');
import cdk = require('@aws-cdk/cdk');

Expand Down Expand Up @@ -44,6 +45,13 @@ export abstract class RepositoryRef extends cdk.Construct {
const parts = cdk.ArnUtils.parse(this.repositoryArn);
return `${parts.account}.dkr.ecr.${parts.region}.amazonaws.com/${parts.resourceName}`;
}

/**
* Refer to a particular image tag from this repository
*/
public getImage(tag: string = "latest"): ecs.ContainerImage {
return new EcrImage(this, tag);
}
}

export interface RepositoryRefProps {
Expand All @@ -67,3 +75,16 @@ class ImportedRepository extends RepositoryRef {
// FIXME: Add annotation about policy we dropped on the floor
}
}

class EcrImage extends ecs.ContainerImage {
public readonly imageName: string;

constructor(repository: RepositoryRef, tag: string) {
super();
this.imageName = `${repository.repositoryUri}:${tag}`;
}

public bind(containerDefinition: ecs.ContainerDefinition): void {
containerDefinition.useEcrImage();
}
}
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-ecr/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
},
"dependencies": {
"@aws-cdk/aws-iam": "^0.14.1",
"@aws-cdk/aws-ecs": "^0.14.1",
"@aws-cdk/cdk": "^0.14.1"
},
"homepage": "https://github.com/awslabs/aws-cdk"
Expand Down
Loading