Skip to content

Commit

Permalink
feat(eks): output update-kubeconfig command (#3669)
Browse files Browse the repository at this point in the history 
* feat(eks): output update-kubeconfig command

Synthesize a CloudFormation output that shows the `aws eks update-kubeconfig`
that needs to be executed in order to connect to the cluster. This command will
include the IAM masters role ARN if applicable.

Disable all other outputs by default, but added `outputXxx` options to enable.

Fixes #3664

BREAKING CHANGE: cluster name output will not be synthesized by default. instead we synthesize an output that includes the full `aws eks update-kubeconfig` command. You can enable synthesis of the cluster name output using the `outputClusterName: true` options.

* update expectations

* update readme
  • Loading branch information
@mergify
Elad Ben-Israel authored and mergify[bot] committed Aug 15, 2019
1 parent 15f01d0 commit 9e46532
Show file tree
Hide file tree
Showing 8 changed files with 268 additions and 47 deletions.
66 changes: 55 additions & 11 deletions packages/@aws-cdk/aws-eks/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,8 @@ new eks.Cluster(this, 'cluster', {
defaultCapacityInstance: new ec2.InstanceType('m2.xlarge')
});
```
To disable the default capacity, simply set `defaultCapacity` to `0`:

To disable the default capacity, simply set `defaultCapacity` to `0`:

```ts
new eks.Cluster(this, 'cluster-with-no-capacity', { defaultCapacity: 0 });
Expand Down Expand Up @@ -121,13 +121,56 @@ new eks.Cluster(this, 'Cluster', {
});
```

Now, given AWS credentials for a user that is trusted by the masters role, you
will be able to interact with your cluster like this:
When you `cdk deploy` this CDK app, you will notice that an output will be printed
with the `update-kubeconfig` command.

Something like this:

```
Outputs:
eks-integ-defaults.ClusterConfigCommand43AAE40F = aws eks update-kubeconfig --name cluster-ba7c166b-c4f3-421c-bf8a-6812e4036a33 --role-arn arn:aws:iam::112233445566:role/eks-integ-defaults-Role1ABCC5F0-1EFK2W5ZJD98Y
```

Copy & paste the "`aws eks update-kubeconfig ...`" command to your shell in
order to connect to your EKS cluster with the "masters" role.

Now, given [AWS CLI](https://aws.amazon.com/cli/) is configured to use AWS
credentials for a user that is trusted by the masters role, you should be able
to interact with your cluster through `kubectl` (the above example will trust
all users in the account).

For example:

```console
$ aws eks update-kubeconfig --name CLUSTER-NAME
$ aws eks update-kubeconfig --name cluster-ba7c166b-c4f3-421c-bf8a-6812e4036a33 --role-arn arn:aws:iam::112233445566:role/eks-integ-defaults-Role1ABCC5F0-1EFK2W5ZJD98Y
Added new context arn:aws:eks:eu-west-2:112233445566:cluster/cluster-ba7c166b-c4f3-421c-bf8a-6812e4036a33 to /Users/boom/.kube/config

$ kubectl get nodes # list all nodes
NAME STATUS ROLES AGE VERSION
ip-10-0-147-66.eu-west-2.compute.internal Ready <none> 21m v1.13.7-eks-c57ff8
ip-10-0-169-151.eu-west-2.compute.internal Ready <none> 21m v1.13.7-eks-c57ff8

$ kubectl get all -n kube-system
...
NAME READY STATUS RESTARTS AGE
pod/aws-node-fpmwv 1/1 Running 0 21m
pod/aws-node-m9htf 1/1 Running 0 21m
pod/coredns-5cb4fb54c7-q222j 1/1 Running 0 23m
pod/coredns-5cb4fb54c7-v9nxx 1/1 Running 0 23m
pod/kube-proxy-d4jrh 1/1 Running 0 21m
pod/kube-proxy-q7hh7 1/1 Running 0 21m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-dns ClusterIP 172.20.0.10 <none> 53/UDP,53/TCP 23m

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/aws-node 2 2 2 2 2 <none> 23m
daemonset.apps/kube-proxy 2 2 2 2 2 <none> 23m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 2/2 2 2 23m

NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-5cb4fb54c7 2 2 2 23m
```

For your convenience, an AWS CloudFormation output will automatically be
Expand Down Expand Up @@ -274,7 +317,7 @@ not have administrative privileges on the EKS cluster.
if you wish to be able to manually interact with your cluster, you will need
to map an IAM role or user to the `system:masters` group. This can be either
done by specifying a `mastersRole` when the cluster is defined, calling
`cluster.addMastersRole` or explicitly mapping an IAM role or IAM user to the
`cluster.awsAuth.addMastersRole` or explicitly mapping an IAM role or IAM user to the
relevant Kubernetes RBAC groups using `cluster.addRoleMapping` and/or
`cluster.addUserMapping`.

Expand All @@ -293,12 +336,13 @@ and a new cluster to be created.

When kubectl is disabled, you should be aware of the following:

1. When you log-in to your cluster, you don't need to specify `--role-arn` as long as you are using the same user that created
the cluster.
1. When you log-in to your cluster, you don't need to specify `--role-arn` as
long as you are using the same user that created the cluster.
2. As described in the Amazon EKS User Guide, you will need to manually
edit the [aws-auth ConfigMap](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) when you add capacity in order to map
the IAM instance role to RBAC to allow nodes to join the cluster.
3. Any `eks.Cluster` APIs that depend on programmatic kubectl support will fail with an error: `cluster.addResource`, `cluster.awsAuth`, `props.mastersRole`.
edit the [aws-auth ConfigMap](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html)
when you add capacity in order to map the IAM instance role to RBAC to allow nodes to join the cluster.
3. Any `eks.Cluster` APIs that depend on programmatic kubectl support will fail
with an error: `cluster.addResource`, `cluster.awsAuth`, `props.mastersRole`.

### Roadmap

Expand Down
53 changes: 47 additions & 6 deletions packages/@aws-cdk/aws-eks/lib/cluster.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,31 @@ export interface ClusterProps {
* @default m5.large
*/
readonly defaultCapacityInstance?: ec2.InstanceType;

/**
* Determines whether a CloudFormation output with the name of the cluster
* will be synthesized.
*
* @default false
*/
readonly outputClusterName?: boolean;

/**
* Determines whether a CloudFormation output with the ARN of the "masters"
* IAM role will be synthesized (if `mastersRole` is specified).
*
* @default false
*/
readonly outputMastersRoleArn?: boolean;

/**
* Determines whether a CloudFormation output with the `aws eks
* update-kubeconfig` command will be synthesized. This command will include
* the cluster name and, if applicable, the ARN of the masters IAM role.
*
* @default true
*/
readonly outputConfigCommand?: boolean;
}

/**
Expand Down Expand Up @@ -362,7 +387,11 @@ export class Cluster extends Resource implements ICluster {
this.clusterEndpoint = resource.attrEndpoint;
this.clusterCertificateAuthorityData = resource.attrCertificateAuthorityData;

new CfnOutput(this, 'ClusterName', { value: this.clusterName });
let configCommand = `aws eks update-kubeconfig --name ${this.clusterName}`;

if (props.outputClusterName) {
new CfnOutput(this, 'ClusterName', { value: this.clusterName });
}

// we maintain a single manifest custom resource handler per cluster since
// permissions and role are scoped. This will return `undefined` if kubectl
Expand All @@ -376,6 +405,12 @@ export class Cluster extends Resource implements ICluster {
}

this.awsAuth.addMastersRole(props.mastersRole);

if (props.outputMastersRoleArn) {
new CfnOutput(this, 'MastersRoleArn', { value: props.mastersRole.roleArn });
}

configCommand += ` --role-arn ${props.mastersRole.roleArn}`;
}

// allocate default capacity if non-zero (or default).
Expand All @@ -384,6 +419,11 @@ export class Cluster extends Resource implements ICluster {
const instanceType = props.defaultCapacityInstance || DEFAULT_CAPACITY_TYPE;
this.defaultCapacity = this.addCapacity('DefaultCapacity', { instanceType, desiredCapacity });
}

const outputConfigCommand = props.outputConfigCommand === undefined ? true : props.outputConfigCommand;
if (outputConfigCommand) {
new CfnOutput(this, 'ConfigCommand', { value: configCommand });
}
}

/**
Expand Down Expand Up @@ -456,11 +496,6 @@ export class Cluster extends Resource implements ICluster {
// EKS Required Tags
autoScalingGroup.node.applyAspect(new Tag(`kubernetes.io/cluster/${this.clusterName}`, 'owned', { applyToLaunchedInstances: true }));

// Create an CfnOutput for the Instance Role ARN (need to paste it into aws-auth-cm.yaml)
new CfnOutput(autoScalingGroup, 'InstanceRoleARN', {
value: autoScalingGroup.role.roleArn
});

if (options.mapRole === true && !this.kubectlEnabled) {
throw new Error(`Cannot map instance IAM role to RBAC if kubectl is disabled for the cluster`);
}
Expand All @@ -477,6 +512,12 @@ export class Cluster extends Resource implements ICluster {
'system:nodes'
]
});
} else {
// since we are not mapping the instance role to RBAC, synthesize an
// output so it can be pasted into `aws-auth-cm.yaml`
new CfnOutput(autoScalingGroup, 'InstanceRoleARN', {
value: autoScalingGroup.role.roleArn
});
}
}

Expand Down
7 changes: 6 additions & 1 deletion packages/@aws-cdk/aws-eks/package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,5 +88,10 @@
"engines": {
"node": ">= 8.10.0"
},
"awslint": {
"exclude": [
"props-no-arn-refs:@aws-cdk/aws-eks.ClusterProps.outputMastersRoleArn"
]
},
"stability": "experimental"
}
}
18 changes: 9 additions & 9 deletions packages/@aws-cdk/aws-eks/test/integ.eks-cluster.defaults.expected.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1192,16 +1192,16 @@
}
},
"Outputs": {
"ClusterClusterNameEB26049E": {
"ClusterConfigCommand43AAE40F": {
"Value": {
"Ref": "Cluster9EE0221C"
}
},
"ClusterDefaultCapacityInstanceRoleARN7DADF219": {
"Value": {
"Fn::GetAtt": [
"ClusterDefaultCapacityInstanceRole3E209969",
"Arn"
"Fn::Join": [
"",
[
"aws eks update-kubeconfig --name ",
{
"Ref": "Cluster9EE0221C"
}
]
]
}
}
Expand Down
12 changes: 10 additions & 2 deletions packages/@aws-cdk/aws-eks/test/integ.eks-cluster.kubectl-disabled.expected.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -912,9 +912,17 @@
}
},
"Outputs": {
"EKSClusterClusterName2B056109": {
"EKSClusterConfigCommand3809C9C9": {
"Value": {
"Ref": "EKSClusterBA6ECF8F"
"Fn::Join": [
"",
[
"aws eks update-kubeconfig --name ",
{
"Ref": "EKSClusterBA6ECF8F"
}
]
]
}
},
"EKSClusterNodesInstanceRoleARN10992C84": {
Expand Down
18 changes: 9 additions & 9 deletions packages/@aws-cdk/aws-eks/test/integ.eks-cluster.lit.expected.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1192,16 +1192,16 @@
}
},
"Outputs": {
"EKSClusterClusterName2B056109": {
"EKSClusterConfigCommand3809C9C9": {
"Value": {
"Ref": "EKSClusterE11008B6"
}
},
"EKSClusterNodesInstanceRoleARN10992C84": {
"Value": {
"Fn::GetAtt": [
"EKSClusterNodesInstanceRoleEE5595D6",
"Arn"
"Fn::Join": [
"",
[
"aws eks update-kubeconfig --name ",
{
"Ref": "EKSClusterE11008B6"
}
]
]
}
}
Expand Down
18 changes: 9 additions & 9 deletions packages/@aws-cdk/aws-eks/test/integ.eks-kubectl.lit.expected.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1118,16 +1118,16 @@
}
},
"Outputs": {
"cluster22ClusterNameC4896469": {
"cluster22ConfigCommand96B20279": {
"Value": {
"Ref": "cluster227BD1CB20"
}
},
"cluster22NodesInstanceRoleARNEE60B01D": {
"Value": {
"Fn::GetAtt": [
"cluster22NodesInstanceRole51CD052F",
"Arn"
"Fn::Join": [
"",
[
"aws eks update-kubeconfig --name ",
{
"Ref": "cluster227BD1CB20"
}
]
]
}
}
Expand Down
Loading

0 comments on commit 9e46532

Please sign in to comment.