[aws-eks] masterRole of aws-auth is not set username cause metrics-server not working correclty

[a60814billy]

I deployed new EKS cluster with mastersRole, and then I connected to k8s cluster by assume masterRole, It worked perfectly fine.

Next I'm following this document to install metrics-server on my cluster. Then using kubectl top node to verify metrics-server is working correctly or not.

Unfortunately, I encountered a server error:

$ kubectl top node
error: You must be logged in to the server (Unauthorized)

In order to debug the error, I use kubectl -n kube-system logs -l k8s-app=metrics-server to print logs, the log of metrics-server is shown as below:

I1130 19:40:37.230865       1 serving.go:312] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
I1130 19:40:37.582772       1 secure_serving.go:116] Serving securely on [::]:443
E1130 19:41:15.309837       1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority
E1130 19:45:23.565685       1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority

After that, I created a support ticket to AWS support, and he pointed out that the aws-auth isn't set correctly, the role in mapRoles should contain username. According to AWS documentation, the default value of username should be roleArn.

To solve this problem, a workaround is adding master role manually with username:

myCluster.awsAuth.addMastersRole(myAdminRole, 'admin');

or modify the addMastersRole function in AwsAuth class, adding default username when username is not set.

aws-cdk/packages/@aws-cdk/aws-eks/lib/aws-auth.ts

Lines 59 to 64 in 1dcba75

	public addMastersRole(role: iam.IRole, username?: string) {
	this.addRoleMapping(role, {
	username,
	groups: [ 'system:masters' ]
	});
	}

Because the constructor of Cluster is not sent username to addMasterRole function

aws-cdk/packages/@aws-cdk/aws-eks/lib/cluster.ts

Line 412 in 1dcba75

this.awsAuth.addMastersRole(props.mastersRole);

Reproduction Steps

1. Stack Source Code

import * as core from '@aws-cdk/core';
import * as eks from '@aws-cdk/aws-eks';
import * as iam from '@aws-cdk/aws-iam';

export class EksTest extends core.Stack {
    constructor(scope: core.Construct, id: string, props: core.StackProps) {
        super(scope, id, props);
        const clusterAdmin = new iam.Role(this, 'EKS-Admin-Role', {
            assumedBy: new iam.AccountRootPrincipal(),
        });

        const myEksCluster = new eks.Cluster(this, 'eks-test', {
            mastersRole: clusterAdmin,
        })
    }
}

2. setup kubeconfig by cdk output

EKS-Stack.ekstestConfigCommand78CC8955 = aws eks update-kubeconfig --name cluster-33eb8e2e-7653-4d3b-a779-05c176085f77 --region ap-northeast-1 --role-arn arn:aws:iam::xxxxxxxxxxxx:role/EKS-Stack-EKSAdminRole08B22A7B-1HN27Q53HJVU2

3. setup metrics-server (https://docs.aws.amazon.com/eks/latest/userguide/metrics-server.html)

$ DOWNLOAD_URL=$(curl --silent "https://api.github.com/repos/kubernetes-sigs/metrics-server/releases/latest" | jq -r .tarball_url)
$ DOWNLOAD_VERSION=$(grep -o '[^/v]*$' <<< $DOWNLOAD_URL)
$ curl -Ls $DOWNLOAD_URL -o metrics-server-$DOWNLOAD_VERSION.tar.gz
$ mkdir metrics-server-$DOWNLOAD_VERSION
$ tar -xzf metrics-server-$DOWNLOAD_VERSION.tar.gz --directory metrics-server-$DOWNLOAD_VERSION --strip-components 1
$ kubectl apply -f metrics-server-$DOWNLOAD_VERSION/deploy/1.8+/

3. wait for metrics-server deployed, and use kubectl top node to reproduce this problem

$ kubectl top node

Error Log

1. Error log of kubectl top node

error: You must be logged in to the server (Unauthorized)

2. Error log of metrics-server

I1130 19:40:37.230865       1 serving.go:312] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
I1130 19:40:37.582772       1 secure_serving.go:116] Serving securely on [::]:443
E1130 19:41:15.309837       1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority
E1130 19:45:23.565685       1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority

Environment

• CLI Version : 1.8.0
• Framework Version: 1.8.0
• OS : macOS 10.13.6
• Language : typescript

Other

Maybe I can contribute the fix I said above, to add default value of username in addMastersRole function

---

This is 🐛 Bug Report

[eladb]

Thanks for the awesome bug report!