(aws-eks): Only one type of update can be allowed with updateVersion

[ariksidney]

Describe the bug

I'm not really sure if this is a bug or if the error message is not really clear (at least to me).
In the latest version of our custom EKS product we're updating the authMode from configMap to api and configMap, however at the same time we would also like to update the control plane version to 1.31. This leads to the following error:

Only one type of update - VpcConfigUpdate, LoggingUpdate, EndpointAccessUpdate, or AuthModeUpdate can be allowed

However, we are only updating one type of the mentioned types (which is AuthModeUpdate), the other type we're updating is updateVersion.
According to my understanding, this should be possible as versionUpdate is not mentioned here?

aws-cdk/packages/@aws-cdk/custom-resource-handlers/lib/aws-eks/cluster-resource-handler/cluster.ts

Lines 131 to 137 in 523e0f0

	// We can only update one type of the UpdateTypes:
	type UpdateTypes = {
	updateLogging: boolean;
	updateAccess: boolean;
	updateVpc: boolean;
	updateAuthMode: boolean;
	};

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

If updating version and authMode at the same time should be possible, it shouldn't fail.
If this shouldn't be possible, versionUpdate should be in the list of updateTypes as well.

Current Behavior

Updating the following properties fails with

Error: Only one type of update - VpcConfigUpdate, LoggingUpdate, EndpointAccessUpdate, or AuthModeUpdate can be allowed,
            at Ti.onUpdate (/var/task/index.js:55:650864),
            at Ti.onEvent (/var/task/index.js:55:647516),
            at Runtime.dR [as handler] (/var/task/index.js:55:660630),
            at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1173:29)

"updates": {
        "replaceName": false,
        "updateVpc": false,
        "updateAccess": false,
        "replaceRole": false,
        "updateVersion": true,
        "updateEncryption": false,
        "updateLogging": false,
        "updateAuthMode": true,
        "updateBootstrapClusterCreatorAdminPermissions": false,
        "updateTags": false
    }

Reproduction Steps

Create a cluster with

Cluster(
            scope=self,
            id="Cluster",
            cluster_name="test",
            cluster_logging=[
                ClusterLoggingTypes.API,
                ClusterLoggingTypes.AUDIT,
                ClusterLoggingTypes.AUTHENTICATOR,
                ClusterLoggingTypes.CONTROLLER_MANAGER,
                ClusterLoggingTypes.SCHEDULER,
            ],
            version=KubernetesVersion.V1_29),
            vpc=vpc
) 

And update it with:

Cluster(
            scope=self,
            id="Cluster",
            cluster_name="test",
            cluster_logging=[
                ClusterLoggingTypes.API,
                ClusterLoggingTypes.AUDIT,
                ClusterLoggingTypes.AUTHENTICATOR,
                ClusterLoggingTypes.CONTROLLER_MANAGER,
                ClusterLoggingTypes.SCHEDULER,
            ],
            version=KubernetesVersion.V1_30),
            vpc=vpc,
            authentication_mode=AuthenticationMode.API_AND_CONFIG_MAP
) 

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.160.0

Framework Version

No response

Node.js Version

20.18.1

OS

Linux

Language

Python

Language Version

Python 3.10.16

Other information

No response

[pahud]

Yes, I can see the bug now. The issue is in how the code validates multiple updates:

type UpdateTypes = {
  updateLogging: boolean;
  updateAccess: boolean;
  updateVpc: boolean;
  updateAuthMode: boolean;
};

But when it checks for multiple updates, it's using Object.keys(updates) which gets ALL keys from the updates object, including:

• replaceName
• replaceRole
• updateVersion
• updateEncryption
• updateBootstrapClusterCreatorAdminPermissions
• updateTags

This causes the validation to incorrectly count ALL update types, not just the four that should be mutually exclusive (logging, access, vpc, auth mode).

The fix would be to explicitly check only the relevant update types:

const relevantUpdateTypes = ['updateLogging', 'updateAccess', 'updateVpc', 'updateAuthMode'];
const enabledUpdateTypes = relevantUpdateTypes.filter(type => updates[type as keyof UpdateMap]);

This way, version updates and other non-conflicting updates would not be included in the validation check.

Making this a p1 and we welcome PRs.

[ariksidney]

Thanks for confirming that this is indeed a bug. I can work on this as it looks like to be a pretty easy fix.

[pahud]

@ariksidney yes that would be awesome! I am requesting the team to confirm this but feel free to draft the PR if you like.

[ariksidney]

@pahud It looks like the eks update API does also not allow version and AuthenticationMode update at the same time. Can you confirm that?

I'm not sure if it would be good practice to wait for the version update in the cluster custom resource before continuing with the other upgrades? Or do you see a better way for implementing that?

[pahud]

@ariksidney Yes it's always a good practice to update version solo before update anything else.

Unfortunately I didn't see any eks API document that "version" and "AuthenticationMode" can't update at the same time but if you run that with SDK calls and see the error, it's a limitation there.

Have you tried to use CLI or SDK call to update the cluster for both version and AuthenticationMode? Did you see any error messages?

[ariksidney]

@pahud yes, I just tried it with the CLI commands (first trigger the asynchronous update-cluster-version followed by update-cluster-config). This resulted in an exception on the second call:

An error occurred (ResourceInUseException) when calling the UpdateClusterConfig operation: Cannot AccessConfigUpdate because cluster Cluster9EE0221C-03ee55a59dca41a9b1aff0726f70b824 currently has update 0585fedb-0fa2-3ad0-9916-344273c73c9a in progress

So I think the only way it could be achieved in the cdk would be to wait for the version update to complete in the custom cluster resource before continuing with the cluster config updates in the same custom resource. Not a good idea I guess?