[aws-eks] Support adding cdk8s charts to the Cluster

[iliapolo]

I'd like to be able to build a CDK EKS application that uses cdk8s for the manifest creation part.

Use Case

The eks.Cluster construct offers a way of applying Kubernetes manifests as part of the CDK application:

cluster.addResource('config', {
  kind: 'ConfigMap',
  apiVersion: 'v1',
  data: {
    key: 'value'
  },
  metadata: {
    name: 'app-config'
  }
});

This is great, as is it keeps all the moving parts in one place, not having to resort to external kubectl commands in order to deploy resources into the cluster.

However, this method doesn't offer any type assistance or special capabilities that relate to kubernetes objects, nor it should. It simply accepts a generic object of type any. To help create manifests, i'd like to utilize the cdk8s project, and combine it with my CDK application.

Since the cdk8s.Chart construct has a toJson method that essentially serializes the objects into a well formatted manifest, i want to do something like this::

import * as cdk from '@aws-cdk/core';
import * as eks from '@aws-cdk/aws-eks';
import * as cdk8s from 'cdk8s';
import * as k8s from '../imports/k8s';

export class EksPlusCdk8SStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here

    const cluster = new eks.Cluster(this, 'Cluster', {
      version: eks.KubernetesVersion.V1_17,
    });

    const kubeApp = new cdk8s.App();
    const kubeChart = new cdk8s.Chart(kubeApp, 'FrontEnd');

    new k8s.ConfigMap(kubeChart, 'Config', {
      data: {
        key: 'value'
      }
    });

    cluster.addResource('chart', kubeChart.toJson());

  }
}

Proposed Solution

Still don't have a concrete solution to this, but it seems mainly related to making aws-cdk and cdk8s work with a mutually compatible version of constructs.

Other

I already did some digging as to the root cause why its actually not working. The snippet i pasted above, produces the following error upon synth:

❯ cdk synth                                                                                                                                                                                                                                        [13:36:53]

/private/tmp/eks-plus-cdk8s/node_modules/cdk8s/src/_tokens.ts:3
const TOKEN_RESOLVER = new DefaultTokenResolver(new StringConcat());
                                                ^
TypeError: constructs_1.StringConcat is not a constructor
    at Object.<anonymous> (/private/tmp/eks-plus-cdk8s/node_modules/cdk8s/src/_tokens.ts:3:49)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
    at Module.require (internal/modules/cjs/loader.js:1026:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/private/tmp/eks-plus-cdk8s/node_modules/cdk8s/src/api-object.ts:4:1)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)

What happens is that chart.toJson will try to resolve any tokens in the object spec, and eventually call the following code from cdk8s:

import { Tokenization, DefaultTokenResolver, StringConcat, Construct } from 'constructs';

const TOKEN_RESOLVER = new DefaultTokenResolver(new StringConcat());

export function resolve<T>(scope: Construct, obj: T): T {
  return Tokenization.resolve(obj, {
    scope,
    resolver: TOKEN_RESOLVER,
  });
}

Notice that if refers to tokens classes (Tokenization, DefaultTokenResolver, StringConcat) from the constructs library. Now, the version of constructs thats being used is the one declared by the @aws-cdk/aws-eks dependency, which is version 3.0.3, and indeed, the token mechanism was removed from this version and these classes no longer exist there.

Its worth mentioning that cdk8s itself declares the correct dependency on constructs (for it), i.e 2.0.2, however, this version is not being installed because the constructs dependency is only declared as a peerDependency in cdk8s. This is intentional because there can only be one version of constructs in the build closure.

With a little npm trickery, I managed to eventually force every part of the code to use the correct version of constructs, and managed to successfully synth.

I then attempted to use a CDK token inside my cdk8s objects, and things broke down again. The following code:

    new k8s.ConfigMap(kubeChart, 'Config', {
      data: {
        key: cluster.clusterArn
      }
    });

produced this error:

❯ cdk synth                                                                                                                                                                                                                                        [14:38:26]

/private/tmp/eks-plus-cdk8s/node_modules/@aws-cdk/core/lib/stack.ts:169
      if (!c.node.scope) {
                  ^
TypeError: Resolution error: Cannot read property 'scope' of undefined.
    at _lookup (/private/tmp/eks-plus-cdk8s/node_modules/@aws-cdk/core/lib/stack.ts:169:19)
    at Function.of (/private/tmp/eks-plus-cdk8s/node_modules/@aws-cdk/core/lib/stack.ts:154:21)
    at Object.resolve (/private/tmp/eks-plus-cdk8s/node_modules/@aws-cdk/core/lib/resource.ts:160:38)
    at DefaultTokenResolver.resolveToken (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/resolvable.ts:134:24)
    at resolve (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/private/resolve.ts:133:29)
    at Object.resolve [as mapToken] (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/private/resolve.ts:49:32)
    at TokenizedStringFragments.mapTokens (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/string-fragments.ts:93:33)
    at DefaultTokenResolver.resolveString (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/resolvable.ts:155:22)
    at resolve (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/private/resolve.ts:91:31)
    at Object.resolve (/private/tmp/eks-plus-cdk8s/node_modules/constructs/lib/private/resolve.ts:49:32)

This happens because cdk8s is trying to resolve a CDK token in the context of a cdk8s construct. Resolving the cdk token involves doing Stack.of(scope), where scope is actually cdk8s.Chart. The Stack.of function eventually calls scope.node, but .node is not available in constructs version 2.0.2.

There is I hit the roadblock, there is no way to synthetically augment the construct of version 2.0.2 to include .node. And actually, even if there was, it would probably still not work since cdk8s.Chart is indeed not a cdk.Stack, so I imagine the following error would have been eventually thrown:

if (!c.node.scope) {
    throw new Error(`No stack could be identified for the construct at path ${construct.node.path}`);
}

Not sure what the correct way forward here, this requires some additional investigation.

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.