[aws-eks] reference a value returned through "kubectl get"

[zxkane]

Provide a capability to reference values returned through kubectl get, then the value could be used by other parts of application later.

Example:

const hostname = new KubernetesGet(this, 'HostName', {
  kind: 'ingress',
  query: '$.items[0].status.loadBalancer.ingress.[0].hostname',
  wait: true // retry until the value becomes available
});

// then you can just reference the value like this:
hostname.valueAsString

Use Case

When orchestrating an application, the EKS cluster and resources created by k8s resource/helm chart are parts of the entire application.

For example, we deploy a helm chart with internal NLB/ALB ingress controller of a service. And the service exposed by EKS just is part of entire application(other services are be provided by ECS or EC2 auto scaling group).

We need the resource arn created by NLB/ALB ingress controller for entire application orchestration.

Proposed Solution

Other

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[eladb]

We would need a more concrete example. Can you share some code?

[zxkane]

There is an example to deploy Sonatype Nexus OSS on EKS.

I need the arn of the ALB created in the EKS in that stack if putting CloudFront in front of the alb.

[eladb]

There is an example to deploy Sonatype Nexus OSS on EKS.

I need the arn of the ALB created in the EKS in that stack if putting CloudFront in front of the alb.

Can you please be more specific? Can you point me to the line of code where this is created and where you would expect to be able to obtain the ARN?

[zxkane]

I'm using cluster.addChart to deploy the Helm chart sonatype-nexus, which deploys Nexus OSS with ingress using ALB. So an ALB will be created after the chart is deployed successfully.

It would be useful to add optional query parameters for addChart and addResource, the query parameters indicating the types of resource, names of resource, output format of kubectl get. And the output of kubectl get is treated as output of custom resource.

Also it would be more flexible if adding new method cluster.getResources via kubectl get. User can use the dependency of CFN resources to get the runtime value of K8S resources.

For example, I would like to get hostname of ingress after deploying it,

kubectl get ingress -n default -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig":
        { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
      alb.ingress.kubernetes.io/auth-type: none
      alb.ingress.kubernetes.io/backend-protocol: HTTP
      alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:ap-southeast-1:account-id:certificate/5dfad8d6-bec9-4353-9ce5-ed4723e56607
      alb.ingress.kubernetes.io/healthcheck-path: /
      alb.ingress.kubernetes.io/healthcheck-port: "8081"
      alb.ingress.kubernetes.io/inbound-cidrs: 0.0.0.0/0
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
      alb.ingress.kubernetes.io/tags: app=nexus3
      alb.ingress.kubernetes.io/target-type: ip
      kubernetes.io/ingress.class: alb
    creationTimestamp: "2020-06-08T13:59:11Z"
    generation: 5
    labels:
      app: sonatype-nexus
      chart: sonatype-nexus-2.1.0
      fullname: nexus3-sonatype-nexus
      heritage: Helm
      release: nexus3
    name: nexus3-sonatype-nexus
    namespace: default
    resourceVersion: "17144"
    selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/nexus3-sonatype-nexus
    uid: 67e53bf6-082f-4d3a-adf0-291ab3e5bc23
  spec:
    rules:
    - host: nexus.mydomain.com
      http:
        paths:
        - backend:
            serviceName: ssl-redirect
            servicePort: use-annotation
          path: /*
        - backend:
            serviceName: nexus3-sonatype-nexus
            servicePort: 8081
          path: /*
  status:
    loadBalancer:
      ingress:
      - hostname: alb-id.ap-southeast-1.elb.amazonaws.com
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

[eladb]

Okay, I think I understand the use case. Basically what you are looking for is a way to reference values returned from a kubectl get in your CDK app.

I think the main problem with this, in k8s, is that most of the apply operations are asynchronous. This means that you would need to "wait" for the resource to stabilize before we can issue the query, but I guess this query operation can also have some sort of retry support.

So perhaps something like this:

const hostname = new KubernetesGet(this, 'HostName', {
  kind: 'ingress',
  query: '$.items[0].status.loadBalancer.ingress.[0].hostname'
});

// then you can just reference the value like this:
hostname.valueAsString

[zxkane]

Okay, I think I understand the use case. Basically what you are looking for is a way to reference values returned from a kubectl get in your CDK app.

I think the main problem with this, in k8s, is that most of the apply operations are asynchronous. This means that you would need to "wait" for the resource to stabilize before we can issue the query, but I guess this query operation can also have some sort of retry support.

So perhaps something like this:

const hostname = new KubernetesGet(this, 'HostName', {
  kind: 'ingress',
  query: '$.items[0].status.loadBalancer.ingress.[0].hostname'
});

// then you can just reference the value like this:
hostname.valueAsString

It's exactly what this feature requests. It could be a feature to verify the status of resources deployed via CDK.