Admission Controller Doesn't display feedback for kubectl "create" and "apply"

[amirbenv]

• terrascan version: 1.5.1
• Operating System: MacOS 11

Description

When using kubectl to create or apply, admission controller responses are not being displayed to the user.
If applicable, the logs endpoint shows the results. Enforcement also works.

What I Did

kubectl create -f vulnerable_pod.yaml

[amirbenv]

@alexfeig

[alexfeig]

Thanks @amirbenv - if you need a sample, here you go:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: green
  namespace: default
spec:
  selector:
    matchLabels:
      app: green
  replicas: 2
  template:
    metadata:
      labels:
        app: green
    spec:
      containers:
      - name: green
        image: alexfeig/bluegreen:latest
        ports:
        - containerPort: 5000
        env:
        - name: app_color
          value: "green"

[devang-gaur]

@alexfeig can you share the webhook yaml file ?

[devang-gaur]

webhooks:

• name: terrascan-validating-hook.terrascan.svc
  rules:
  • apiGroups:
    • ""
      apiVersions:
      • v1
        operations:
      • CREATE
      • UPDATE
        resources:
      • pods
      • services

Did you try tinkering with these rule properties? maybe add deployment, secret, configmap, replicaset, replicacontroller in webhooks.rules[0].resources list ?

[alexfeig]

Sure, here you go. One thing is that using kubectl run works just fine, so I don't think it's the ValidatingWebhookConfiguration

  apiVersion: admissionregistration.k8s.io/v1
  kind: ValidatingWebhookConfiguration
  metadata:
    name: terrascan.hook.all
  webhooks:
    - name: terrascan.hook.all
      rules:
        - apiGroups:
            - ""
          apiVersions:
            - v1
          operations:
            - CREATE
            - UPDATE
          resources:
            - "*/*"
      failurePolicy: Fail
      clientConfig:
        url: https://192.168.49.1:9443/v1/k8s/webhooks/12345/scan/validate
        caBundle: xxxx
      sideEffects: None
      admissionReviewVersions: ["v1"]

[alexfeig]

@dev-gaur */* covers everything. Note that the admission controller actually does block creation, so the webhook works. The reporting does not.

> kubectl apply -f kubernetes/deployment.yaml
deployment.apps/green created
> kubectl get deployment
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
green   0/2     0            0           12s

[devang-gaur]

Thanks, looking into it. @alexfeig

[devang-gaur]

@alexfeig Kind Deployment is not a part of the default api group.

╰─$ kubectl api-resources --api-group=""                                                                                                                                                                                                                  1 ↵
NAME                     SHORTNAMES   APIGROUP   NAMESPACED   KIND
bindings                                         true         Binding
componentstatuses        cs                      false        ComponentStatus
configmaps               cm                      true         ConfigMap
endpoints                ep                      true         Endpoints
events                   ev                      true         Event
limitranges              limits                  true         LimitRange
namespaces               ns                      false        Namespace
nodes                    no                      false        Node
persistentvolumeclaims   pvc                     true         PersistentVolumeClaim
persistentvolumes        pv                      false        PersistentVolume
pods                     po                      true         Pod
podtemplates                                     true         PodTemplate
replicationcontrollers   rc                      true         ReplicationController
resourcequotas           quota                   true         ResourceQuota
secrets                                          true         Secret
serviceaccounts          sa                      true         ServiceAccount
services                 svc                     true         Service

[devang-gaur]

its part of the apps api group. use kubectl api-resources for reference.

[alexfeig]

That was my bad actually, surprised I didn't catch that "" didn't work for apiGroups. I fixed it by adding * for all. Whoops.

This does work:

  apiVersion: admissionregistration.k8s.io/v1
  kind: ValidatingWebhookConfiguration
  metadata:
    name: terrascan.hook.all
  webhooks:
    - name: terrascan.hook.all
      rules:
        - apiGroups:
            - "*"
          apiVersions:
            - v1
          operations:
            - CREATE
            - UPDATE
          resources:
            - "*/*"
      failurePolicy: Fail
      clientConfig:
        url: https://192.168.49.1:9443/v1/k8s/webhooks/12345/scan/validate
        caBundle: xxxx
      sideEffects: None
      admissionReviewVersions: ["v1"]

> kubectl apply -f kubernetes/deployment.yaml
Error from server: error when creating "kubernetes/deployment.yaml": admission webhook "terrascan.hook.all" denied the request:
{"rule_name":"MemoryRequestsCheck","description":"Memory Request Not Set in config file.","rule_id":"AC-K8-OE-PK-M-0157","severity":"Medium","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"runAsNonRootCheck","description":"Minimize Admission of Root Containers","rule_id":"AC-K8-IA-PO-H-0168","severity":"HIGH","category":"Identity and Access Management","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"readOnlyFileSystem","description":"Container images with readOnlyRootFileSystem set as false mounts the container root file system with write permissions","rule_id":"AC-K8-IA-PO-M-0140","severity":"MEDIUM","category":"Identity and Access Management","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"imageWithoutDigest","description":"Image without digest affects the integrity principle of image security","rule_id":"AC-K8-NS-PO-M-0133","severity":"MEDIUM","category":"Infrastructure Security","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"noReadinessProbe","description":"No readiness probe will affect automatic recovery in case of unexpected errors","rule_id":"AC-K8-OE-PO-L-0130","severity":"LOW","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"imageWithLatestTag","description":"No tag or container image with :Latest tag makes difficult to rollback and track","rule_id":"AC-K8-OE-PO-L-0134","severity":"LOW","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"MemorylimitsCheck","description":"Memory Limits Not Set in config file.","rule_id":"AC-K8-OE-PK-M-0158","severity":"Medium","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"nolivenessProbe","description":"No liveness probe will ensure there is no recovery in case of unexpected errors","rule_id":"AC-K8-OE-PO-L-0129","severity":"LOW","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"otherNamespace","description":"Default Namespace Should Not be Used","rule_id":"AC-K8-OE-PO-M-0166","severity":"MEDIUM","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"defaultNamespaceUsed2","description":"The default namespace should not be used","rule_id":"accurics.kubernetes.OPS.461","severity":"LOW","category":"Operational Efficiency","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"secCompProfile","description":"Default seccomp profile not enabled will make the container to make non-essential system calls","rule_id":"AC-K8-IA-PO-M-0141","severity":"MEDIUM","category":"Identity and Access Management","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"CpulimitsCheck","description":"CPU Limits Not Set in config file.","rule_id":"AC-K8-OE-PK-M-0156","severity":"Medium","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"appArmorProfile","description":"AppArmor profile not set to default or custom profile will make the container vulnerable to kernel level threats","rule_id":"AC-K8-IA-PO-M-0135","severity":"MEDIUM","category":"Identity and Access Management","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"privilegeEscalationCheck","description":"Containers Should Not Run with AllowPrivilegeEscalation","rule_id":"AC-K8-CA-PO-H-0165","severity":"HIGH","category":"Compliance Validation","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"securityContextUsed","description":"Apply Security Context to Your Pods and Containers","rule_id":"AC-K8-NS-PO-M-0122","severity":"MEDIUM","category":"Infrastructure Security","resource_name":"green","resource_type":"kubernetes_deployment"}
{"rule_name":"CpuRequestsCheck","description":"CPU Request Not Set in config file.","rule_id":"AC-K8-OE-PK-M-0155","severity":"Medium","category":"Security Best Practices","resource_name":"green","resource_type":"kubernetes_deployment"}

[devang-gaur]

I think we can close this issue now. Please re-open if there's more queries in this context.

[devang-gaur]

@alexfeig For more future proofing and stricter checks maybe try using the "*" wildcard for apiVersions list as well.

[alexfeig]

Yes, please see the example from earlier that I gave that works: #731 (comment)

Cheers!