Limit Default RBAC Rules For Kanister Operator

[r4rajat]

Change Overview

Current default RBAC role bindings for kanister operator allow creation of pods in any namespaces with any service accounts, thus enabling privilege escalations.

Kanister should adopt safer default, such as only allowing to create pods in the kanister namespace.

Current Docs Screenshots

New Docs Screenshot

Pull request type

Please check the type of change your PR introduces:

• 🚧 Work in Progress
• 🌈 Refactoring (no functional changes, no api changes)
• 🐹 Trivial/Minor
• 🐛 Bugfix
• 🌻 Feature
• 🗺️ Documentation
• 🤖 Test
• 🏗️ Build

Issues

• fixes Security: limit default RBAC rules for kanister operator #3050

Test Plan

• 💪 Manual
• ⚡ Unit test
• 💚 E2E

Manual Testing

1) Install Kanister and MySQL

helm install kanister ./helm/kanister-operator --namespace kanister --create-namespace

helm install mysql bitnami/mysql --namespace mysql \
  --set auth.rootPassword=rootPasSw0rd \
  --set auth.database=mydb \
  --create-namespace

2) Create Blueprint, Profile and ActionSet to Backup mysql StatefulSet

kubectl --namespace kanister apply -f \
    https://raw.githubusercontent.com/kanisterio/kanister/0.110.0/examples/mysql/mysql-blueprint.yaml

blueprint.cr.kanister.io/mysql-blueprint created

kanctl create profile s3compliant --access-key $AWS_ACCESS_KEY_ID \
--secret-key $AWS_SECRET_ACCESS_KEY \
--bucket $AWS_S3_BUCKET --region $AWS_REGION \
--namespace mysql

secret 's3-secret-rf3iic' created
profile 's3-profile-hln6k' created

kanctl create actionset --action backup --namespace kanister --blueprint mysql-blueprint \
--statefulset mysql/mysql --profile mysql/s3-profile-hln6k \
--secrets mysql=mysql/mysql

actionset backup-tlfgj created

With Updated Permissions to Kanister Operator, It SHOULD NOT be able to create any Pod as part of KubeTask Function in mysql Namespace.

kubectl describe actionsets -n kanister backup-tlfgj

Name:         backup-tlfgj
Namespace:    kanister
Labels:       <none>
Annotations:  <none>
API Version:  cr.kanister.io/v1alpha1
Kind:         ActionSet
Metadata:
  Creation Timestamp:  2024-09-17T06:21:41Z
  Generation:          4
  Resource Version:    17316
  UID:                 cb52eac2-a886-413c-a14b-c65ee2cb4379
Spec:
  Actions:
    Blueprint:  mysql-blueprint
    Name:       backup
    Object:
      API Version:
      Group:
      Kind:             statefulset
      Name:             mysql
      Namespace:        mysql
      Resource:
    Preferred Version:
    Profile:
      API Version:
      Group:
      Kind:
      Name:         s3-profile-hln6k
      Namespace:    mysql
      Resource:
    Secrets:
      Mysql:
        API Version:
        Group:
        Kind:
        Name:         mysql
        Namespace:    mysql
        Resource:
Status:
  Actions:
    Artifacts:
      Mysql Cloud Dump:
        Key Value:
          s3path:  {{ .Phases.dumpToObjectStore.Output.s3path }}
    Blueprint:     mysql-blueprint
    Defer Phase:
      Name:
      Progress:
      State:
    Name:     backup
    Object:
      API Version:
      Group:
      Kind:         statefulset
      Name:         mysql
      Namespace:    mysql
      Resource:
    Phases:
      Name:  dumpToObjectStore
      Progress:
      State:  failed
  Error:
    Message:  secrets "mysql" is forbidden: User "system:serviceaccount:kanister:kanister-kanister-operator" cannot get resource "secrets" in API group "" in the namespace "mysql"
  Progress:
  State:  failed
Events:
  Type     Reason                          Age   From                 Message
  ----     ------                          ----  ----                 -------
  Normal   Started Action                  16s   Kanister Controller  Executing action backup
  Warning  ActionSetFailed Action: backup  16s   Kanister Controller  Failed to launch Action backup-tlfgj: secrets "mysql" is forbidden: User "system:serviceaccount:kanister:kanister-kanister-operator" cannot get resource "secrets" in API group "" in the namespace "mysql"

3) How to take Backup Now ?

Create a Role in mysql Namespace with edit ClusterRole and assign it to Kanister Operator's ServiceAccount

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: bind-mysql-namespace-access
  namespace: mysql
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: edit
subjects:
  - kind: ServiceAccount
    name: kanister-kanister-operator
    namespace: kanister

Take Backup Again

kanctl create actionset --action backup --namespace kanister --blueprint mysql-blueprint \
--statefulset mysql/mysql --profile mysql/s3-profile-hln6k \
--secrets mysql=mysql/mysql

actionset backup-z4cqc created

Check Status

kubectl describe actionsets -n kanister backup-z4cqc

Name:         backup-z4cqc
Namespace:    kanister
Labels:       <none>
Annotations:  <none>
API Version:  cr.kanister.io/v1alpha1
Kind:         ActionSet
Metadata:
  Creation Timestamp:  2024-09-17T06:24:31Z
  Generation:          5
  Resource Version:    19590
  UID:                 4d50905c-77e5-41f1-9fa9-8709dd390406
Spec:
  Actions:
    Blueprint:  mysql-blueprint
    Name:       backup
    Object:
      API Version:
      Group:
      Kind:             statefulset
      Name:             mysql
      Namespace:        mysql
      Resource:
    Preferred Version:
    Profile:
      API Version:
      Group:
      Kind:
      Name:         s3-profile-hln6k
      Namespace:    mysql
      Resource:
    Secrets:
      Mysql:
        API Version:
        Group:
        Kind:
        Name:         mysql
        Namespace:    mysql
        Resource:
Status:
  Actions:
    Artifacts:
      Mysql Cloud Dump:
        Key Value:
          s3path:  {{ .Phases.dumpToObjectStore.Output.s3path }}
    Blueprint:     mysql-blueprint
    Defer Phase:
      Name:
      Progress:
      State:
    Name:     backup
    Object:
      API Version:
      Group:
      Kind:         statefulset
      Name:         mysql
      Namespace:    mysql
      Resource:
    Phases:
      Name:  dumpToObjectStore
      Progress:
        Last Transition Time:  2024-09-17T06:24:36Z
        Progress Percent:      0
      State:                   running
  Error:
    Message:
  Progress:
    Last Transition Time:  2024-09-17T06:24:36Z
    Percent Completed:     0
    Running Phase:         dumpToObjectStore
  State:                   running
Events:
  Type    Reason           Age   From                 Message
  ----    ------           ----  ----                 -------
  Normal  Started Action   13s   Kanister Controller  Executing action backup
  Normal  Started Phase    13s   Kanister Controller  Executing phase dumpToObjectStore
  Normal  Ended Phase      0s    Kanister Controller  Completed phase dumpToObjectStore
  Normal  Update Complete  0s    Kanister Controller  Updated ActionSet 'backup-z4cqc' Status->complete

[viveksinghggits]

Can you please add the docs changes as well?

[r4rajat]

Can you please add the docs changes as well?

@viveksinghggits I've Added. Please take a look

[viveksinghggits]

@r4rajat We should make changes in docs_new as well and add a release note too.

[viveksinghggits]

I think we should try to rephrase it, maybe something like below

To enhance the security of the cluster, default installation of Kanister doesn't have access 
to any resource in other namespaces apart from where it's installed. Because of this Kanister
might not be able to snapshot or restore the applications by default. If the Blueprint requires access to the resources in other namespaces, please follow the steps here to configure the access properly. 

[viveksinghggits]

Please feel free to change above suggestion to make it better.

[r4rajat]

@viveksinghggits I've rephrased your suggestion, LMK if it looks good.

To improve the cluster's security, the default installation of Kanister is restricted to access only the resources within its own namespace. As a result, Kanister may not be able to snapshot or restore applications by default in other namespaces. If Blueprint needs access to resources in other namespaces, please follow the steps provided here to configure the access correctly.

[viveksinghggits]

I think we should mentioned that edit is a system created cluster role to make sure users don't confused about where did edit cluster role come from.

[r4rajat]

Updated

[viveksinghggits]

I think we should mention that if your blueprint doesn't require access to all the resources that are mentioned in edit cluster role, you can create the role with just the resources/verbs that the blueprint requires access to.

[r4rajat]

Updated

[viveksinghggits]

Suggested change

	To allow Kanister to perform backup/restore operations in a specific
	namespace, create a `RoleBinding` that assigns the `edit` `ClusterRole`
	to Kanister's Service Account:
	To allow Kanister to perform backup/restore operations in an application
	namespace, create a `RoleBinding` in the application namespace that associates the `edit` `ClusterRole`
	to Kanister's Service Account:

[r4rajat]

Updated

[viveksinghggits]

This looks good but I think we should talk about it when we are saying edit cluster role has been removed in above section.
Maybe immediately after RBAC Configuration, after explaining what edit cluster role is, we can say that kanister's service account used to get associated with the kaniter service account which doesn't happen anymore.

And I like how you have reformatted the above paragraphs based on my suggestion so please do that with above suggestion as well.

[r4rajat]

Could you check once @viveksinghggits

[viveksinghggits]

edit is a cluster role right?

[r4rajat]

Yeah, will update that

[viveksinghggits]

Suggested change

	1. Create a `Role` with the required permissions:
	1. Create a `Role` with the permissions required by your blueprint:

[r4rajat]

Done !

[viveksinghggits]

Let's add a line after this, saying after doing this kanister would be able to snapshot/restore the namespace successfully.

[r4rajat]

Done !

[viveksinghggits]

I think these two can be removed? because we are able to convey enough information using the upgrade and security notes.

[viveksinghggits]

LGTM.
We should see if someone can have a quick look, otherwise I will approve.

[viveksinghggits]

Suggested change

	Earlier, the Kanister operator was bound to the `edit` `ClusterRole`
	using a `ClusterRoleBinding` provided by the Helm Chart.
	The `edit` `ClusterRole` is a built-in Kubernetes system role that offers
	permissions to modify most objects within a namespace, excluding roles,
	role bindings, and resource quotas. This role allowed access to create,
	update, delete, and view resources such as Deployments, Pods, Services,
	ConfigMaps, PersistentVolumeClaims, and more.
	The `edit` `ClusterRole` is a built-in Kubernetes system role that offers
	permissions to modify most objects within a namespace, excluding roles,
	role bindings, and resource quotas. This role allowed access to create,
	update, delete, and view resources such as Deployments, Pods, Services,
	ConfigMaps, PersistentVolumeClaims, and more. Kanister helm chart used
	assign `edit` ClusterRole to the Kanister service account which gave all the
	permissions mentioned in the `edit` ClusterRole to the Kanister application.

[r4rajat]

Done !

[viveksinghggits]

is over-permisssioning correct english work? 😆

[r4rajat]

Updated !

[hairyhum]

LGTM.

@r4rajat One question: did you test the helm chart upgrade? Would it delete existing binding when upgraded?

[r4rajat]

LGTM.

@r4rajat One question: did you test the helm chart upgrade? Would it delete existing binding when upgraded?

@hairyhum I tested it by

• Installing older kanister version let's say 0.109.0, and I was able to see the edit ClusterRoleBinding for kanister operator and was able to take backup of mysql application.

• Upgraded kanister operator using Local Helm Chart, When I was listing ClusterRoleBindings, I was not seeing any edit ClusterRoleBinding for kanister operator, and the backup action failed with upgraded kanister as expected.