KEP: Security Profile Design Proposal #2052

easeway · 2018-04-18T23:24:59Z

This is the detailed design proposal for Security Profile.

Security Profile helps managing a secured Kubernetes cluster by solving problems of:

Management complexity
- Complicated configuration of control plane components' security-relevant command line flags, correctness of cert/key/config file permissions;
- Requires deep understanding of Kubernetes policy objects like RBAC, PodSecurityPolicy, NetworkPolicy, ResourceQuota etc;
- No support for automatic provisioning of namespaced policy objects upon namespace creation;
- Requires of domain knowledge about security;
- Requires keeping up with fast evolving Kubernetes security/isolation features, and update configurations accordingly;
Extra efforts building in-house solutions
- No standard/common ways to solve the problems;
- It’s very difficult to make in-house solutions sharable/reusable, as a result, efforts are duplicated solving same/similar problems.

by defining curated settings of:

Cluster bootstrapping configuration, like command line flags, file permissions, etc
Cluster-scoped policy objects, like PodSecurityPolicy
Namespace-scoped policy objects, like NetworkPolicy

And simplify the cluster management by specifying the name of security profile.

This proposal is looking for sponsorship from sig-auth,
to create a public repo as https://github.com/kubernetes-sigs/security-profiles,
that community can help curate commonly used security profiles.
Details are described in Release section in the proposal.

/sig auth

/cc @davidopp
/cc @tallclair
/cc @ericchiang
/cc @liggitt

easeway · 2018-04-18T23:26:27Z

/assign @tallclair

k8s-ci-robot · 2018-04-19T23:34:51Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: jbeda

Assign the PR to them by writing /assign @jbeda in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

keps/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ericchiang · 2018-04-23T17:06:19Z

I'm concerned with the lack of scoping in this proposal. Autopopulating namespaces, config as code, bootstrapping config, conformance testing, meta-policies… there are at least three or four unique sub-projects in this document.

Before we get into the nitty gritty of the proposal, does it make sense to keep all of these concepts as one framework? Naively I could see conformance profiles and namespace templating being separate. Conformance profiles are a way to express the capabilities a cluster provides. Namespace templates are a tool for admins to quickly provision namespaces for users with sane security defaults.

easeway · 2018-04-23T23:16:19Z

@ericchiang you are right. The unique pieces covered in the proposal are relatively independent/separate. They can be used individually. The reason putting all things together is for a really simple user experience and also the possibility to share the experience across Kubernetes clusters in different environments.

I agree it will be better to break up and focus on individual pieces. I will send out individual proposals, covering:

Conformance profiles KEP: security conformance proposal #2081
Namespace population KEP: Namespace Population #2177
Cluster bootstrapping
Config as code

And later update this KEP as a complete solution to demonstrate the ultimate user experience we want to achieve after things are integrated.

mayakacz · 2018-06-12T22:09:46Z

keps/draft-20180418-security-profile.md

+  - TLS required for communication
+  - Explicitly specify `--wal-dir`, and `--max-wals=0`
+- kube-apiserver
+  - TLS required for communication


how is this enforced/ verified?

They are defined in bootstrapping rules, the enforcement of bootstrapping rules are implementation specific, thus out of scope in this proposal.
Verification is out of scope.
I will call out these explicitly.

mayakacz · 2018-06-12T22:12:12Z

keps/draft-20180418-security-profile.md

+  - Audit logging must be explicitly configured
+  - No anonymous auth, basic auth (unset `--anonymous-auth`, `--basic-auth-file`)
+  - Required authorization-mode: Node, RBAC
+  - Excluded authorization-mode: ABAC, AlwaysAllow


please add: automountServiceAccountToken: false

automountServiceAccountToken is a field in ServiceAccount or Pod spec. Currently, it's difficult to define that in security profile as it's can't be enforced by any component (though security profile can introduce a admission webhook to mutate that field, however, the intention of security profile is to leveraging existing mechanism, not introduce something new). So I think this will be a good feature in PodSecurityPolicy, then it can be defined in security profile.

ack - that makes sense

mayakacz · 2018-06-12T22:13:44Z

keps/draft-20180418-security-profile.md

+- file-permissions
+  - ... # correct setup of cert/key/config file permissions
+- PodSecurityPolicy
+  - No priviledged


can you specify? I'm thinking this is: privileged: false, allowPrivilegeEscalation: false, MustRunAsNonRoot

Or is this stronger than what you were thinking for 'default'?

I think it's stronger than what the default is proposed. I'm even thinking of removing No privileged in this profile. The default profile is designed to give the maximum compatibility with existing workloads. Add restrictions to block privileged will definitely break some workloads. So I think I need rephrase the goal for each security profiles, and add one more between default and saas-multitenancy.

mayakacz · 2018-06-12T22:14:50Z

keps/draft-20180418-security-profile.md

+Enforcement: in addition to `default`
+
+- PodSecurityPolicy
+  - No privileged pods


Isn't this already in 'default'?

I should remove that from default.

so 'default' would allow privileged pods?
Obviously I'm coming at this from a security angle, but if I turn on security profiles, I'm probably OK with it warning me that I shouldn't be running pods as privileged and then fix that - or are you saying that there isn't a lower level so they're stuck with that issue?

This is my personal opinion (I'd like to hear other thoughts, as I'm probably wrong): the default security profile is provided to bring in the lowest level (there's no lower levels) of security (eliminate some really bad insecure configurations) without breaking compatibility of how people are using Kubernetes today. So we can use default to help Kubernetes users smoothly transit from insecure to secure. I believe privileged containers are allowed in most Kubernetes clusters with trusted users (like inside the same corp or team) and sometimes useful. That's the reason I removed "no-privileged containers" from default profile.

got it, I see you defined default as "to be compatible with most workloads". That makes sense then.

mayakacz · 2018-06-12T22:15:21Z

keps/draft-20180418-security-profile.md

+    - persistentVolumeClaim
+    - projected
+    - secret
+- NetworkPolicy


Why wouldn't we include this in 'default'?

Adding NetworkPolicy (blocking all traffic by default) will break a lot of existing workloads which doesn't assume the existence of NetworkPolicy. So I think it doesn't belong to default.

got it, agreed.

mayakacz · 2018-06-12T22:24:49Z

keps/draft-20180418-security-profile.md

+
+#### Release
+Security profiles are maintained as an individual project similarly to other Kubernetes open source projects, like heapster, dashboard etc.
+It has its own release lifecycle.


What is the goal? To roughly release in line with K8s major releases, for example?

I will add more details.

mayakacz · 2018-06-12T22:27:36Z

keps/draft-20180418-security-profile.md

+Uninstall is easy as resources from a bundle are labeled with bundle name.
+With multiple bundles installed, it's possible multiple admission controls, controllers are running side-by-side.
+With help of labeled bundle name, the admission controls, controllers will stop enforcement if the currently selected security profile is not from the same bundle.
+With this mechanism, it's very simple and lightweight to switch back to the previous security profile if something unexpected happens with a new security profile.


to the previous security profile version?

not exactly the version. The user may upgrade from default to saas, and find out breaking workloads and decided to switch back to default.

Makes sense

mayakacz · 2018-06-12T22:28:05Z

keps/draft-20180418-security-profile.md

+With help of labeled bundle name, the admission controls, controllers will stop enforcement if the currently selected security profile is not from the same bundle.
+With this mechanism, it's very simple and lightweight to switch back to the previous security profile if something unexpected happens with a new security profile.
+
+When upgrading a cluster, it's possible the current security profile may not work on the new cluster version.


may not work, or may not be supported? (i.e., 'deprecated')

to be accurate: may not be supported. (it's not 'deprecated' which means the profile may still work, but may not guarantee the security level it's promised to provide because of imperfection of rules, bugs in implementations, etc.)

mayakacz · 2018-06-12T22:29:23Z

keps/draft-20180418-security-profile.md

+
+- Any bug found in enforcement logic (admission control, controllers etc)
+- Improper/Incomplete rules in the profile
+- Other security concerns


would we ever deprecate a security profile when it no longer ad some critical security functionality, even though it was the best practice for that K8s version?

For example, would we deprecate a security profile for K8s 1.5, because it does not have the ability to enforce RBAC? Is there some minimum K8s version (or recency) we require for security profiles?

In this case, a security profile won't be deprecated for the supported K8S version. In the example mentioned, the profile will not be deprecated for K8s 1.5, however it's not supported in K8s 1.6 or above with RBAC. So for users upgrading K8s from 1.5 to 1.6 they will have to upgrade the security profile to a new version which supports both 1.5 and 1.6.

mayakacz · 2018-06-12T22:30:18Z

keps/draft-20180418-security-profile.md

+The alternative is put test suite together with Kubernetes E2E tests in the official Kubernetes repository,
+It will be a little difficult to synchronize the development work in two repos.
+
+The details is out of scope of this doc. A separate doc will be focus on conformance tests.


please share a link to the KEP

I will add that.

easeway · 2018-06-13T18:37:30Z

Removed Conformance tests which is on a different layer and should be separate from security profile.

idealhack · 2018-07-04T08:08:57Z

/ok-to-test

k8s-ci-robot · 2018-07-04T08:10:13Z

@easeway: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-community-verify	`a8cb206`	link	`/test pull-community-verify`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

fejta-bot · 2018-10-02T08:27:00Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

justaugustus · 2018-10-13T03:47:04Z

/kind kep
/remove-lifecycle stale

mattfarina · 2018-11-07T16:28:16Z

@mikedanese @enj @tallclair This KEP is listed under sig-auth. Have y'all reviewed it yet?

justaugustus · 2018-11-20T04:46:05Z

REMINDER: KEPs are moving to k/enhancements on November 30. Please attempt to merge this KEP before then to signal consensus.
For more details on this change, review this thread.

Any questions regarding this move should be directed to that thread and not asked on GitHub.

justaugustus · 2018-12-01T08:11:43Z

KEPs have moved to k/enhancements.
This PR will be closed and any additional changes to this KEP should be submitted to k/enhancements.
For more details on this change, review this thread.

Any questions regarding this move should be directed to that thread and not asked on GitHub.
/close

k8s-ci-robot · 2018-12-01T08:11:44Z

@justaugustus: Closed this PR.

In response to this:

KEPs have moved to k/enhancements.
This PR will be closed and any additional changes to this KEP should be submitted to k/enhancements.
For more details on this change, review this thread.

Any questions regarding this move should be directed to that thread and not asked on GitHub.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

surajssd · 2019-01-15T06:24:12Z

What is the new location of this PR? I don't see it in k/enhancements?

k8s-ci-robot added sig/auth Categorizes an issue or PR as relevant to SIG Auth. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 18, 2018

k8s-ci-robot requested review from ericchiang and tallclair April 18, 2018 23:25

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. kind/design Categorizes issue or PR as related to design. labels Apr 18, 2018

k8s-ci-robot assigned tallclair Apr 18, 2018

easeway force-pushed the securityprofile-proposal branch from b9aedc7 to b628c64 Compare April 19, 2018 23:34

k8s-ci-robot added the sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. label Apr 19, 2018

easeway changed the title ~~Security Profile Design Proposal~~ KEP: Security Profile Design Proposal Apr 19, 2018

easeway mentioned this pull request May 22, 2018

KEP: Namespace Population #2177

Closed

mayakacz reviewed Jun 12, 2018

View reviewed changes

KEP: Security Profile Design Proposal

a8cb206

easeway force-pushed the securityprofile-proposal branch from b628c64 to a8cb206 Compare June 13, 2018 18:36

k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 4, 2018

jdumars mentioned this pull request Jul 12, 2018

KEP# TBD: Security Design kubernetes-retired/architecture-tracking#23

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 2, 2018

jorgemoralespou mentioned this pull request Oct 2, 2018

R&D - component configuration semantics redhat-developer/odo#675

Closed

k8s-ci-robot added kind/kep and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Oct 13, 2018

kkasravi mentioned this pull request Oct 28, 2018

create a self-serve component for data-scientists kubeflow/kubeflow#1872

Merged

k8s-ci-robot closed this Dec 1, 2018

KEP: Security Profile Design Proposal #2052

KEP: Security Profile Design Proposal #2052

Conversation

easeway commented Apr 18, 2018 • edited Loading

easeway commented Apr 18, 2018

k8s-ci-robot commented Apr 19, 2018

ericchiang commented Apr 23, 2018

easeway commented Apr 23, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

easeway commented Jun 13, 2018

idealhack commented Jul 4, 2018

k8s-ci-robot commented Jul 4, 2018

fejta-bot commented Oct 2, 2018

justaugustus commented Oct 13, 2018

mattfarina commented Nov 7, 2018

justaugustus commented Nov 20, 2018

justaugustus commented Dec 1, 2018

k8s-ci-robot commented Dec 1, 2018

surajssd commented Jan 15, 2019

easeway commented Apr 18, 2018 •

edited

Loading

easeway commented Apr 23, 2018 •

edited

Loading