Update pod security standards to use PodOS field #35772

ravisantoshgudimetla · 2022-08-07T22:58:03Z

netlify · 2022-08-07T22:58:13Z

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Name	Link
🔨 Latest commit	`a1f6615`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/62fe97647e40e8000854980e

ravisantoshgudimetla · 2022-08-07T23:00:33Z

/cc @tallclair @liggitt

/sig auth
/sig windows

sftim

We try to write documentation to be timeless - see https://developers.google.com/style/timeless-documentation for an example.

Here are some suggestions with that in mind.

content/en/docs/concepts/security/pod-security-standards.md

sftim · 2022-08-08T10:14:52Z

content/en/docs/concepts/security/pod-security-standards.md

 					Containers must drop <code>ALL</code> capabilities, and are only permitted to add back
-					the <code>NET_BIND_SERVICE</code> capability.
+					the <code>NET_BIND_SERVICE</code> capability. Starting v1.25, windows pods would be exempted from this control if pod.spec.os.Name is set to <code>windows</code>


Suggested change

Containers must drop <code>ALL</code> capabilities, and are only permitted to add back

the <code>NET_BIND_SERVICE</code> capability.

the <code>NET_BIND_SERVICE</code> capability. Starting v1.25, windows pods would be exempted from this control if pod.spec.os.Name is set to <code>windows</code>

For Linux pods, require that containers drop all capabilities:

<code>ALL</code>.

As a special case, containers are permitted to add back the

<code>NET_BIND_SERVICE</code> capability (and only this capability).

This control only applies when you set `spec.os` for the overall Pod to

<code>linux</code>, or if you do not specify a

Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of

operating system.

sftim · 2022-08-08T10:21:59Z

content/en/docs/concepts/security/pod-security-standards.md

-profile. However, the lack of enforcement means that there is no additional restriction, for Pods
-that use Windows containers, compared to the baseline profile.
+
+In v1.25, the `pod.spec.os` field has graduated to stable which helps identify the windows pods authoritatively. Pod Security admission plugin has been updated to use this field which allows certain relaxations for the windows pods. The restricted profile has been updated to ensure that the Linux-specific restrictions such as seccomp profile, disallowing privilege escalation, capabilities are ignored if the `pod.spec.os.Name` is set to `windows`.


Suggested change

In v1.25, the `pod.spec.os` field has graduated to stable which helps identify the windows pods authoritatively. Pod Security admission plugin has been updated to use this field which allows certain relaxations for the windows pods. The restricted profile has been updated to ensure that the Linux-specific restrictions such as seccomp profile, disallowing privilege escalation, capabilities are ignored if the `pod.spec.os.Name` is set to `windows`.

For Kubernetes v{{< skew currentVersion >}} it is good practice to specify

the `.spec.os` field for every Pod. The `.spec.os` field is available in Kubernetes

v1.23 and later but was behind a

[feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/)

for Kubernetes v1.23 and v1.24.

If you set the Pod OS field to `linux`, only Linux nodes try to run the Pod, and the enforced security

controls are appropriate to Linux.

Setting the Pod OS field to `windows` means that only Windows nodes would try to run the Pod, and

you don't need to apply Linux-specific restrictions. For example, a Windows Pod does not

need to explicitly drop all capabilities for each container, even if you are applying the `restricted`

Pod security standard.

If you don't specify the Pod OS, Kubernetes takes a precautionary approach and applies all the Pod

security measures that could be relevant.

I would move this section out of the FAQ and move it to a 2nd-level heading just above the FAQ.
The section heading I'd use would be:

## Security standards for different operating systems

I am fine with @tallclair any concerns?

I agree. I was going to suggest the same thing.

chrisnegus

I just have one capitalization suggestion that occurs several time.

chrisnegus · 2022-08-09T01:04:03Z

content/en/docs/concepts/security/pod-security-standards.md

@@ -407,7 +407,7 @@ fail validation.
 			<td>
 				<p>
 					Containers must drop <code>ALL</code> capabilities, and are only permitted to add back
-					the <code>NET_BIND_SERVICE</code> capability.
+					the <code>NET_BIND_SERVICE</code> capability. Starting v1.25, windows pods would be exempted from this control if pod.spec.os.Name is set to <code>windows</code>


Other k8s docs capitalize the "W" in "Windows pods". I suggest changing the lowercase "windows pods" instances in this document.

Sure. Will change it.

reylejano · 2022-08-09T14:39:11Z

/milestone 1.25
/assign @cathchu
/cc @kcmartin

tallclair · 2022-08-12T20:53:36Z

/assign

tallclair · 2022-08-12T21:02:48Z

content/en/docs/concepts/security/pod-security-standards.md

@@ -326,7 +326,7 @@ fail validation.
 		<tr>
 			<td style="white-space: nowrap">Privilege Escalation (v1.8+)</td>
 			<td>
-				<p>Privilege escalation (such as via set-user-ID or set-group-ID file mode) should not be allowed.</p>
+				<p>On Linux, do not allow privilege escalation (such as via set-user-ID or set-group-ID file mode). This control only applies when you set `spec.os` to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.</p>


Rather than integrating this text in here, I'd suggest just having a standard header that you can add to all the linux-only fields. I prefer this approach because it makes it easier to identify the linux-only fields with a visual scan.

Maybe something like this:

<a href="#what-profiles-should-i-apply-to-my-windows-pods">Linux only</a> in v1.25+ (`spec.os != windows`)

It's a bit odd to hyperlink “Linux only” to #what-profiles-should-i-apply-to-my-windows-pods - maybe we could have a new page section that covers OS specifics, with a subheading for Linux and another subheading for Windows.

The content for Windows is pretty small with only 3 policies changes in restricted profile for now. May be we can create a new page section when the content increases for Windows? How about just a note?

tallclair · 2022-08-12T21:14:18Z

content/en/docs/concepts/security/pod-security-standards.md

@@ -381,7 +381,7 @@ fail validation.
 		<tr>
  			<td style="white-space: nowrap">Seccomp (v1.19+)</td>
  			<td>
-  				<p>Seccomp profile must be explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the <em>absence</em> of a profile are prohibited.</p>
+  				<p>On Linux, require that the seccomp profile is explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the <em>absence</em> of a profile are prohibited.  This control only applies when you set `spec.os` to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.</p>


... when you set spec.os to linux ...

It's actually coded the other way - skip the check when it's set to windows. A subtle difference, but it could matter if we added another OS.

I wonder if I should also file a feature request issue to invert the logic there?

exempting only for an explicitly windows pod is intentional, so we default in the safe direction for new hitherto unknown OSes

I thought about it but did not want to include negations(skips) in the docs for skipping. However I agree since there is a chance of expanding OS'es later it is better to talk about skips.

tallclair · 2022-08-12T21:16:50Z

content/en/docs/concepts/security/pod-security-standards.md

@@ -381,7 +381,7 @@ fail validation.
 		<tr>
  			<td style="white-space: nowrap">Seccomp (v1.19+)</td>
  			<td>
-  				<p>Seccomp profile must be explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the <em>absence</em> of a profile are prohibited.</p>
+  				<p>On Linux, require that the seccomp profile is explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the <em>absence</em> of a profile are prohibited.  This control only applies when you set `spec.os` to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.</p>


Unfortunately markdown syntax isn't processed in this table, so you need to use raw HTML tags.

Suggested change

On Linux, require that the seccomp profile is explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the absence of a profile are prohibited. This control only applies when you set `spec.os` to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.

On Linux, require that the seccomp profile is explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the absence of a profile are prohibited. This control only applies when you set <code>spec.os</code> to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.

Also, is it actually .spec.os.name?

tallclair · 2022-08-12T21:17:46Z

content/en/docs/concepts/security/pod-security-standards.md

-profile. However, the lack of enforcement means that there is no additional restriction, for Pods
-that use Windows containers, compared to the baseline profile.
+
+In v1.25, the `pod.spec.os` field has graduated to stable which helps identify the windows pods authoritatively. Pod Security admission plugin has been updated to use this field which allows certain relaxations for the windows pods. The restricted profile has been updated to ensure that the Linux-specific restrictions such as seccomp profile, disallowing privilege escalation, capabilities are ignored if the `pod.spec.os.Name` is set to `windows`.


I agree. I was going to suggest the same thing.

ravisantoshgudimetla · 2022-08-15T15:18:12Z

I slightly tweaked your suggestions @sftim and @tallclair. PTAL

sftim

I do recommend some more changes here.

content/en/docs/concepts/security/pod-security-standards.md

sftim · 2022-08-15T16:16:40Z

content/en/docs/concepts/security/pod-security-standards.md

+to Linux. 
+
+### Restricted Pod Security Standard changes
+Another important change starting Kubernetes v{{ < skew currentVersion >}} is that the pod restricted standard


Suggested change

Another important change starting Kubernetes v{{ < skew currentVersion >}} is that the pod restricted standard

Another important change, made in Kubernetes v1.25, is that the _restricted_ Pod security standard

If you can, rewrite the text to define and document the current (oost-v1.25) behaviour, rather than the state change.

I was actually thinking it might be worth having an explicit changelog on this page. This is a bit different from documenting Kubernetes changes, as the older profiles are supported indefinitely, across Kubernetes versions.

@tallclair - You mean a link to the changelog on this page?

content/en/docs/concepts/security/pod-security-standards.md

sftim · 2022-08-15T16:20:46Z

content/en/docs/concepts/security/pod-security-standards.md

+workloads. Specifically, many of the Pod SecurityContext fields
+[have no effect on Windows](/docs/concepts/windows/intro/#compatibility-v1-pod-spec-containers-securitycontext).
+
+### Pod API validation changes


Suggested change

### Pod API validation changes

### Pod security checks based on Pod operating system

Document the behavior, not the change in behavior (that's for the release notes)/

I think we should drop this section. Pod API validation is not part of the Pod Security Standards.

ravisantoshgudimetla

Yup. Those changes look good to me. Thank you for making them @sftim

content/en/docs/concepts/security/pod-security-standards.md

tallclair · 2022-08-17T17:57:25Z

content/en/docs/concepts/security/pod-security-standards.md

+workloads. Specifically, many of the Pod SecurityContext fields
+[have no effect on Windows](/docs/concepts/windows/intro/#compatibility-v1-pod-spec-containers-securitycontext).
+
+### Pod API validation changes


I think we should drop this section. Pod API validation is not part of the Pod Security Standards.

tallclair · 2022-08-17T17:58:01Z

content/en/docs/concepts/security/pod-security-standards.md

+to Linux. 
+
+### Restricted Pod Security Standard changes
+Another important change starting Kubernetes v{{ < skew currentVersion >}} is that the pod restricted standard


I was actually thinking it might be worth having an explicit changelog on this page. This is a bit different from documenting Kubernetes changes, as the older profiles are supported indefinitely, across Kubernetes versions.

content/en/docs/concepts/security/pod-security-standards.md

tallclair · 2022-08-17T18:07:16Z

content/en/docs/concepts/security/pod-security-standards.md

+Windows in Kubernetes has some limitations and differentiators from standard Linux-based
+workloads. Specifically, many of the Pod SecurityContext fields
+[have no effect on Windows](/docs/concepts/windows/intro/#compatibility-v1-pod-spec-containers-securitycontext).
+


Add a warning here about how Kubelets prior to v1.24 don't enforce the pod OS field, and if a cluster has nodes on versions earlier than v1.24 the restricted policies should be pinned to a version prior to v1.25.

Added note here: https://github.com/kubernetes/website/pull/35772/files#diff-1eb43daa07b175394d3d428f40b35e9010ad719ce9fa8c0e858a0f773e4330faR472

reylejano · 2022-08-18T18:29:28Z

Hi @ravisantoshgudimetla , are you able to address the comments and feedback
The 1.25 Docs complete deadline passed and this PR needs to address the comments and feedback as soon as possible

ravisantoshgudimetla

Thanks for the suggestions @sftim and @tallclair

@reylejano - I am on PTO till the end of this week with limited access to the internet so I'm making changes as soon as I can. If someone can pick this PR up to push it along, I am fine with it.

sftim · 2022-08-22T08:49:34Z

Should we:

merge this as-is
postpone the v1.25 release?

ravisantoshgudimetla · 2022-08-22T13:18:53Z

Is this the only outstanding comment on which we need to get feedback?

reylejano · 2022-08-22T14:40:48Z

let's merge as-is and I'll create a follow-up PR to address comments and suggestions
/approve

k8s-ci-robot · 2022-08-22T14:41:23Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: reylejano

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [reylejano]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sftim · 2022-08-22T15:05:32Z

/lgtm

We should review this after the release, as we're merging updates to docs about a security control, and we don't yet have a tech review signoff from the right SIGs.

k8s-ci-robot · 2022-08-22T15:05:52Z

LGTM label has been added.

Git tree hash: 805ba4d3f310232a934431e96bf864d447a65d9b

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 7, 2022

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. language/en Issues or PRs related to English language labels Aug 7, 2022

k8s-ci-robot requested review from daminisatya and tallclair August 7, 2022 22:58

k8s-ci-robot added the sig/docs Categorizes an issue or PR as relevant to SIG Docs. label Aug 7, 2022

This was referenced Aug 7, 2022

Promote PodOS field to GA #35590

Merged

Identify Windows pods at API admission level authoritatively kubernetes/enhancements#2802

Closed

k8s-ci-robot requested a review from liggitt August 7, 2022 23:00

k8s-ci-robot added sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/windows Categorizes an issue or PR as relevant to SIG Windows. labels Aug 7, 2022

sftim reviewed Aug 8, 2022

View reviewed changes

chrisnegus reviewed Aug 9, 2022

View reviewed changes

k8s-ci-robot assigned cathchu Aug 9, 2022

k8s-ci-robot added this to the 1.25 milestone Aug 9, 2022

k8s-ci-robot requested a review from kcmartin August 9, 2022 14:39

k8s-ci-robot assigned tallclair Aug 12, 2022

tallclair reviewed Aug 12, 2022

View reviewed changes

ravisantoshgudimetla force-pushed the psp-restricted branch from f597040 to 808316f Compare August 15, 2022 14:43

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 15, 2022

ravisantoshgudimetla force-pushed the psp-restricted branch 4 times, most recently from 1071f44 to ac09f46 Compare August 15, 2022 15:15

sftim reviewed Aug 15, 2022

View reviewed changes

ravisantoshgudimetla commented Aug 16, 2022

View reviewed changes

tallclair reviewed Aug 17, 2022

View reviewed changes

ravisantoshgudimetla commented Aug 18, 2022

View reviewed changes

ravisantoshgudimetla force-pushed the psp-restricted branch from 7fca67b to a98df83 Compare August 18, 2022 19:45

Update pod security standards to use PodOS field

a1f6615

ravisantoshgudimetla force-pushed the psp-restricted branch from a98df83 to a1f6615 Compare August 18, 2022 19:47

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 22, 2022

k8s-ci-robot assigned sftim Aug 22, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 22, 2022

k8s-ci-robot merged commit 735d499 into kubernetes:dev-1.25 Aug 22, 2022

dipesh-rawat mentioned this pull request Apr 27, 2023

Broken Link in Security Pod Security Standard #40867

Closed

-In v1.25, the `pod.spec.os` field has graduated to stable which helps identify the windows pods authoritatively. Pod Security admission plugin has been updated to use this field which allows certain relaxations for the windows pods. The restricted profile has been updated to ensure that the Linux-specific restrictions such as seccomp profile, disallowing privilege escalation, capabilities are ignored if the `pod.spec.os.Name` is set to `windows`.
+For Kubernetes v{{< skew currentVersion >}} it is good practice to specify
+the `.spec.os` field for every Pod. The `.spec.os` field is available in Kubernetes
+v1.23 and later but was behind a
+[feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/)
+for Kubernetes v1.23 and v1.24.
+If you set the Pod OS field to `linux`, only Linux nodes try to run the Pod, and the enforced security
+controls are appropriate to Linux.
+Setting the Pod OS field to `windows` means that only Windows nodes would try to run the Pod, and
+you don't need to apply Linux-specific restrictions. For example, a Windows Pod does not
+need to explicitly drop all capabilities for each container, even if you are applying the `restricted`
+Pod security standard.
+If you don't specify the Pod OS, Kubernetes takes a precautionary approach and applies all the Pod
+security measures that could be relevant.

	<p>On Linux, require that the seccomp profile is explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the <em>absence</em> of a profile are prohibited. This control only applies when you set `spec.os` to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.</p>
	<p>On Linux, require that the seccomp profile is explicitly set to one of the allowed values. Both the <code>Unconfined</code> profile and the <em>absence</em> of a profile are prohibited. This control only applies when you set <code>spec.os</code> to <code>linux</code>, or if you do not specify a Pod OS. Versions of Kubernetes before v1.25 enforce this control regardless of operating system.</p>

	Another important change starting Kubernetes v{{ < skew currentVersion >}} is that the pod restricted standard
	Another important change, made in Kubernetes v1.25, is that the _restricted_ Pod security standard

	### Pod API validation changes
	### Pod security checks based on Pod operating system

Update pod security standards to use PodOS field #35772

Update pod security standards to use PodOS field #35772

Conversation

ravisantoshgudimetla commented Aug 7, 2022

netlify bot commented Aug 7, 2022 • edited Loading

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

ravisantoshgudimetla commented Aug 7, 2022

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sftim Aug 8, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisnegus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reylejano commented Aug 9, 2022

tallclair commented Aug 12, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravisantoshgudimetla Aug 15, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravisantoshgudimetla commented Aug 15, 2022 • edited Loading

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravisantoshgudimetla left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reylejano commented Aug 18, 2022

ravisantoshgudimetla left a comment

Choose a reason for hiding this comment

sftim commented Aug 22, 2022

ravisantoshgudimetla commented Aug 22, 2022

reylejano commented Aug 22, 2022

k8s-ci-robot commented Aug 22, 2022

sftim commented Aug 22, 2022

k8s-ci-robot commented Aug 22, 2022

netlify bot commented Aug 7, 2022 •

edited

Loading

sftim Aug 8, 2022 •

edited

Loading

ravisantoshgudimetla Aug 15, 2022 •

edited

Loading

ravisantoshgudimetla commented Aug 15, 2022 •

edited

Loading

ravisantoshgudimetla left a comment •

edited

Loading