From 85be93a18941889567a2fa6f087dff22da88845a Mon Sep 17 00:00:00 2001 From: Brandon Smith Date: Thu, 22 Jul 2021 11:17:46 -0700 Subject: [PATCH 01/10] Rebasing HostProcess security changes. --- .../security/pod-security-standards.md | 16 ++ .../security/windows-pod-security-profile.md | 225 ++++++++++++++++++ .../en/docs/concepts/workloads/pods/_index.md | 5 +- .../feature-gates.md | 2 + .../create-hostprocess-container.md | 52 ++++ 5 files changed, 298 insertions(+), 2 deletions(-) create mode 100644 content/en/docs/concepts/security/windows-pod-security-profile.md create mode 100644 content/en/docs/tasks/configure-pod-container/create-hostprocess-container.md diff --git a/content/en/docs/concepts/security/pod-security-standards.md b/content/en/docs/concepts/security/pod-security-standards.md index 39fccb4230323..e31b7b51ac015 100644 --- a/content/en/docs/concepts/security/pod-security-standards.md +++ b/content/en/docs/concepts/security/pod-security-standards.md @@ -257,6 +257,22 @@ fail validation. + + Windows HostProcess + +

Windows pods offer the ability to run HostProcess containers which enables privileged access to the Windows node. As is similar to privileged containers this must be disallowed in the baseline policy.

+

Restricted Fields

+ +

Allowed Values

+ + + diff --git a/content/en/docs/concepts/security/windows-pod-security-profile.md b/content/en/docs/concepts/security/windows-pod-security-profile.md new file mode 100644 index 0000000000000..05327bb840011 --- /dev/null +++ b/content/en/docs/concepts/security/windows-pod-security-profile.md @@ -0,0 +1,225 @@ +--- +reviewers: +- tallclair +title: Windows Pod Security Policy +description: > + Clarification of which pod security policies apply to Windows pods +content_type: concept +weight: 10 +--- + + + +Windows in Kubernetes has some differentiators from standard Linux-based workloads. Many of the Pod SecurityContext fields [have no effect on +Windows](/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#v1-podsecuritycontext). Windows HostProcess containers also differ from traditional privileged containers, which causes some additional policies to lose their applicability for Windows. + +This guide outlines the applicable _policies_ for Windows Pod Security Standards and redefines the [existing profiles](/docs/concepts/security/pod-security-standards) for the Windows context. + +| Profile | Description | +| ------ | ----------- | +| Privileged | Unrestricted policy, providing the widest possible level of permissions. This policy allows for full access to the Windows host | +| Baseline | Minimally restrictive policy which prevents known privilege escalations. Allows the default (minimally specified) Pod configuration while blocking HostProcess container support. | +| Restricted | Unsupported until a standardized identifier for Windows pods is implemented. Windows pods _may_ be broken by the restricted field, which requires setting linux-specific settings (such as seccomp profile, run as non root, and disallow privilege escalation). If the Kubelet and/or container runtime choose to ignore these linux-specific values at runtime, then windows pods should still be allowed under the restricted profile, although the profile will not add additional enforcement over baseline (for Windows). | + + + +## Profile Details +Each of the profiles below detail which policies must be explicitly set. Any policy **not** in the profile which is also not in the [list of policies](./#policies-ignored-on-windows) ignored by Windows can assume supported configuration on a Windows node. + +### Privileged + +**As is true in the default Privileged policy, the _Windows Privileged_ policy is purposely-open, and entirely unrestricted.** This type of policy is aimed at system- and infrastructure-level workloads managed by privileged, trusted users. Pods running under this policy will be limited to only HostProcess containers and will have full visibility into the Windows node. + + + + + + + + + + + + + + + + + + +
Privileged policy specification
ControlWindows ApplicablePolicy
Windows HostProcess +

Windows pods offer the ability to run HostProcess containers which enables privileged access to the Windows node.

+

Restricted Fields

+
    +
  • spec.securityContext.windowsOptions.hostProcess
    • +
  • spec.containers[*].securityContext.windowsOptions.hostProcess
    • +
+

Allowed Values

+
    +
  • true
    • +
+
Host Namespaces +

Will be in host network by default initially. Support to set network to a different compartment may be desirable in the future.

+

Restricted Fields

+
    +
  • spec.hostNetwork
    • +
  • spec.hostPID
    • +
  • spec.hostIPC
    • +
+

Allowed Values

+
    +
  • true
    • +
+
+ +### Baseline + +**The _Baseline_ policy is aimed at ease of adoption for common containerized workloads while preventing known privilege escalations.** This policy is targeted at application operators and developers of non-critical applications. The following listed controls should be enforced/disallowed: + + + + + + + + + + + + + + + + + +
Baseline policy specification
ControlPolicy
Windows HostProcess +

As is similar to privileged containers this must be disallowed in the baseline policy.

+

Restricted Fields

+
    +
  • spec.securityContext.windowsOptions.hostProcess
    • +
  • spec.containers[*].securityContext.windowsOptions.hostProcess
    • +
+

Allowed Values

+
    +
  • Undefined/nil
    • +
  • false
    • +
+
Host Namespaces +

Not applicable. Windows privileged containers will be controlled with a new `WindowsSecurityContextOptions.HostProcess` instead of the existing `privileged` field due to fundamental differences in their implementation on Windows.

+

Restricted Fields

+
    +
  • spec.hostNetwork
    • +
  • spec.hostPID
    • +
  • spec.hostIPC
    • +
+

Allowed Values

+
    +
  • Undefined
    • +
  • false
    • +
+
+ +## Policies Ignored on Windows +Several policies in the Pod Security Standards do not apply to Windows nodes due to architectural differences between Linux and Windows. These policies are as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Privileged policy specification
ControlPolicy
Host Namespaces +

Windows does not have configurable PID/IPC namespaces (unlike Linux). Windows containers are always assigned their own process namespace. HostProcess containers always run in the host's process namespace. These behaviors are not configurable. Future plans in this area include improvements to enable scheduling pods that can contain both Windows Server and HostProcess containers. These fields would not makes in this scenario because Windows cannot configure PID/IPC namespaces like in Linux.

+

Unsupported Fields

+
    +
  • spec.hostNetwork
    • +
  • spec.hostPID
    • +
  • spec.hostIPC
    • +
+
Privileged Containers +

Not applicable. Windows privileged containers will be controlled with a new `WindowsSecurityContextOptions.HostProcess` instead of the existing `privileged` field due to fundamental differences in their implementation on Windows.

+

Unsupported Fields

+
    +
  • spec.containers[*].securityContext.privileged
    • +
  • spec.initContainers[*].securityContext.privileged
    • +
+
Capabilities +

Windows OS has a concept of “capabilities” (referred to as “privileged constants” but they are not supported in the platform today.

+

Unsupported Fields

+
    +
  • spec.containers[*].securityContext.capabilities.add
    • +
  • spec.initContainers[*].securityContext.capabilities.add
    • +
+
AppArmor +

AppArmor is not supported on Windows nodes.

+

Unsupported Fields

+
    +
  • metadata.annotations["container.apparmor.security.beta.kubernetes.io/*"]
    • +
+
SELinux +

SELinux is not supported by Windows nodes.

+

Unsupported Fields

+
    +
  • spec.securityContext.seLinuxOptions.*
    • +
  • spec.containers[*].securityContext.seLinuxOptions.*
    • +
  • spec.initContainers[*].securityContext.seLinuxOptions.*
    • +
+
/proc Mount Type +

/proc masks are not supported on Windows nodes.

+

Unsupported Fields

+
    +
  • spec.containers[*].securityContext.procMount
    • +
  • spec.initContainers[*].securityContext.procMount
    • +
+
Seccomp +

Seccomp is not supported by Windows nodes.

+

Unsupported Fields

+
    +
  • spec.securityContext.seccompProfile.*
    • +
  • spec.containers[*].securityContext.seccompProfile.*
    • +
  • spec.initContainers[*].securityContext.seccompProfile.*
    • +
+
Sysctls +

These are part of the Linux sysctl interface, which has no equivalent on Windows.

+

Unsupported Fields

+
    +
  • spec.securityContext.sysctls
    • +
+
+ +## FAQ + diff --git a/content/en/docs/concepts/workloads/pods/_index.md b/content/en/docs/concepts/workloads/pods/_index.md index 5dd6bac9de6be..3cb432b976a81 100644 --- a/content/en/docs/concepts/workloads/pods/_index.md +++ b/content/en/docs/concepts/workloads/pods/_index.md @@ -257,8 +257,9 @@ section. ## Privileged mode for containers -Any container in a Pod can enable privileged mode, using the `privileged` flag on the [security context](/docs/tasks/configure-pod-container/security-context/) of the container spec. This is useful for containers that want to use operating system administrative capabilities such as manipulating the network stack or accessing hardware devices. -Processes within a privileged container get almost the same privileges that are available to processes outside a container. +Any container in a Pod can enable privileged mode, using either the `privileged` (Linux) or `hostProcess` (Windows) flag on the [security context](/docs/tasks/configure-pod-container/security-context/) of the container spec. This is useful for containers that want to use operating system administrative capabilities such as manipulating the network stack or accessing hardware devices. + +Processes within a either a [Windows HostProcess](/docs/tasks/configure-pod-container/create-hostprocess-container/) or Linux privileged container get almost the same privileges that are available to processes outside a container. {{< note >}} Your {{< glossary_tooltip text="container runtime" term_id="container-runtime" >}} must support the concept of a privileged container for this setting to be relevant. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 6ebc960521e22..c2d9935fd2a73 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -448,6 +448,7 @@ different Kubernetes components. | `WindowsGMSA` | `false` | Alpha | 1.14 | 1.15 | | `WindowsGMSA` | `true` | Beta | 1.16 | 1.17 | | `WindowsGMSA` | `true` | GA | 1.18 | - | +| `WindowsHostProcessContainers` | `false` | Alpha | 1.22 | | `WindowsRunAsUserName` | `false` | Alpha | 1.16 | 1.16 | | `WindowsRunAsUserName` | `true` | Beta | 1.17 | 1.17 | | `WindowsRunAsUserName` | `true` | GA | 1.18 | - | @@ -954,6 +955,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `WinDSR`: Allows kube-proxy to create DSR loadbalancers for Windows. - `WinOverlay`: Allows kube-proxy to run in overlay mode for Windows. - `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. +- `WindowsHostProcessContainers`: Enables support for Windows HostProcess containers. - `WindowsRunAsUserName` : Enable support for running applications in Windows containers with as a non-default user. See [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) diff --git a/content/en/docs/tasks/configure-pod-container/create-hostprocess-container.md b/content/en/docs/tasks/configure-pod-container/create-hostprocess-container.md new file mode 100644 index 0000000000000..da799de7b4d47 --- /dev/null +++ b/content/en/docs/tasks/configure-pod-container/create-hostprocess-container.md @@ -0,0 +1,52 @@ +--- +title: Create a Windows HostProcess Container +content_type: task +weight: 20 +--- + + + +{{< feature-state for_k8s_version="v1.22" state="alpha" >}} + +Windows HostProcess containers enable you to run containerized administrative workloads on a Windows host. These containers operate as normal processes and have access to the host network namespace, storage, and devices when given the associated user privileges. HostProcess containers can be used to deploy networking policies, storage configurations, device plugins, kube-proxy, and other components to Windows nodes without the need for dedicated proxies or the direct installation of services on the Windows nodes. + +Administrative tasks such as installation of security patches, event log collection, and more can be performed without requiring cluster operations to log onto each Window node. HostProcess containers can run as any user that is available on the host or is in the domain of the host machine, allowing administrators to restrict resource access through user permissions. While neither filesystem or process isolation are supported, a new volume is created on the host upon starting the container to give it a clean and consolidated workspace. HostProcess containers can also be built on top of existing Windows base images and do not inherit the same [compatibility requirements](https://docs.microsoft.com/virtualization/windowscontainers/deploy-containers/version-compatibility) as Windows server containers. + +### When should I use a Windows HostProcess container? + +- When you need to perform tasks which require the networking namespace of the host. HostProcess containers have access to the host's network interfaces and IP. +- You need access to the host's filesystem +- Installation of specific devices drivers or services +- Consolidation of administrative tasks and security policies. This reduces the degree of privileges needed by Windows nodes. + +### Important Notes + +- HostProcess container will **only run via containerd**. +- As of v1.22 HostProcess containers cannot share the same pod as non-privileged containers. This is a current limitation of the Windows OS; non-privileged containers cannot share a vNIC with the host IP namespace. +- HostProcess containers run as a process on the host and do not have any degree of isolation other than resource constraints imposed on the HostProcess user account. Neither filesystem or Hyper-V isolation are supported for HostProcess containers. +- Resource limits (disk, memory, cpu count) are supported in the same fashion as processes on the host. +- Named pipe mounts will **not** be supported and should instead be accessed via their path on the host (\\.\pipe\*). + - Unix domain sockets, however, are supported +- Volume mounts are supported and are mounted under the container volume. +- [todo] Skipping host file mounting + + From bc9786a012a7688bb20bca8935f91c3a7db97c6e Mon Sep 17 00:00:00 2001 From: Brandon Smith Date: Thu, 22 Jul 2021 14:50:31 -0700 Subject: [PATCH 02/10] Incorporated initial round of feedback --- .../security/pod-security-standards.md | 2 +- .../security/windows-pod-security-profile.md | 81 +++++++++++++++---- .../en/docs/concepts/workloads/pods/_index.md | 4 +- .../create-hostprocess-container.md | 52 ------------ .../create-hostprocess-pod.md | 75 +++++++++++++++++ 5 files changed, 143 insertions(+), 71 deletions(-) delete mode 100644 content/en/docs/tasks/configure-pod-container/create-hostprocess-container.md create mode 100644 content/en/docs/tasks/configure-pod-container/create-hostprocess-pod.md diff --git a/content/en/docs/concepts/security/pod-security-standards.md b/content/en/docs/concepts/security/pod-security-standards.md index e31b7b51ac015..4b8d653165bd5 100644 --- a/content/en/docs/concepts/security/pod-security-standards.md +++ b/content/en/docs/concepts/security/pod-security-standards.md @@ -260,7 +260,7 @@ fail validation. Windows HostProcess -

Windows pods offer the ability to run HostProcess containers which enables privileged access to the Windows node. As is similar to privileged containers this must be disallowed in the baseline policy.

+

Windows pods offer the ability to run HostProcess containers which enables privileged access to the Windows node. As is similar to privileged containers this must be disallowed in the baseline policy.

Restricted Fields