Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit node access to API #279

Closed
saad-ali opened this issue Apr 26, 2017 · 51 comments
Closed

Limit node access to API #279

saad-ali opened this issue Apr 26, 2017 · 51 comments
Assignees
@liggitt @mikedanese
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/auth Categorizes an issue or PR as relevant to SIG Auth. stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status

Comments

@saad-ali
Copy link
Member

saad-ali commented Apr 26, 2017 •
edited by kacole2
Loading

Limit node access to API
@saad-ali saad-ali added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/node Categorizes an issue or PR as relevant to SIG Node. labels Apr 26, 2017
@saad-ali saad-ali added this to the v1.7 milestone Apr 26, 2017
@saad-ali saad-ali assigned liggitt, saad-ali and wojtek-t and unassigned wojtek-t Apr 26, 2017
@saad-ali saad-ali added sig/auth Categorizes an issue or PR as relevant to SIG Auth. and removed sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Apr 26, 2017
@liggitt
Copy link
Member

liggitt commented Apr 26, 2017

c.f. kubernetes/kubernetes#40476

@liggitt liggitt mentioned this issue Apr 26, 2017
Merged
@liggitt
Copy link
Member

liggitt commented Apr 26, 2017

c.f. kubernetes/community#585

@timstclair timstclair mentioned this issue May 2, 2017
Closed
@idvoretskyi idvoretskyi added the stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status label May 3, 2017
@liggitt liggitt mentioned this issue May 13, 2017
Merged
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue May 16, 2017
Merge pull request #45775 from liggitt/mirror-pod-validation
7cd32ab 
Automatic merge from submit-queue (batch tested with PRs 44337, 45775, 45832, 45574, 45758)

Tighten validation of mirror pod annotations

Tightens validation for pods with a mirror pod annotation:
1. spec.nodeName must be set
2. makes the mirror pod annotation immutable
3. starts validating pod-specific annotations during pod status update

None of these changes affect usage of the mirror pod annotation by kubelets, which only set it on pod creation (verified this is true back to 1.5.x)

the second commit updates the pod validation tests to look for specific error messages (best reviewed ignoring whitespace changes)

This is the validation portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and kubernetes/enhancements#279

```release-note
Mirror pods must now indicate the nodeName they are bound to on creation. The mirror pod annotation is now treated as immutable and cannot be added to an existing pod, removed from a pod, or modified.
```
This was referenced May 17, 2017
Merged
Merged
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue May 31, 2017
Merge pull request #46076 from liggitt/node-authorizer
5995690 
Automatic merge from submit-queue

Node authorizer

This PR implements the authorization portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and kubernetes/enhancements#279:
* Adds a new authorization mode (`Node`) that authorizes requests from nodes based on a graph of related pods,secrets,configmaps,pvcs, and pvs:
  * Watches pods, adds edges (secret -> pod, configmap -> pod, pvc -> pod, pod -> node)
  * Watches pvs, adds edges (secret -> pv, pv -> pvc)
  * When both Node and RBAC authorization modes are enabled, the default RBAC binding that grants the `system:node` role to the `system:nodes` group is not automatically created.
* Tightens the `NodeRestriction` admission plugin to require identifiable nodes for requests from users in the `system:nodes` group.

This authorization mode is intended to be used in combination with the `NodeRestriction` admission plugin, which limits the pods and nodes a node may modify. To enable in combination with RBAC authorization and the NodeRestriction admission plugin:
* start the API server with `--authorization-mode=Node,RBAC --admission-control=...,NodeRestriction,...`
* start kubelets with TLS boostrapping or with client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>`

```release-note
kube-apiserver: a new authorization mode (`--authorization-mode=Node`) authorizes nodes to access secrets, configmaps, persistent volume claims and persistent volumes related to their pods.
* Nodes must use client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>` in order to be authorized by the node authorizer (the credentials obtained by the kubelet via TLS bootstrapping satisfy these requirements)
* When used in combination with the `RBAC` authorization mode (`--authorization-mode=Node,RBAC`), the `system:node` role is no longer automatically granted to the `system:nodes` group.
```

```release-note
RBAC: the automatic binding of the `system:node` role to the `system:nodes` group is deprecated and will not be created in future releases. It is recommended that nodes be authorized using the new `Node` authorization mode instead. Installations that wish to continue giving all members of the `system:nodes` group the `system:node` role (which grants broad read access, including all secrets and configmaps) must create an installation-specific ClusterRoleBinding.
```

Follow-up:
- [ ] enable e2e CI environment with admission and authorizer enabled (blocked by kubelet TLS bootstrapping enablement in #40760)
- [ ] optionally enable this authorizer and admission plugin in kubeadm
- [ ] optionally enable this authorizer and admission plugin in kube-up
@liggitt liggitt added stage/beta Denotes an issue tracking an enhancement targeted for Beta status and removed stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status labels Jun 2, 2017
@liggitt liggitt changed the title Reduce scope of node access to API server Limit API access by nodes Jun 3, 2017
@liggitt liggitt changed the title Limit API access by nodes Limit node access to API Jun 3, 2017
@liggitt liggitt mentioned this issue Jun 10, 2017
Merged
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Jun 13, 2017
Merge pull request #46921 from liggitt/kubemark-node-auth
d81f71d 
Automatic merge from submit-queue (batch tested with PRs 46441, 43987, 46921, 46823, 47276)

Enable Node authorizer and NodeRestriction admission in kubemark

xref kubernetes/enhancements#279

We want to ensure scale testing covers use of the authorizer/admission pair that partitions nodes. This includes enabling the authorizer, which populates a graph of existing nodes and pods.

Kubemark is still running all nodes with a single credential, so a follow-up step is to generate unique credentials per node (or enable TLS bootstrapping) and remove the temporary rolebinding added in this PR so the node authorizer is the one authorizing each call by a hollow node.
@liggitt liggitt mentioned this issue Jun 14, 2017
Merged
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 16, 2020
@palnabarun
Copy link
Member

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 17, 2020
@liggitt liggitt mentioned this issue Apr 20, 2020
Merged
@liggitt
Copy link
Member

liggitt commented Apr 20, 2020

1.19-targeted work: kubernetes/kubernetes#90307

@kikisdeliveryservice
Copy link
Member

Hi @saad-ali @liggitt ,

1.19 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating in 1.19?

In order to have this part of the release:

The KEP PR must be merged in an implementable state
The KEP must have test plans
The KEP must have graduation criteria.

The current release schedule is:

Monday, April 13: Week 1 - Release cycle begins
Tuesday, May 19: Week 6 - Enhancements Freeze
Thursday, June 25: Week 11 - Code Freeze
Thursday, July 9: Week 14 - Docs must be completed and reviewed
Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

Please let me know and I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

@liggitt
Copy link
Member

liggitt commented May 1, 2020

Yes, this is completed in 1.19. Relevant PRs:

@kikisdeliveryservice
Copy link
Member

@liggitt just to clarify will this be GA in 1.19?

@kikisdeliveryservice kikisdeliveryservice added tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team and removed tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team labels May 4, 2020
@kikisdeliveryservice
Copy link
Member

/milestone v1.19

@mikedanese
Copy link
Member

This didn't follow alpha, beta, GA cycle, but basically it's GA now.

@kikisdeliveryservice kikisdeliveryservice added the stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status label May 4, 2020
@kikisdeliveryservice
Copy link
Member

@mikedanese @liggitt

I notice that the underlying KEP does not have a Test Plan, can you please update the KEP to have one by the Enhancements Freeze (Tuesday May 19th EOD PST) as this is required for inclusion in 1.19 release.

As an additional note, #1620 merged recently, adding production readiness review questions to the KEP template. We are not making it mandatory for the 1.19 release cycle, but it would be great if the PRR questionnaire is filled since the KEP needs to be updated to include the test plan. You can find the template at https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template

If you have any questions please let me know.

Thanks!

@kikisdeliveryservice
Copy link
Member

As this predates the current KEP format and the linked issues do have tests, we will be marking this as tracked.

@tallclair
Copy link
Member

There is partial e2e coverage for this feature, but the tests are currently broken and not run by any of our test jobs, and don't cover the label or taint restrictions. See kubernetes/kubernetes#82551.

@liggitt
Copy link
Member

liggitt commented May 18, 2020

The unit and integration tests exercise all the API restrictions implemented by the feature. This feature does not lend itself to e2e test environments which do not control the kube-apiserver configuration (so cannot guarantee the feature is in use) and cannot easily create false Node objects in cloud environments where the node controller detects and deletes the dummy Node object.

@tallclair
Copy link
Member

Do you think we should just delete the e2e tests?

@liggitt
Copy link
Member

liggitt commented May 19, 2020

It's unclear to me what the goal of the e2e tests are currently. To verify functionality, integration tests seem much better suited to me. Are they trying to validate that the feature is enabled?

@annajung
Copy link
Contributor

Hi @liggitt 👋 1.19 docs shadow here! Does this enhancement work planned for 1.19 require new or modification to docs?

Friendly reminder that if new/modification to docs are required, a placeholder PR against k/website (branch dev-1.19) are needed by Friday, June 12.

@liggitt
Copy link
Member

liggitt commented May 21, 2020

No docs updates are required, https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#noderestriction already describes the user-facing aspects of the feature.

@annajung
Copy link
Contributor

Thank you! I will update the tracking sheet accordingly

@kikisdeliveryservice
Copy link
Member

Hi @liggitt !

To follow-up on the email sent to k-dev today, I wanted to let you know that Code Freeze has been extended to Thursday, July 9th. You can see the revised schedule here: https://github.com/kubernetes/sig-release/tree/master/releases/release-1.19

We expect all PRs to be merged by that time. Please let me know if you have any questions. 😄

Best,
Kirsten

@mikedanese mikedanese removed the stage/beta Denotes an issue tracking an enhancement targeted for Beta status label Sep 3, 2020
@mikedanese
Copy link
Member

This is done.

@kikisdeliveryservice kikisdeliveryservice removed the tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team label Sep 12, 2020
@kikisdeliveryservice kikisdeliveryservice removed this from the v1.19 milestone Sep 12, 2020
@ehashman ehashman mentioned this issue Jun 24, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@liggitt liggitt

@mikedanese mikedanese

Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/auth Categorizes an issue or PR as relevant to SIG Auth. stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests