-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document how to configure a second scheduler to run as static Pod(s) #22802
Comments
/retitle Document how to configure a second scheduler to run as static Pod(s) |
/sig scheduling |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
This is a Bug Report
Problem:
This page shows how to implement the solution (a second scheduler) with a deployment. I'd think that since the existing scheduler is by default a static pod, wouldn't it make sense to at least also document the process for a static pod?
Proposed Solution:
Update the documentation for changes needed for a static pod.
Page to Update:
https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/
1.18.x
Below is what I've posted at https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/learn/lecture/14298694#questions/11881944, where I tried to adapt these instructions for a static pod, but am having issues.
I’ve been trying to implement multiple schedulers using the official Kubernetes documentation at https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/; following the steps in that link work fine. However, when I try to adapt it for a static pod (like in the Kodekloud lab; I think it makes a lot more sense to do this as a static pod and not a deployment), I run into problems.
I seem to get stuck when trying to use the service account, which is needed to get information on the endpoints, i.e. I think the nodes, as part of the leader election. Note that I realize I’m using leader-elect=true, which is different from the lab. Not only is setting it to false a single point of failure, but the Kubernetes documentation says to do the opposite, i.e. it starts out with it false, but then says to change it. Since I think the CKA exam will be tested based on the official documentation, I’m trying to get this to work. I’ve documented my steps here. Please review this and think about what I’m missing. I’ve listed the specific questions I have that I’d like answers to at the end of this post.
ASSUMPTIONS
My assumptions regarding the content in https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/ and adapting it for a static pod:
The section under ‘Package the scheduler’ can be skipped if I want to use the same image as my current default scheduler; I just use the current image and not a new one.
Setting up the cluster role, cluster role binding and service account are the same.
Path of kube-scheduler as well as image should mirror what’s in manifest of currently running yaml
=================================================================
ATTEMPT #1: BASELINE - STEPS IN K8S DOCUMENTATION WORK
So, I first try going through the steps in https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/, with adaptations for assumption #3 above. I open https://kodekloud.com/courses/certified-kubernetes-administrator-with-practice-tests-labs/lectures/12038844 and the associated quiz tab.
I copy the yaml from the URL, with the changes for the path of kube-scheduler and image:
I create the elements in the yaml, which seem to run fine:
I update the nginx yaml in the lab and run it.
The pod seems to have been created OK, and assigned using the alternate my-scheduler scheduler.
This gives a baseline, showing that the steps on the Kubernetes page works fine. Note that in this case, we have leader-elect set to true, not false.
==========================================================
ATTEMPT #2: CHANGE STEPS TO STATIC POD - PROBLEM WITH SERVICEACCOUNT
Here, I try to adapt the previous steps to use a static pod and not a deployment. I open (i.e. refresh the page to start over) https://kodekloud.com/courses/certified-kubernetes-administrator-with-practice-tests-labs/lectures/12038844 and the associated quiz tab.
The following is what’s in acct-setup.yml. It’s the first part of the yaml in the Kubernetes documentation, with the yaml for the deployment removed, since we want to deploy a static POD and not a deployment. I’d think these other steps for the service account, cluster role, and cluster role bindings should be the same.
I also edit the cluster role as specified.
I remove the serviceAccountName entry:
And I see it’s restarted, but with an error.
=============================================
ATTEMPT #3: EVEN WITH A WORKING CONFIGURATION, IT’S DIFFERENT
Specifically, it’s different from what’s in the lab as well as in the Kubernetes documentation.
That said, as an exercise in finding something that will actually work, I did change the port (and a couple other settings) and came up with the following yaml, which worked for me, i.e. it deployed a second scheduler as a static pod that I was used to deploy the nginx pod as specified in the quiz/lab. I’ve given an sdiff of my working yaml and what’s in the answer key for the lab of https://kodekloud.com/courses/certified-kubernetes-administrator-with-practice-tests-labs/lectures/12038844.
Some key differences in the output below:
address is set to 0.0.0.0 and not 127.0.0.1 (this is done in favor of the official documentation; see URL I mentioned above)
I have —port and —secure-port defined differently; I think the way it’s defined in the answer key allows sensitive information to be passed in clear text, which is a huge security vulnerability and not something to be done in production. Likewise, I’ve removed scheme; I think Kubernetes just figures this out to be https as a result.
I have updated the image name to be different; I think the one in the answer key is old.
That said, again, this implementation of multiple schedulers is not what’s in the official Kubernetes documentation, and so not what I would think would be on the CKA exam.
=================================
MY QUESTIONS
Why isn’t the static pod based on the YAML from the kubernetes documentation not starting?
Why is it running into a problem related to the service account?
With the following specific deviations between the lab material and the official Kubernetes website, are there plans to update the course? Or am I missing something? I'd say 3.3 and 3.4 are especially significant.
3.1. command options for container
3.2. image for container
3.3. ports used, for liveness probe and otherwise
3.4. setting leader-elect to true and not false
In summary, while the lab and the Kubernetes documentation are similar at a high level, they are significantly different in at least two areas (a and b below). I have 2 implementations/configurations (attempts 1 and 2 above) that don’t match what’s in the multiple schedulers lab; the most critical differences are:
a) use of insecure HTTP and not HTTPS
b) setting leader-elect to false and not true
The text was updated successfully, but these errors were encountered: