forked from openshift/machine-config-operator
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
baremetal: Add coredns, mdns-publisher, haproxy, keepalived static pods
Adds pods to master and worker nodes as appropriate Updates haproxy container to use openshift/router-haproxy image instead of docker.io/library/haproxy Also adds liveness tests for the coredns,mdns-publisher, haproxy and keepalived static pods, changes worker node /etc/resolv.conf to point to node's IP instead of 127.0.0.1 and fix the bug generating haproxy cfg file Due to the fact that both use the same image, there was a bit of confusion here. We want keepalived to track OCP Router 1936 and we want API LB Haproxy pod to have health checked at 50936, which is where we configure haproxy to expose health at. Depends-On: openshift#943 Depends-On: openshift#984
- Loading branch information
Showing
18 changed files
with
784 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7 changes: 7 additions & 0 deletions
7
templates/common/baremetal/files/baremetal-NetworkManager-kni-conf.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| filesystem: "root" | ||
| mode: 0644 | ||
| path: "/etc/NetworkManager/conf.d/99-kni.conf" | ||
| contents: | ||
| inline: | | ||
| [main] | ||
| dhcp=dhclient |
17 changes: 17 additions & 0 deletions
17
templates/common/baremetal/files/baremetal-coredns-corefile.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| filesystem: "root" | ||
| mode: 0644 | ||
| path: "/etc/kubernetes/static-pod-resources/coredns/Corefile.tmpl" | ||
| contents: | ||
| inline: | | ||
| . { | ||
| errors | ||
| health | ||
| mdns {{ .EtcdDiscoveryDomain }} 0 {{`{{.Cluster.Name}}`}} | ||
| forward . {{`{{- range $upstream := .DNSUpstreams}} {{$upstream}}{{- end}}`}} | ||
| cache 30 | ||
| reload | ||
| hosts /etc/coredns/api-int.hosts {{ .EtcdDiscoveryDomain }} { | ||
| {{ .Infra.Status.PlatformStatus.BareMetal.APIServerInternalIP }} api-int.{{ .EtcdDiscoveryDomain }} | ||
| fallthrough | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| filesystem: "root" | ||
| mode: 0644 | ||
| path: "/etc/kubernetes/manifests/coredns.yaml" | ||
| contents: | ||
| inline: | | ||
| kind: Pod | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: coredns | ||
| namespace: openshift-kni-infra | ||
| creationTimestamp: | ||
| deletionGracePeriodSeconds: 65 | ||
| labels: | ||
| app: kni-infra-mdns | ||
| spec: | ||
| volumes: | ||
| - name: resource-dir | ||
| hostPath: | ||
| path: "/etc/kubernetes/static-pod-resources/coredns" | ||
| - name: kubeconfig | ||
| hostPath: | ||
| path: "/etc/kubernetes/kubeconfig" | ||
| - name: conf-dir | ||
| hostPath: | ||
| path: "/etc/coredns" | ||
| initContainers: | ||
| - name: render-config | ||
| image: {{ .Images.baremetalRuntimeCfgImage }} | ||
| command: | ||
| - runtimecfg | ||
| - render | ||
| - "/etc/kubernetes/kubeconfig" | ||
| - "--api-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.APIServerInternalIP }}" | ||
| - "--dns-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.NodeDNSIP }}" | ||
| - "--ingress-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.IngressIP }}" | ||
| - "/config" | ||
| - "--out-dir" | ||
| - "/etc/coredns" | ||
| resources: {} | ||
| volumeMounts: | ||
| - name: kubeconfig | ||
| mountPath: "/etc/kubernetes/kubeconfig" | ||
| - name: resource-dir | ||
| mountPath: "/config" | ||
| - name: conf-dir | ||
| mountPath: "/etc/coredns" | ||
| imagePullPolicy: IfNotPresent | ||
| containers: | ||
| - name: coredns | ||
| securityContext: | ||
| privileged: true | ||
| image: {{.Images.corednsImage}} | ||
| args: | ||
| - "--conf" | ||
| - "/etc/coredns/Corefile" | ||
| resources: | ||
| requests: | ||
| cpu: 150m | ||
| memory: 1Gi | ||
| volumeMounts: | ||
| - name: conf-dir | ||
| mountPath: "/etc/coredns" | ||
| readinessProbe: | ||
| httpGet: | ||
| path: /health | ||
| port: 8080 | ||
| scheme: HTTP | ||
| initialDelaySeconds: 10 | ||
| periodSeconds: 10 | ||
| successThreshold: 1 | ||
| failureThreshold: 3 | ||
| timeoutSeconds: 10 | ||
| livenessProbe: | ||
| httpGet: | ||
| path: /health | ||
| port: 8080 | ||
| scheme: HTTP | ||
| initialDelaySeconds: 60 | ||
| timeoutSeconds: 5 | ||
| successThreshold: 1 | ||
| failureThreshold: 5 | ||
| terminationMessagePolicy: FallbackToLogsOnError | ||
| imagePullPolicy: IfNotPresent | ||
| hostNetwork: true | ||
| tolerations: | ||
| - operator: Exists | ||
| priorityClassName: system-node-critical | ||
| status: {} |
84 changes: 84 additions & 0 deletions
84
templates/common/baremetal/files/baremetal-keepalived.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| filesystem: "root" | ||
| mode: 0644 | ||
| path: "/etc/kubernetes/manifests/keepalived.yaml" | ||
| contents: | ||
| inline: | | ||
| kind: Pod | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: keepalived | ||
| namespace: openshift-kni-infra | ||
| creationTimestamp: | ||
| deletionGracePeriodSeconds: 65 | ||
| labels: | ||
| app: kni-infra-vrrp | ||
| spec: | ||
| volumes: | ||
| - name: resource-dir | ||
| hostPath: | ||
| path: "/etc/kubernetes/static-pod-resources/keepalived" | ||
| - name: kubeconfig | ||
| hostPath: | ||
| path: "/etc/kubernetes/kubeconfig" | ||
| - name: conf-dir | ||
| hostPath: | ||
| path: "/etc/keepalived" | ||
| initContainers: | ||
| - name: render-config | ||
| image: {{ .Images.baremetalRuntimeCfgImage }} | ||
| command: | ||
| - runtimecfg | ||
| - render | ||
| - "/etc/kubernetes/kubeconfig" | ||
| - "--api-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.APIServerInternalIP }}" | ||
| - "--dns-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.NodeDNSIP }}" | ||
| - "--ingress-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.IngressIP }}" | ||
| - "/config" | ||
| - "--out-dir" | ||
| - "/etc/keepalived" | ||
| resources: {} | ||
| volumeMounts: | ||
| - name: resource-dir | ||
| mountPath: "/config" | ||
| - name: kubeconfig | ||
| mountPath: "/etc/kubernetes/kubeconfig" | ||
| - name: conf-dir | ||
| mountPath: "/etc/keepalived" | ||
| imagePullPolicy: IfNotPresent | ||
| containers: | ||
| - name: keepalived | ||
| securityContext: | ||
| privileged: true | ||
| image: {{.Images.keepalivedImage}} | ||
| command: | ||
| - /usr/sbin/keepalived | ||
| args: | ||
| - "-f" | ||
| - "/etc/keepalived/keepalived.conf" | ||
| - "--dont-fork" | ||
| - "--vrrp" | ||
| - "--log-detail" | ||
| - "--log-console" | ||
| resources: | ||
| requests: | ||
| cpu: 150m | ||
| memory: 1Gi | ||
| volumeMounts: | ||
| - name: conf-dir | ||
| mountPath: "/etc/keepalived" | ||
| livenessProbe: | ||
| exec: | ||
| command: | ||
| - pgrep | ||
| - keepalived | ||
| initialDelaySeconds: 10 | ||
| terminationMessagePolicy: FallbackToLogsOnError | ||
| imagePullPolicy: IfNotPresent | ||
| hostNetwork: true | ||
| tolerations: | ||
| - operator: Exists | ||
| priorityClassName: system-node-critical | ||
| status: {} |
75 changes: 75 additions & 0 deletions
75
templates/common/baremetal/files/baremetal-mdns-publisher.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| filesystem: "root" | ||
| mode: 0644 | ||
| path: "/etc/kubernetes/manifests/mdns-publisher.yaml" | ||
| contents: | ||
| inline: | | ||
| kind: Pod | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: mdns-publisher | ||
| namespace: openshift-kni-infra | ||
| creationTimestamp: | ||
| deletionGracePeriodSeconds: 65 | ||
| labels: | ||
| app: kni-infra-mdns | ||
| spec: | ||
| volumes: | ||
| - name: resource-dir | ||
| hostPath: | ||
| path: "/etc/kubernetes/static-pod-resources/mdns" | ||
| - name: kubeconfig | ||
| hostPath: | ||
| path: "/etc/kubernetes/kubeconfig" | ||
| - name: conf-dir | ||
| hostPath: | ||
| path: "/etc/mdns" | ||
| initContainers: | ||
| - name: render-config | ||
| image: {{ .Images.baremetalRuntimeCfgImage }} | ||
| command: | ||
| - runtimecfg | ||
| - render | ||
| - "/etc/kubernetes/kubeconfig" | ||
| - "--api-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.APIServerInternalIP }}" | ||
| - "--dns-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.NodeDNSIP }}" | ||
| - "--ingress-vip" | ||
| - "{{ .Infra.Status.PlatformStatus.BareMetal.IngressIP }}" | ||
| - "/config" | ||
| - "--out-dir" | ||
| - "/etc/mdns" | ||
| resources: {} | ||
| volumeMounts: | ||
| - name: kubeconfig | ||
| mountPath: "/etc/kubernetes/kubeconfig" | ||
| - name: resource-dir | ||
| mountPath: "/config" | ||
| - name: conf-dir | ||
| mountPath: "/etc/mdns" | ||
| imagePullPolicy: IfNotPresent | ||
| containers: | ||
| - name: mdns-publisher | ||
| image: {{.Images.mdnsPublisherImage}} | ||
| args: | ||
| - "--debug" | ||
| resources: | ||
| requests: | ||
| cpu: 150m | ||
| memory: 1Gi | ||
| volumeMounts: | ||
| - name: conf-dir | ||
| mountPath: "/etc/mdns" | ||
| livenessProbe: | ||
| exec: | ||
| command: | ||
| - pgrep | ||
| - mdns-publisher | ||
| initialDelaySeconds: 10 | ||
| terminationMessagePolicy: FallbackToLogsOnError | ||
| imagePullPolicy: IfNotPresent | ||
| hostNetwork: true | ||
| tolerations: | ||
| - operator: Exists | ||
| priorityClassName: system-node-critical | ||
| status: {} |
39 changes: 39 additions & 0 deletions
39
templates/master/00-master/baremetal/files/baremetal-haproxy-haproxy.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| filesystem: "root" | ||
| mode: 0644 | ||
| path: "/etc/kubernetes/static-pod-resources/haproxy/haproxy.cfg.tmpl" | ||
| contents: | ||
| inline: | | ||
| defaults | ||
| mode tcp | ||
| log global | ||
| option dontlognull | ||
| retries 3 | ||
| timeout http-request 10s | ||
| timeout queue 1m | ||
| timeout connect 10s | ||
| timeout client 86400s | ||
| timeout server 86400s | ||
| timeout tunnel 86400s | ||
| frontend main | ||
| bind :{{`{{ .LBConfig.LbPort }}`}} | ||
| default_backend masters | ||
| listen health_check_http_url | ||
| bind :50936 | ||
| mode http | ||
| monitor-uri /healthz | ||
| option dontlognull | ||
| listen stats | ||
| bind 127.0.0.1:{{`{{ .LBConfig.StatPort }}`}} | ||
| mode http | ||
| stats enable | ||
| stats hide-version | ||
| stats uri /haproxy_stats | ||
| stats refresh 30s | ||
| stats auth Username:Password | ||
| backend masters | ||
| option httpchk GET /healthz HTTP/1.0 | ||
| option log-health-checks | ||
| balance roundrobin | ||
| {{`{{- range .LBConfig.Backends }} | ||
| server {{ .Host }} {{ .Address }}:{{ .Port }} weight 1 verify none check check-ssl inter 3s fall 3 rise 3 | ||
| {{- end }}`}} |
Oops, something went wrong.