Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add possibility to disable kube-proxy #2088

Merged
merged 2 commits into from
Mar 4, 2021
Merged

Add possibility to disable kube-proxy #2088

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6cfdd79
Add possibility to disable kube-proxy
brb Feb 24, 2021
abf42e2
config: Change kube-proxy Mode postfix to ProxyMode
brb Feb 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions pkg/cluster/internal/create/actions/kubeadminit/init.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,19 @@ import (
"sigs.k8s.io/kind/pkg/cluster/nodeutils"

"sigs.k8s.io/kind/pkg/cluster/internal/create/actions"
"sigs.k8s.io/kind/pkg/internal/apis/config"
)

// kubeadmInitAction implements action for executing the kubeadm init
// and a set of default post init operations like e.g. install the
// CNI network plugin.
type action struct{}
type action struct {
skipKubeProxy bool
}

// NewAction returns a new action for kubeadm init
func NewAction() actions.Action {
return &action{}
func NewAction(cfg *config.Cluster) actions.Action {
return &action{skipKubeProxy: cfg.Networking.KubeProxyMode == config.NoneMode}
}

// Execute runs the action
Expand All @@ -56,13 +59,18 @@ func (a *action) Execute(ctx *actions.ActionContext) error {
return err
}

// skip preflight checks, as these have undesirable side effects
// and don't tell us much. requires kubeadm 1.13+
skipPhases := "preflight"
if a.skipKubeProxy {
skipPhases += ",addon/kube-proxy"
}

// run kubeadm
cmd := node.Command(
// init because this is the control plane node
"kubeadm", "init",
// skip preflight checks, as these have undesirable side effects
// and don't tell us much. requires kubeadm 1.13+
"--skip-phases=preflight",
"--skip-phases="+skipPhases,
// specify our generated config file
"--config=/kind/kubeadm.conf",
"--skip-token-print",
Expand Down
2 changes: 1 addition & 1 deletion pkg/cluster/internal/create/create.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ func Cluster(logger log.Logger, p providers.Provider, opts *ClusterOptions) erro
}
if !opts.StopBeforeSettingUpKubernetes {
actionsToRun = append(actionsToRun,
kubeadminit.NewAction(), // run kubeadm init
kubeadminit.NewAction(opts.Config), // run kubeadm init
)
// this step might be skipped, but is next after init
if !opts.Config.Networking.DisableDefaultCNI {
Expand Down
4 changes: 4 additions & 0 deletions pkg/cluster/internal/kubeadm/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,7 @@ evictionHard:
{{ range $key := .SortedFeatureGateKeys }}
"{{ $key }}": {{$.FeatureGates $key }}
{{end}}{{end}}
{{if ne .KubeProxyMode "None"}}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
Expand All @@ -252,6 +253,7 @@ mode: "{{ .KubeProxyMode }}"
{{end}}{{end}}
iptables:
minSyncPeriod: 1s
{{end}}
`

// ConfigTemplateBetaV2 is the kubeadm config template for API version v1beta2
Expand Down Expand Up @@ -360,6 +362,7 @@ evictionHard:
{{ range $key := .SortedFeatureGateKeys }}
"{{ $key }}": {{ index $.FeatureGates $key }}
{{end}}{{end}}
{{if ne .KubeProxyMode "None"}}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
Expand All @@ -372,6 +375,7 @@ mode: "{{ .KubeProxyMode }}"
{{end}}{{end}}
iptables:
minSyncPeriod: 1s
{{end}}
`

// Config returns a kubeadm config generated from config data, in particular
Expand Down
2 changes: 2 additions & 0 deletions pkg/internal/apis/config/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ const (
IPTablesMode ProxyMode = "iptables"
// IPVSMode sets ProxyMode to iptables
IPVSMode ProxyMode = "ipvs"
// NoneMode disables kube-proxy
NoneMode ProxyMode = "none"
Copy link
Member

@neolit123 neolit123 Feb 24, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the alternative value of this field?
if "" implies kube-proxy mode this should be reflected in a comment above.

for the API change, as is, this could be useful if one day kind decides to implement it's own proxy ALA Cilium.
adding another ProxyMode.

but if there are no plans for that and if the demand to skip kube-proxy is not high, the better alternative would have been to configure this via kubeadm's v1beta3 ClusterConfiguration.

of course, a couple of problems with that are:

  • kubeadm v1beta3 is not ready yet, but this exact skip phases feature is tracked here
  • it would only work with kubeadm / k8s latest

a more flexible feature in kind would be to allow passing extra args to the kubeadm init/join commands the same way kubeadm allows passing such to the kubelet / control-plane. as per how cobra, go's CLI works, subsequent flags with the same key would override prior ones.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the alternative value of this field?
if "" implies kube-proxy mode this should be reflected in a comment above.

"" implies iptables mode.

but if there are no plans for that and if the demand to skip kube-proxy is not high, the better alternative would have been to configure this via kubeadm's v1beta3 ClusterConfiguration.

Once it's ready, we can change the way we disable kube-proxy.

but if there are no plans for that and if the demand to skip kube-proxy is not high, the better alternative would have been to configure this via kubeadm's v1beta3 ClusterConfiguration.

If we need to pass more flags to kubeadm, then it makes sense. At the moment, it's only one flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the API of this field I think it should be

  • "iptables" : default value
  • "ipvs"
  • "none"

if there are other proxies they should provide their installation, same we are doing with the cni

a more flexible feature in kind would be to allow passing extra args to the kubeadm init/join commands the same way kubeadm allows passing such to the kubelet / control-plane. as per how cobra, go's CLI works, subsequent flags with the same key would override prior ones.

I think we already discussed this in another issue 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a more flexible feature in kind would be to allow passing extra args to the kubeadm init/join commands the same way kubeadm allows passing such to the kubelet / control-plane. as per how cobra, go's CLI works, subsequent flags with the same key would override prior ones.

kubelet etc though are moving away from this in favor of config, which also seems to be the case for kubeadm (phases in v1beta3), I don't think we should promote reliance on dependency flags versus structured configuration.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flags are indeed the riskier option.

)

// PatchJSON6902 represents an inline kustomize json 6902 patch
Expand Down
3 changes: 2 additions & 1 deletion pkg/internal/apis/config/validate.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ func (c *Cluster) Validate() error {
}

// KubeProxyMode should be iptables or ipvs
if c.Networking.KubeProxyMode != IPTablesMode && c.Networking.KubeProxyMode != IPVSMode {
if c.Networking.KubeProxyMode != IPTablesMode && c.Networking.KubeProxyMode != IPVSMode &&
c.Networking.KubeProxyMode != NoneMode {
errs = append(errs, errors.Errorf("invalid kubeProxyMode: %s", c.Networking.KubeProxyMode))
}

Expand Down
4 changes: 3 additions & 1 deletion site/content/docs/user/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,8 @@ networking:
kubeProxyMode: "ipvs"
{{< /codeFromInline >}}

To disable kube-proxy, set the mode to `"none"`.

### Nodes
The `kind: Cluster` object has a `nodes` field containing a list of `node`
objects. If unset this defaults to:
Expand Down Expand Up @@ -356,4 +358,4 @@ nodes:
{{< /codeFromInline >}}

[YAML]: https://yaml.org/
[feature gates]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
[feature gates]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/