Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cni/cilium] Add support for additional config options #11678

Merged
merged 1 commit into from
Jun 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 56 additions & 5 deletions k8s/crds/kops.k8s.io_clusters.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3427,6 +3427,31 @@ spec:
description: 'BPFCTGlobalTCPMax is the maximum number of entries
in the TCP CT table. Default: 524288'
type: integer
bpfLBAlgorithm:
description: 'BPFLBAlgorithm is the load balancing algorithm
("random", "maglev"). Default: random'
type: string
bpfLBMaglevTableSize:
description: 'BPFLBMaglevTableSize is the per service backend
table size when going with Maglev (parameter M). Default:
16381'
type: string
bpfLBMapMax:
description: 'BPFLBMapMax is the maximum number of entries
in bpf lb service, backend and affinity maps. Default: 65536'
type: integer
bpfNATGlobalMax:
description: 'BPFNATGlobalMax is the the maximum number of
entries in the BPF NAT table. Default: 524288'
type: integer
bpfNeighGlobalMax:
description: 'BPFNeighGlobalMax is the the maximum number
of entries in the BPF Neighbor table. Default: 524288'
type: integer
bpfPolicyMapMax:
description: 'BPFPolicyMapMax is the maximum number of entries
in endpoint policy map. Default: 16384'
type: integer
bpfRoot:
description: BPFRoot is not implemented and may be removed
in the future. Setting this has no effect.
Expand Down Expand Up @@ -3484,6 +3509,10 @@ spec:
description: DisableConntrack is not implemented and may be
removed in the future. Setting this has no effect.
type: boolean
disableEndpointCRD:
description: 'DisableEndpointCRD disables usage of CiliumEndpoint
CRD. Default: false'
type: boolean
disableIpv4:
description: 'DisableIpv4 is deprecated: Use EnableIpv4 instead.
Setting this flag has no effect.'
Expand All @@ -3496,16 +3525,29 @@ spec:
description: DisableMasquerade disables masquerading traffic
to external destinations behind the node IP.
type: boolean
enableBPFMasquerade:
description: 'EnableBPFMasquerade enables masquerading packets
from endpoints leaving the host with BPF instead of iptables.
Default: false'
type: boolean
enableEncryption:
description: 'EnableEncryption enables Cilium Encryption.
Default: false'
type: boolean
enableEndpointHealthChecking:
description: 'EnableEndpointHealthChecking enables connectivity
health checking between virtual endpoints. Default: true'
type: boolean
enableHostReachableServices:
description: 'EnableHostReachableServices configures Cilium
to enable services to be reached from the host namespace
in addition to pod namespaces. https://docs.cilium.io/en/v1.9/gettingstarted/host-services/
Default: false'
type: boolean
enableL7Proxy:
description: 'EnableL7Proxy enables L7 proxy for L7 policy
enforcement. Default: true'
type: boolean
enableNodePort:
description: 'EnableNodePort replaces kube-proxy with Cilium''s
BPF implementation. Requires spec.kubeProxy.enabled be set
Expand Down Expand Up @@ -3553,8 +3595,8 @@ spec:
agent.
properties:
enabled:
description: Enabled specifies whether Hubble is enabled
on the agent.
description: Enabled decides if Hubble is enabled on the
agent or not
type: boolean
metrics:
description: Metrics is a list of metrics to collect.
Expand All @@ -3563,14 +3605,23 @@ spec:
type: string
type: array
type: object
identityAllocationMode:
description: 'IdentityAllocationMode specifies in which backend
identities are stored ("crd", "kvstore"). Default: crd'
type: string
identityChangeGracePeriod:
description: 'IdentityChangeGracePeriod specifies the duration
to wait before using a changed identity. Default: 5s'
type: string
ipam:
description: Ipam specifies the IP address allocation mode
to use. Possible values are "crd" and "eni". "eni" will
use AWS native networking for pods. Eni requires masquerade
to be set to false. "crd" will use CRDs for controlling
IP address management. "hostscope" will use hostscope IPAM
mode. "kubernetes" will use addersing based on node pod
CIDR. Empty value will use host-scope address management.
CIDR. Empty value will use hostscope for cilum <= 1.7 and
"kubernetes" otherwise.
type: string
ipv4ClusterCidrMaskSize:
description: Ipv4ClusterCIDRMaskSize is not implemented and
Expand Down Expand Up @@ -3760,7 +3811,7 @@ spec:
classic:
description: ClassicNetworkingSpec is the specification of classic
networking mode, integrated into kubernetes. Support been removed
since kubernetes 1.4.
since Kubernetes 1.4.
type: object
cni:
description: CNINetworkingSpec is the specification for networking
Expand Down Expand Up @@ -3811,7 +3862,7 @@ spec:
type: object
lyftvpc:
description: LyftVPCNetworkingSpec declares that we want to use
the cni-ipvlan-vpc-k8s CNI networking
the cni-ipvlan-vpc-k8s CNI networking.
properties:
subnetTags:
additionalProperties:
Expand Down
36 changes: 36 additions & 0 deletions pkg/apis/kops/networking.go
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,9 @@ type CiliumNetworkingSpec struct {
// DisableConntrack is not implemented and may be removed in the future.
// Setting this has no effect.
DisableConntrack bool `json:"disableConntrack,omitempty"`
// DisableEndpointCRD disables usage of CiliumEndpoint CRD.
// Default: false
DisableEndpointCRD bool `json:"disableEndpointCRD,omitempty"`
// DisableIpv4 is deprecated: Use EnableIpv4 instead.
// Setting this flag has no effect.
DisableIpv4 bool `json:"disableIpv4,omitempty"`
Expand All @@ -315,6 +318,15 @@ type CiliumNetworkingSpec struct {
// "never": Cilium allows all traffic regardless of policies in place.
// If unspecified, "default" policy mode will be used.
EnablePolicy string `json:"enablePolicy,omitempty"`
// EnableL7Proxy enables L7 proxy for L7 policy enforcement.
// Default: true
EnableL7Proxy *bool `json:"enableL7Proxy,omitempty"`
// EnableBPFMasquerade enables masquerading packets from endpoints leaving the host with BPF instead of iptables.
// Default: false
EnableBPFMasquerade *bool `json:"enableBPFMasquerade,omitempty"`
// EnableEndpointHealthChecking enables connectivity health checking between virtual endpoints.
// Default: true
EnableEndpointHealthChecking *bool `json:"enableEndpointHealthChecking,omitempty"`
// EnableTracing is not implemented and may be removed in the future.
// Setting this has no effect.
EnableTracing bool `json:"enableTracing,omitempty"`
Expand All @@ -326,6 +338,12 @@ type CiliumNetworkingSpec struct {
// EnvoyLog is not implemented and may be removed in the future.
// Setting this has no effect.
EnvoyLog string `json:"envoyLog,omitempty"`
// IdentityAllocationMode specifies in which backend identities are stored ("crd", "kvstore").
// Default: crd
IdentityAllocationMode string `json:"identityAllocationMode,omitempty"`
// IdentityChangeGracePeriod specifies the duration to wait before using a changed identity.
// Default: 5s
IdentityChangeGracePeriod string `json:"identityChangeGracePeriod,omitempty"`
// Ipv4ClusterCIDRMaskSize is not implemented and may be removed in the future.
// Setting this has no effect.
Ipv4ClusterCIDRMaskSize int `json:"ipv4ClusterCidrMaskSize,omitempty"`
Expand Down Expand Up @@ -436,6 +454,24 @@ type CiliumNetworkingSpec struct {
// BPFCTGlobalAnyMax is the maximum number of entries in the non-TCP CT table.
// Default: 262144
BPFCTGlobalAnyMax int `json:"bpfCTGlobalAnyMax,omitempty"`
// BPFLBAlgorithm is the load balancing algorithm ("random", "maglev").
// Default: random
BPFLBAlgorithm string `json:"bpfLBAlgorithm,omitempty"`
// BPFLBMaglevTableSize is the per service backend table size when going with Maglev (parameter M).
// Default: 16381
BPFLBMaglevTableSize string `json:"bpfLBMaglevTableSize,omitempty"`
// BPFNATGlobalMax is the the maximum number of entries in the BPF NAT table.
// Default: 524288
BPFNATGlobalMax int `json:"bpfNATGlobalMax,omitempty"`
// BPFNeighGlobalMax is the the maximum number of entries in the BPF Neighbor table.
// Default: 524288
BPFNeighGlobalMax int `json:"bpfNeighGlobalMax,omitempty"`
// BPFPolicyMapMax is the maximum number of entries in endpoint policy map.
// Default: 16384
BPFPolicyMapMax int `json:"bpfPolicyMapMax,omitempty"`
// BPFLBMapMax is the maximum number of entries in bpf lb service, backend and affinity maps.
// Default: 65536
BPFLBMapMax int `json:"bpfLBMapMax,omitempty"`
// PreallocateBPFMaps reduces the per-packet latency at the expense of up-front memory allocation.
// Default: true
PreallocateBPFMaps bool `json:"preallocateBPFMaps,omitempty"`
Expand Down
46 changes: 42 additions & 4 deletions pkg/apis/kops/v1alpha2/networking.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ type NetworkingSpec struct {
}

// ClassicNetworkingSpec is the specification of classic networking mode, integrated into kubernetes.
// Support been removed since kubernetes 1.4.
// Support been removed since Kubernetes 1.4.
type ClassicNetworkingSpec struct {
}

Expand Down Expand Up @@ -256,6 +256,8 @@ type AmazonVPCNetworkingSpec struct {
Env []EnvVar `json:"env,omitempty"`
}

const CiliumIpamEni = "eni"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you suggest including the above to CiliumSpec if i understand correctly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of something like

type CiliumIpam string

const CiliumIpamENI CiliumIpam = "eni" 

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about moving this into here to make things simpler? --> https://github.com/kubernetes/kops/blob/master/pkg/model/components/cilium.go

wdyt?

Copy link
Contributor Author

@dntosas dntosas Jun 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh, i believe we can completely remove it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i made the changes as you suggested ^^ let's solve to a different PR if we need to revisit where this should be placed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted, as this one will need additional changes, out of scope of PR i guess:

# k8s.io/kops/pkg/model/iam
pkg/model/iam/iam_builder.go:272:122: invalid operation: b.Cluster.Spec.Networking.Cilium.Ipam == kops.CiliumIpamEni (mismatched types string and kops.CiliumIpam)
pkg/model/iam/iam_builder.go:327:122: invalid operation: b.Cluster.Spec.Networking.Cilium.Ipam == kops.CiliumIpamEni (mismatched types string and kops.CiliumIpam)
# k8s.io/kops/pkg/model/iam
vet: pkg/model/iam/iam_builder.go:272:84: cannot compare b.Cluster.Spec.Networking.Cilium.Ipam == kops.CiliumIpamEni (mismatched types string and kops.CiliumIpam)


// CiliumNetworkingSpec declares that we want Cilium networking
type CiliumNetworkingSpec struct {
// Version is the version of the Cilium agent and the Cilium Operator.
Expand Down Expand Up @@ -301,6 +303,9 @@ type CiliumNetworkingSpec struct {
// DisableConntrack is not implemented and may be removed in the future.
// Setting this has no effect.
DisableConntrack bool `json:"disableConntrack,omitempty"`
// DisableEndpointCRD disables usage of CiliumEndpoint CRD.
// Default: false
DisableEndpointCRD bool `json:"disableEndpointCRD,omitempty"`
// DisableIpv4 is deprecated: Use EnableIpv4 instead.
// Setting this flag has no effect.
DisableIpv4 bool `json:"disableIpv4,omitempty"`
Expand All @@ -313,6 +318,15 @@ type CiliumNetworkingSpec struct {
// "never": Cilium allows all traffic regardless of policies in place.
// If unspecified, "default" policy mode will be used.
EnablePolicy string `json:"enablePolicy,omitempty"`
// EnableL7Proxy enables L7 proxy for L7 policy enforcement.
// Default: true
EnableL7Proxy *bool `json:"enableL7Proxy,omitempty"`
// EnableBPFMasquerade enables masquerading packets from endpoints leaving the host with BPF instead of iptables.
// Default: false
EnableBPFMasquerade *bool `json:"enableBPFMasquerade,omitempty"`
// EnableEndpointHealthChecking enables connectivity health checking between virtual endpoints.
// Default: true
EnableEndpointHealthChecking *bool `json:"enableEndpointHealthChecking,omitempty"`
// EnableTracing is not implemented and may be removed in the future.
// Setting this has no effect.
EnableTracing bool `json:"enableTracing,omitempty"`
Expand All @@ -324,6 +338,12 @@ type CiliumNetworkingSpec struct {
// EnvoyLog is not implemented and may be removed in the future.
// Setting this has no effect.
EnvoyLog string `json:"envoyLog,omitempty"`
// IdentityAllocationMode specifies in which backend identities are stored ("crd", "kvstore").
// Default: crd
IdentityAllocationMode string `json:"identityAllocationMode,omitempty"`
// IdentityChangeGracePeriod specifies the duration to wait before using a changed identity.
// Default: 5s
IdentityChangeGracePeriod string `json:"identityChangeGracePeriod,omitempty"`
// Ipv4ClusterCIDRMaskSize is not implemented and may be removed in the future.
// Setting this has no effect.
Ipv4ClusterCIDRMaskSize int `json:"ipv4ClusterCidrMaskSize,omitempty"`
Expand Down Expand Up @@ -434,6 +454,24 @@ type CiliumNetworkingSpec struct {
// BPFCTGlobalAnyMax is the maximum number of entries in the non-TCP CT table.
// Default: 262144
BPFCTGlobalAnyMax int `json:"bpfCTGlobalAnyMax,omitempty"`
// BPFLBAlgorithm is the load balancing algorithm ("random", "maglev").
// Default: random
BPFLBAlgorithm string `json:"bpfLBAlgorithm,omitempty"`
// BPFLBMaglevTableSize is the per service backend table size when going with Maglev (parameter M).
// Default: 16381
BPFLBMaglevTableSize string `json:"bpfLBMaglevTableSize,omitempty"`
// BPFNATGlobalMax is the the maximum number of entries in the BPF NAT table.
// Default: 524288
BPFNATGlobalMax int `json:"bpfNATGlobalMax,omitempty"`
// BPFNeighGlobalMax is the the maximum number of entries in the BPF Neighbor table.
// Default: 524288
BPFNeighGlobalMax int `json:"bpfNeighGlobalMax,omitempty"`
// BPFPolicyMapMax is the maximum number of entries in endpoint policy map.
// Default: 16384
BPFPolicyMapMax int `json:"bpfPolicyMapMax,omitempty"`
// BPFLBMapMax is the maximum number of entries in bpf lb service, backend and affinity maps.
// Default: 65536
BPFLBMapMax int `json:"bpfLBMapMax,omitempty"`
// PreallocateBPFMaps reduces the per-packet latency at the expense of up-front memory allocation.
// Default: true
PreallocateBPFMaps bool `json:"preallocateBPFMaps,omitempty"`
Expand Down Expand Up @@ -463,7 +501,7 @@ type CiliumNetworkingSpec struct {
// "crd" will use CRDs for controlling IP address management.
// "hostscope" will use hostscope IPAM mode.
// "kubernetes" will use addersing based on node pod CIDR.
// Empty value will use host-scope address management.
// Empty value will use hostscope for cilum <= 1.7 and "kubernetes" otherwise.
Ipam string `json:"ipam,omitempty"`
// IPTablesRulesNoinstall disables installing the base IPTables rules used for masquerading and kube-proxy.
// Default: false
Expand Down Expand Up @@ -509,15 +547,15 @@ type CiliumNetworkingSpec struct {

// HubbleSpec configures the Hubble service on the Cilium agent.
type HubbleSpec struct {
// Enabled specifies whether Hubble is enabled on the agent.
// Enabled decides if Hubble is enabled on the agent or not
Enabled *bool `json:"enabled,omitempty"`

// Metrics is a list of metrics to collect. If empty or null, metrics are disabled.
// See https://docs.cilium.io/en/stable/configuration/metrics/#hubble-exported-metrics
Metrics []string `json:"metrics,omitempty"`
}

// LyftVPCNetworkingSpec declares that we want to use the cni-ipvlan-vpc-k8s CNI networking
// LyftVPCNetworkingSpec declares that we want to use the cni-ipvlan-vpc-k8s CNI networking.
type LyftVPCNetworkingSpec struct {
SubnetTags map[string]string `json:"subnetTags,omitempty"`
}
Expand Down
24 changes: 24 additions & 0 deletions pkg/apis/kops/v1alpha2/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading