Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added missing parameters in kubelet and linux node configuration #1918

Merged
merged 1 commit into from
Dec 11, 2023
Merged

Added missing parameters in kubelet and linux node configuration #1918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions modules/gke-nodepool/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,22 +143,22 @@ module "cluster-1-nodepool-gpu-1" {
|---|---|:---:|:---:|:---:|
| [cluster_name](variables.tf#L23) | Cluster name. | <code>string</code> | ✓ | |
| [location](variables.tf#L41) | Cluster location. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L166) | Cluster project id. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L170) | Cluster project id. | <code>string</code> | ✓ | |
| [cluster_id](variables.tf#L17) | Cluster id. Optional, but providing cluster_id is recommended to prevent cluster misconfiguration in some of the edge cases. | <code>string</code> | | <code>null</code> |
| [gke_version](variables.tf#L28) | Kubernetes nodes version. Ignored if auto_upgrade is set in management_config. | <code>string</code> | | <code>null</code> |
| [labels](variables.tf#L34) | Kubernetes labels applied to each node. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> |
| [max_pods_per_node](variables.tf#L46) | Maximum number of pods per node. | <code>number</code> | | <code>null</code> |
| [name](variables.tf#L52) | Optional nodepool name. | <code>string</code> | | <code>null</code> |
| [node_config](variables.tf#L58) | Node-level configuration. | <code title="object&#40;&#123;&#10; boot_disk_kms_key &#61; optional&#40;string&#41;&#10; disk_size_gb &#61; optional&#40;number&#41;&#10; disk_type &#61; optional&#40;string&#41;&#10; ephemeral_ssd_count &#61; optional&#40;number&#41;&#10; gcfs &#61; optional&#40;bool, false&#41;&#10; guest_accelerator &#61; optional&#40;object&#40;&#123;&#10; count &#61; number&#10; type &#61; string&#10; gpu_driver &#61; optional&#40;object&#40;&#123;&#10; version &#61; string&#10; partition_size &#61; optional&#40;string&#41;&#10; max_shared_clients_per_gpu &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; local_nvme_ssd_count &#61; optional&#40;number&#41;&#10; gvnic &#61; optional&#40;bool, false&#41;&#10; image_type &#61; optional&#40;string&#41;&#10; kubelet_config &#61; optional&#40;object&#40;&#123;&#10; cpu_manager_policy &#61; string&#10; cpu_cfs_quota &#61; optional&#40;bool&#41;&#10; cpu_cfs_quota_period &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; linux_node_config_sysctls &#61; optional&#40;map&#40;string&#41;&#41;&#10; local_ssd_count &#61; optional&#40;number&#41;&#10; machine_type &#61; optional&#40;string&#41;&#10; metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10; min_cpu_platform &#61; optional&#40;string&#41;&#10; preemptible &#61; optional&#40;bool&#41;&#10; sandbox_config_gvisor &#61; optional&#40;bool&#41;&#10; shielded_instance_config &#61; optional&#40;object&#40;&#123;&#10; enable_integrity_monitoring &#61; optional&#40;bool&#41;&#10; enable_secure_boot &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; spot &#61; optional&#40;bool&#41;&#10; workload_metadata_config_mode &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; disk_type &#61; &#34;pd-balanced&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [node_count](variables.tf#L113) | Number of nodes per instance group. Initial value can only be changed by recreation, current is ignored when autoscaling is used. | <code title="object&#40;&#123;&#10; current &#61; optional&#40;number&#41;&#10; initial &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; initial &#61; 1&#10;&#125;">&#123;&#8230;&#125;</code> |
| [node_locations](variables.tf#L125) | Node locations. | <code>list&#40;string&#41;</code> | | <code>null</code> |
| [nodepool_config](variables.tf#L131) | Nodepool-level configuration. | <code title="object&#40;&#123;&#10; autoscaling &#61; optional&#40;object&#40;&#123;&#10; location_policy &#61; optional&#40;string&#41;&#10; max_node_count &#61; optional&#40;number&#41;&#10; min_node_count &#61; optional&#40;number&#41;&#10; use_total_nodes &#61; optional&#40;bool, false&#41;&#10; &#125;&#41;&#41;&#10; management &#61; optional&#40;object&#40;&#123;&#10; auto_repair &#61; optional&#40;bool&#41;&#10; auto_upgrade &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; upgrade_settings &#61; optional&#40;object&#40;&#123;&#10; max_surge &#61; number&#10; max_unavailable &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [pod_range](variables.tf#L153) | Pod secondary range configuration. | <code title="object&#40;&#123;&#10; secondary_pod_range &#61; object&#40;&#123;&#10; name &#61; string&#10; cidr &#61; optional&#40;string&#41;&#10; create &#61; optional&#40;bool&#41;&#10; enable_private_nodes &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [reservation_affinity](variables.tf#L171) | Configuration of the desired reservation which instances could take capacity from. | <code title="object&#40;&#123;&#10; consume_reservation_type &#61; string&#10; key &#61; optional&#40;string&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [service_account](variables.tf#L181) | Nodepool service account. If this variable is set to null, the default GCE service account will be used. If set and email is null, a service account will be created. If scopes are null a default will be used. | <code title="object&#40;&#123;&#10; create &#61; optional&#40;bool, false&#41;&#10; email &#61; optional&#40;string&#41;&#10; oauth_scopes &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [sole_tenant_nodegroup](variables.tf#L192) | Sole tenant node group. | <code>string</code> | | <code>null</code> |
| [tags](variables.tf#L198) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> | | <code>null</code> |
| [taints](variables.tf#L204) | Kubernetes taints applied to all nodes. | <code title="map&#40;object&#40;&#123;&#10; value &#61; string&#10; effect &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [node_config](variables.tf#L58) | Node-level configuration. | <code title="object&#40;&#123;&#10; boot_disk_kms_key &#61; optional&#40;string&#41;&#10; disk_size_gb &#61; optional&#40;number&#41;&#10; disk_type &#61; optional&#40;string&#41;&#10; ephemeral_ssd_count &#61; optional&#40;number&#41;&#10; gcfs &#61; optional&#40;bool, false&#41;&#10; guest_accelerator &#61; optional&#40;object&#40;&#123;&#10; count &#61; number&#10; type &#61; string&#10; gpu_driver &#61; optional&#40;object&#40;&#123;&#10; version &#61; string&#10; partition_size &#61; optional&#40;string&#41;&#10; max_shared_clients_per_gpu &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; local_nvme_ssd_count &#61; optional&#40;number&#41;&#10; gvnic &#61; optional&#40;bool, false&#41;&#10; image_type &#61; optional&#40;string&#41;&#10; kubelet_config &#61; optional&#40;object&#40;&#123;&#10; cpu_manager_policy &#61; string&#10; cpu_cfs_quota &#61; optional&#40;bool&#41;&#10; cpu_cfs_quota_period &#61; optional&#40;string&#41;&#10; pod_pids_limit &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; linux_node_config &#61; optional&#40;object&#40;&#123;&#10; sysctls &#61; optional&#40;map&#40;string&#41;&#41;&#10; cgroup_mode &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; local_ssd_count &#61; optional&#40;number&#41;&#10; machine_type &#61; optional&#40;string&#41;&#10; metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10; min_cpu_platform &#61; optional&#40;string&#41;&#10; preemptible &#61; optional&#40;bool&#41;&#10; sandbox_config_gvisor &#61; optional&#40;bool&#41;&#10; shielded_instance_config &#61; optional&#40;object&#40;&#123;&#10; enable_integrity_monitoring &#61; optional&#40;bool&#41;&#10; enable_secure_boot &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; spot &#61; optional&#40;bool&#41;&#10; workload_metadata_config_mode &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; disk_type &#61; &#34;pd-balanced&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [node_count](variables.tf#L117) | Number of nodes per instance group. Initial value can only be changed by recreation, current is ignored when autoscaling is used. | <code title="object&#40;&#123;&#10; current &#61; optional&#40;number&#41;&#10; initial &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; initial &#61; 1&#10;&#125;">&#123;&#8230;&#125;</code> |
| [node_locations](variables.tf#L129) | Node locations. | <code>list&#40;string&#41;</code> | | <code>null</code> |
| [nodepool_config](variables.tf#L135) | Nodepool-level configuration. | <code title="object&#40;&#123;&#10; autoscaling &#61; optional&#40;object&#40;&#123;&#10; location_policy &#61; optional&#40;string&#41;&#10; max_node_count &#61; optional&#40;number&#41;&#10; min_node_count &#61; optional&#40;number&#41;&#10; use_total_nodes &#61; optional&#40;bool, false&#41;&#10; &#125;&#41;&#41;&#10; management &#61; optional&#40;object&#40;&#123;&#10; auto_repair &#61; optional&#40;bool&#41;&#10; auto_upgrade &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; upgrade_settings &#61; optional&#40;object&#40;&#123;&#10; max_surge &#61; number&#10; max_unavailable &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [pod_range](variables.tf#L157) | Pod secondary range configuration. | <code title="object&#40;&#123;&#10; secondary_pod_range &#61; object&#40;&#123;&#10; name &#61; string&#10; cidr &#61; optional&#40;string&#41;&#10; create &#61; optional&#40;bool&#41;&#10; enable_private_nodes &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [reservation_affinity](variables.tf#L175) | Configuration of the desired reservation which instances could take capacity from. | <code title="object&#40;&#123;&#10; consume_reservation_type &#61; string&#10; key &#61; optional&#40;string&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [service_account](variables.tf#L185) | Nodepool service account. If this variable is set to null, the default GCE service account will be used. If set and email is null, a service account will be created. If scopes are null a default will be used. | <code title="object&#40;&#123;&#10; create &#61; optional&#40;bool, false&#41;&#10; email &#61; optional&#40;string&#41;&#10; oauth_scopes &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [sole_tenant_nodegroup](variables.tf#L196) | Sole tenant node group. | <code>string</code> | | <code>null</code> |
| [tags](variables.tf#L202) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> | | <code>null</code> |
| [taints](variables.tf#L208) | Kubernetes taints applied to all nodes. | <code title="map&#40;object&#40;&#123;&#10; value &#61; string&#10; effect &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |

## Outputs

Expand Down
6 changes: 4 additions & 2 deletions modules/gke-nodepool/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,12 +201,14 @@ resource "google_container_node_pool" "nodepool" {
cpu_manager_policy = var.node_config.kubelet_config.cpu_manager_policy
cpu_cfs_quota = var.node_config.kubelet_config.cpu_cfs_quota
cpu_cfs_quota_period = var.node_config.kubelet_config.cpu_cfs_quota_period
pod_pids_limit = var.node_config.kubelet_config.pod_pids_limit
}
}
dynamic "linux_node_config" {
for_each = var.node_config.linux_node_config_sysctls != null ? [""] : []
for_each = var.node_config.linux_node_config != null ? [""] : []
content {
sysctls = var.node_config.linux_node_config_sysctls
sysctls = var.node_config.linux_node_config.sysctls
cgroup_mode = try(var.node_config.linux_node_config.cgroup_mode, "CGROUP_MODE_UNSPECIFIED")
}
}
dynamic "reservation_affinity" {
Expand Down
18 changes: 11 additions & 7 deletions modules/gke-nodepool/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,14 +79,18 @@ variable "node_config" {
cpu_manager_policy = string
cpu_cfs_quota = optional(bool)
cpu_cfs_quota_period = optional(string)
pod_pids_limit = optional(number)
}))
linux_node_config_sysctls = optional(map(string))
local_ssd_count = optional(number)
machine_type = optional(string)
metadata = optional(map(string))
min_cpu_platform = optional(string)
preemptible = optional(bool)
sandbox_config_gvisor = optional(bool)
linux_node_config = optional(object({
sysctls = optional(map(string))
cgroup_mode = optional(string)
}))
local_ssd_count = optional(number)
machine_type = optional(string)
metadata = optional(map(string))
min_cpu_platform = optional(string)
preemptible = optional(bool)
sandbox_config_gvisor = optional(bool)
shielded_instance_config = optional(object({
enable_integrity_monitoring = optional(bool)
enable_secure_boot = optional(bool)
Expand Down