From 583c7cba1121e1cdd2ef574af44f8cb0e6d8c0c0 Mon Sep 17 00:00:00 2001 From: Brent Bain <120047351+brent-anyscale@users.noreply.github.com> Date: Thu, 19 Sep 2024 09:14:53 -0700 Subject: [PATCH 1/2] add: Example for existing GKE cluster - initial commit (#3) * add: Example for existing GKE cluster - initial commit Changes to be committed: new file: README.md new file: main.tf new file: outputs.tf new file: variables.tf new file: versions.tf * upd: README for example with current status Changes to be committed: modified: README.md * upd: README Changes to be committed: modified: README.md --- examples/gcp/gke-existing_cluster/README.md | 74 ++++++++++++ examples/gcp/gke-existing_cluster/main.tf | 83 ++++++++++++++ examples/gcp/gke-existing_cluster/outputs.tf | 20 ++++ .../gcp/gke-existing_cluster/variables.tf | 108 ++++++++++++++++++ examples/gcp/gke-existing_cluster/versions.tf | 50 ++++++++ 5 files changed, 335 insertions(+) create mode 100644 examples/gcp/gke-existing_cluster/README.md create mode 100644 examples/gcp/gke-existing_cluster/main.tf create mode 100644 examples/gcp/gke-existing_cluster/outputs.tf create mode 100644 examples/gcp/gke-existing_cluster/variables.tf create mode 100644 examples/gcp/gke-existing_cluster/versions.tf diff --git a/examples/gcp/gke-existing_cluster/README.md b/examples/gcp/gke-existing_cluster/README.md new file mode 100644 index 0000000..bc46f96 --- /dev/null +++ b/examples/gcp/gke-existing_cluster/README.md @@ -0,0 +1,74 @@ +[![Build Status][badge-build]][build-status] +[![Terraform Version][badge-terraform]](https://github.com/hashicorp/terraform/releases) +[![Google Provider Version][badge-tf-google]](https://github.com/terraform-providers/terraform-provider-google/releases) + +# Anyscale GCP GKE Example - Existing Cluster +This example creates the resources to run Anyscale on GCP GKE with an existing cluster +**Work in progress** + +## Needs to Create: +- DONE - filestore +- DONE - IAM Service Accounts for ControlPlane +- DONE - Firewall +- IAM Service Accounts for Dataplane (?) (needs a cluster role for GKE) +- DONE - storage bucket +- namespace +- helm charts +- configmap + + +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.0 | +| [google](#requirement\_google) | ~> 5.0 | +| [helm](#requirement\_helm) | ~> 2.0 | +| [kubernetes](#requirement\_kubernetes) | ~> 2.0 | + +## Providers + +No providers. + +## Modules + +| Name | Source | Version | +|------|--------|---------| +| [anyscale\_cloudstorage](#module\_anyscale\_cloudstorage) | github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-cloudstorage | n/a | +| [anyscale\_filestore](#module\_anyscale\_filestore) | github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-filestore | n/a | +| [anyscale\_firewall](#module\_anyscale\_firewall) | github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-vpc-firewall | n/a | +| [anyscale\_iam](#module\_anyscale\_iam) | github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-iam | n/a | + +## Resources + +No resources. + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [anyscale\_org\_id](#input\_anyscale\_org\_id) | (Required) Anyscale Organization ID | `string` | n/a | yes | +| [customer\_ingress\_cidr\_ranges](#input\_customer\_ingress\_cidr\_ranges) | The IPv4 CIDR blocks that allows access Anyscale clusters.
These are added to the firewall and allows port 443 (https) and 22 (ssh) access.
ex: `52.1.1.23/32,10.1.0.0/16'
` | `string` | n/a | yes | +| [existing\_subnet\_cidr](#input\_existing\_subnet\_cidr) | The CIDR range of the existing subnet | `string` | n/a | yes | +| [existing\_vpc\_id](#input\_existing\_vpc\_id) | The ID of the existing VPC | `string` | n/a | yes | +| [existing\_vpc\_name](#input\_existing\_vpc\_name) | The name of the existing VPC | `string` | n/a | yes | +| [google\_project\_id](#input\_google\_project\_id) | ID of the Project to put these resources in | `string` | n/a | yes | +| [google\_region](#input\_google\_region) | The Google region in which all resources will be created. | `string` | n/a | yes | +| [anyscale\_cloud\_id](#input\_anyscale\_cloud\_id) | (Optional) Anyscale Cloud ID | `string` | `null` | no | +| [anyscale\_deploy\_env](#input\_anyscale\_deploy\_env) | (Optional) Anyscale deploy environment. Used in resource names and tags.

ex:
anyscale_deploy_env = "production"
| `string` | `"production"` | no | +| [labels](#input\_labels) | (Optional) A map of labels to all resources that accept labels. | `map(string)` | 
{
  "environment": "test",
  "test": true
}
| no | + +## Outputs + +| Name | Description | +|------|-------------| +| [anyscale\_registration\_command](#output\_anyscale\_registration\_command) | The Anyscale registration command. | + + + +[Terraform]: https://www.terraform.io +[Issues]: https://github.com/anyscale/sa-terraform-google-cloudfoundation-modules/issues +[badge-build]: https://github.com/anyscale/sa-terraform-google-cloudfoundation-modules/workflows/CI/CD%20Pipeline/badge.svg +[badge-terraform]: https://img.shields.io/badge/terraform-1.x%20-623CE4.svg?logo=terraform +[badge-tf-google]: https://img.shields.io/badge/GCP-5.+-F8991D.svg?logo=terraform +[build-status]: https://github.com/anyscale/sa-terraform-google-cloudfoundation-modules/actions diff --git a/examples/gcp/gke-existing_cluster/main.tf b/examples/gcp/gke-existing_cluster/main.tf new file mode 100644 index 0000000..2b00ba1 --- /dev/null +++ b/examples/gcp/gke-existing_cluster/main.tf @@ -0,0 +1,83 @@ +# ---------------------------------------------------------------------------------------------------------------------# Example Anyscale K8s Resources - Public Networking +# This template cretes resources for Anyscale with existing GKE Cluster +# It creates: +# - Storage Bucket +# - Filestore +# - IAM Service Accounts +# - Firewall Policy +# - Helm Charts +# It expects the following to be already created: +# - GCP Project +# - GKE Cluster +# - GKE Node Pool +# - VPC +# --------------------------------------------------------------------------------------------------------------------- +locals { + full_labels = merge(tomap({ + anyscale-cloud-id = var.anyscale_cloud_id, + anyscale-deploy-environment = var.anyscale_deploy_env + }), + var.labels + ) +} + +module "anyscale_cloudstorage" { + #checkov:skip=CKV_TF_1: Example code should use the latest version of the module + #checkov:skip=CKV_TF_2: Example code should use the latest version of the module + source = "github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-cloudstorage" + module_enabled = true + + anyscale_project_id = var.google_project_id + labels = local.full_labels +} + +module "anyscale_iam" { + #checkov:skip=CKV_TF_1: Example code should use the latest version of the module + #checkov:skip=CKV_TF_2: Example code should use the latest version of the module + source = "github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-iam" + module_enabled = true + + anyscale_org_id = var.anyscale_org_id + create_anyscale_access_role = true + create_anyscale_cluster_node_service_acct = false + + anyscale_project_id = var.google_project_id +} + +module "anyscale_filestore" { + #checkov:skip=CKV_TF_1: Example code should use the latest version of the module + #checkov:skip=CKV_TF_2: Example code should use the latest version of the module + source = "github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-filestore" + module_enabled = true + + filestore_vpc_name = var.existing_vpc_name + filestore_tier = "STANDARD" + filestore_location = "us-central1-b" + + anyscale_project_id = var.google_project_id + labels = local.full_labels +} + +module "anyscale_firewall" { + #checkov:skip=CKV_TF_1: Example code should use the latest version of the module + #checkov:skip=CKV_TF_2: Example code should use the latest version of the module + source = "github.com/anyscale/terraform-google-anyscale-cloudfoundation-modules//modules/google-anyscale-vpc-firewall" + module_enabled = true + + vpc_name = var.existing_vpc_name + vpc_id = var.existing_vpc_id + + ingress_with_self_cidr_range = [var.existing_subnet_cidr] + ingress_from_cidr_map = [ + { + rule = "https-443-tcp" + cidr_blocks = var.customer_ingress_cidr_ranges + }, + { + rule = "ssh-tcp" + cidr_blocks = var.customer_ingress_cidr_ranges + } + ] + + anyscale_project_id = var.google_project_id +} diff --git a/examples/gcp/gke-existing_cluster/outputs.tf b/examples/gcp/gke-existing_cluster/outputs.tf new file mode 100644 index 0000000..ca10ca3 --- /dev/null +++ b/examples/gcp/gke-existing_cluster/outputs.tf @@ -0,0 +1,20 @@ +output "anyscale_registration_command" { + description = "The Anyscale registration command." + value = <<-EOT + anyscale cloud register --provider gcp \ + --name \ + --compute-stack k8s \ + --project-id ${var.google_project_id} \ + --vpc-name ${var.existing_vpc_name} \ + --region ${var.google_region} \ + --cloud-storage-bucket-name ${module.anyscale_cloudstorage.cloudstorage_bucket_name} \ + --filestore-instance-id ${module.anyscale_filestore.anyscale_filestore_name} \ + --filestore-location ${module.anyscale_filestore.anyscale_filestore_location} \ + --anyscale-service-account-email ${module.anyscale_iam.iam_anyscale_access_service_acct_email} \ + --provider-name ${module.anyscale_iam.iam_workload_identity_provider_name} \ + --kubernetes-namespaces + --kubernetes-ingress-external-address \ + --kubernetes-zones \ + --kubernetes-dataplane-identity + EOT +} diff --git a/examples/gcp/gke-existing_cluster/variables.tf b/examples/gcp/gke-existing_cluster/variables.tf new file mode 100644 index 0000000..e456978 --- /dev/null +++ b/examples/gcp/gke-existing_cluster/variables.tf @@ -0,0 +1,108 @@ +# --------------------------------------------------------------------------------------------------------------------- +# ENVIRONMENT VARIABLES +# Define these secrets as environment variables +# --------------------------------------------------------------------------------------------------------------------- + + +# --------------------------------------------------------------------------------------------------------------------- +# REQUIRED VARIABLES +# These variables must be set when using this module. +# --------------------------------------------------------------------------------------------------------------------- +variable "google_region" { + description = "The Google region in which all resources will be created." + type = string +} + +variable "google_project_id" { + description = "ID of the Project to put these resources in" + type = string +} + +variable "anyscale_org_id" { + description = "(Required) Anyscale Organization ID" + type = string + validation { + condition = ( + length(var.anyscale_org_id) > 4 && + substr(var.anyscale_org_id, 0, 4) == "org_" + ) + error_message = "The anyscale_org_id value must start with \"org_\"." + } +} + + +variable "customer_ingress_cidr_ranges" { + description = <<-EOT + The IPv4 CIDR blocks that allows access Anyscale clusters. + These are added to the firewall and allows port 443 (https) and 22 (ssh) access. + ex: `52.1.1.23/32,10.1.0.0/16' + EOT + type = string +} + +# ----------------- +# Networking +# ----------------- +variable "existing_vpc_name" { + description = "The name of the existing VPC" + type = string +} + +variable "existing_vpc_id" { + description = "The ID of the existing VPC" + type = string +} + +variable "existing_subnet_cidr" { + description = "The CIDR range of the existing subnet" + type = string +} + + +# ------------------------------------------------------------------------------ +# OPTIONAL PARAMETERS +# These variables have defaults, but may be overridden. +# ------------------------------------------------------------------------------ +variable "anyscale_deploy_env" { + description = <<-EOT + (Optional) Anyscale deploy environment. Used in resource names and tags. + + ex: + ``` + anyscale_deploy_env = "production" + ``` + EOT + + type = string + default = "production" + validation { + condition = ( + var.anyscale_deploy_env == "production" || var.anyscale_deploy_env == "development" || var.anyscale_deploy_env == "test" + ) + error_message = "The anyscale_deploy_env only allows `production`, `test`, or `development`" + } +} + +variable "anyscale_cloud_id" { + description = "(Optional) Anyscale Cloud ID" + type = string + default = null + validation { + condition = ( + var.anyscale_cloud_id == null ? true : ( + length(var.anyscale_cloud_id) > 4 && + substr(var.anyscale_cloud_id, 0, 4) == "cld_" + ) + ) + error_message = "The anyscale_cloud_id value must start with \"cld_\"." + } +} + +variable "labels" { + description = "(Optional) A map of labels to all resources that accept labels." + type = map(string) + default = { + "test" : true, + "environment" : "test" + } +} diff --git a/examples/gcp/gke-existing_cluster/versions.tf b/examples/gcp/gke-existing_cluster/versions.tf new file mode 100644 index 0000000..1638fc2 --- /dev/null +++ b/examples/gcp/gke-existing_cluster/versions.tf @@ -0,0 +1,50 @@ +terraform { + required_version = ">= 1.0" + required_providers { + google = { + source = "hashicorp/google" + version = "~> 5.0" + } + + helm = { + source = "hashicorp/helm" + version = "~> 2.0" + } + + kubernetes = { + source = "hashicorp/kubernetes" + version = "~> 2.0" + } + } +} + + +provider "helm" { + kubernetes { + host = module.anyscale_eks_cluster.eks_kubeconfig.endpoint + cluster_ca_certificate = base64decode(module.anyscale_eks_cluster.eks_kubeconfig.cluster_ca_certificate) + + # https://registry.terraform.io/providers/hashicorp/helm/latest/docs#exec-plugins + exec { + api_version = "client.authentication.k8s.io/v1beta1" + args = ["eks", "get-token", "--cluster-name", module.anyscale_eks_cluster.eks_cluster_name] + command = "aws" + } + } +} + +provider "kubernetes" { + host = module.anyscale_eks_cluster.eks_kubeconfig.endpoint + cluster_ca_certificate = base64decode(module.anyscale_eks_cluster.eks_kubeconfig.cluster_ca_certificate) + + exec { + api_version = "client.authentication.k8s.io/v1beta1" + args = ["eks", "get-token", "--cluster-name", module.anyscale_eks_cluster.eks_cluster_name] + command = "aws" + } +} + +provider "google" { + project = var.google_project_id + region = var.google_region +} From afccf7acfcd44ef1abff7a3fb1b86c084b4b4409 Mon Sep 17 00:00:00 2001 From: hongchaodeng Date: Thu, 19 Sep 2024 19:48:42 -0700 Subject: [PATCH 2/2] Add example for existing GKE cluster Signed-off-by: hongchaodeng --- .gitignore | 2 +- examples/gcp/gke-existing_cluster/README.md | 54 +++++++++++++------ examples/gcp/gke-existing_cluster/main.tf | 51 +++++++++++++++--- examples/gcp/gke-existing_cluster/outputs.tf | 4 +- .../gcp/gke-existing_cluster/variables.tf | 29 +++++----- examples/gcp/gke-existing_cluster/versions.tf | 40 +++++++------- modules/anyscale-k8s-helm/README.md | 4 +- modules/anyscale-k8s-namespace/README.md | 37 +++++++++++++ 8 files changed, 160 insertions(+), 61 deletions(-) diff --git a/.gitignore b/.gitignore index 12e6978..8268763 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,5 @@ # Local .terraform directories -**/.terraform/* +.terraform # Terraform lockfile .terraform.lock.hcl diff --git a/examples/gcp/gke-existing_cluster/README.md b/examples/gcp/gke-existing_cluster/README.md index bc46f96..3839f63 100644 --- a/examples/gcp/gke-existing_cluster/README.md +++ b/examples/gcp/gke-existing_cluster/README.md @@ -3,18 +3,29 @@ [![Google Provider Version][badge-tf-google]](https://github.com/terraform-providers/terraform-provider-google/releases) # Anyscale GCP GKE Example - Existing Cluster -This example creates the resources to run Anyscale on GCP GKE with an existing cluster -**Work in progress** - -## Needs to Create: -- DONE - filestore -- DONE - IAM Service Accounts for ControlPlane -- DONE - Firewall -- IAM Service Accounts for Dataplane (?) (needs a cluster role for GKE) -- DONE - storage bucket -- namespace -- helm charts -- configmap + +This example creates the resources to run Anyscale on GCP GKE with an existing GKE cluster. + +## Known Issues on GKE + +- Autopilot GKE clusters are not supported. +- Node auto-provisioning for GKE failing with GPU nodes: https://github.com/GoogleCloudPlatform/container-engine-accelerators/issues/407 +- When choosing "GPU Driver installation", select "Google-managed". + +## terraform.tfvars + +```hcl +anyscale_deploy_env = "..." +anyscale_org_id = "..." # Troubleshooting Org Id + +google_region = "..." +google_project_id = "..." +existing_vpc_name = "..." +existing_subnet_name = "..." +customer_ingress_cidr_ranges = "0.0.0.0/0" +gke_endpoint = "..." +gke_ca_certificate = "..." +``` ## Requirements @@ -28,7 +39,10 @@ This example creates the resources to run Anyscale on GCP GKE with an existing c ## Providers -No providers. +| Name | Version | +|------|---------| +| [google](#provider\_google) | 5.44.1 | +| [helm](#provider\_helm) | 2.15.0 | ## Modules @@ -41,7 +55,12 @@ No providers. ## Resources -No resources. +| Name | Type | +|------|------| +| [helm_release.ingress_nginx](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource | +| [google_client_config.provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/client_config) | data source | +| [google_compute_network.existing_vpc](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_network) | data source | +| [google_compute_subnetwork.exising_subnet](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_subnetwork) | data source | ## Inputs @@ -49,14 +68,15 @@ No resources. |------|-------------|------|---------|:--------:| | [anyscale\_org\_id](#input\_anyscale\_org\_id) | (Required) Anyscale Organization ID | `string` | n/a | yes | | [customer\_ingress\_cidr\_ranges](#input\_customer\_ingress\_cidr\_ranges) | The IPv4 CIDR blocks that allows access Anyscale clusters.
These are added to the firewall and allows port 443 (https) and 22 (ssh) access.
ex: `52.1.1.23/32,10.1.0.0/16'
` | `string` | n/a | yes | -| [existing\_subnet\_cidr](#input\_existing\_subnet\_cidr) | The CIDR range of the existing subnet | `string` | n/a | yes | -| [existing\_vpc\_id](#input\_existing\_vpc\_id) | The ID of the existing VPC | `string` | n/a | yes | +| [existing\_subnet\_name](#input\_existing\_subnet\_name) | The name of the existing Subnet | `string` | n/a | yes | | [existing\_vpc\_name](#input\_existing\_vpc\_name) | The name of the existing VPC | `string` | n/a | yes | +| [gke\_ca\_certificate](#input\_gke\_ca\_certificate) | Base64 encoded PEM certificate for the cluster | `string` | n/a | yes | +| [gke\_endpoint](#input\_gke\_endpoint) | The endpoint for the GKE cluster | `string` | n/a | yes | | [google\_project\_id](#input\_google\_project\_id) | ID of the Project to put these resources in | `string` | n/a | yes | | [google\_region](#input\_google\_region) | The Google region in which all resources will be created. | `string` | n/a | yes | | [anyscale\_cloud\_id](#input\_anyscale\_cloud\_id) | (Optional) Anyscale Cloud ID | `string` | `null` | no | | [anyscale\_deploy\_env](#input\_anyscale\_deploy\_env) | (Optional) Anyscale deploy environment. Used in resource names and tags.

ex:
anyscale_deploy_env = "production"
| `string` | `"production"` | no | -| [labels](#input\_labels) | (Optional) A map of labels to all resources that accept labels. | `map(string)` | 
{
  "environment": "test",
  "test": true
}
| no | +| [labels](#input\_labels) | (Optional) A map of labels to all resources that accept labels. | `map(string)` | 
{
  "environment": "test"
}
| no | ## Outputs diff --git a/examples/gcp/gke-existing_cluster/main.tf b/examples/gcp/gke-existing_cluster/main.tf index 2b00ba1..ec6d13e 100644 --- a/examples/gcp/gke-existing_cluster/main.tf +++ b/examples/gcp/gke-existing_cluster/main.tf @@ -5,12 +5,14 @@ # - Filestore # - IAM Service Accounts # - Firewall Policy -# - Helm Charts +# - Nginx ingress controller (Helm Chart) # It expects the following to be already created: # - GCP Project -# - GKE Cluster -# - GKE Node Pool -# - VPC +# - GKE cluster +# - VPC and Subnet +# - Dataplane service account: See https://docs.anyscale.com/administration/cloud-deployment/deploy-gcp-cloud +# - Workload Identity Provider +# # --------------------------------------------------------------------------------------------------------------------- locals { full_labels = merge(tomap({ @@ -29,6 +31,8 @@ module "anyscale_cloudstorage" { anyscale_project_id = var.google_project_id labels = local.full_labels + + bucket_force_destroy = true } module "anyscale_iam" { @@ -58,6 +62,15 @@ module "anyscale_filestore" { labels = local.full_labels } +data "google_compute_network" "existing_vpc" { + name = var.existing_vpc_name +} + +data "google_compute_subnetwork" "exising_subnet" { + name = var.existing_subnet_name + region = var.google_region +} + module "anyscale_firewall" { #checkov:skip=CKV_TF_1: Example code should use the latest version of the module #checkov:skip=CKV_TF_2: Example code should use the latest version of the module @@ -65,9 +78,9 @@ module "anyscale_firewall" { module_enabled = true vpc_name = var.existing_vpc_name - vpc_id = var.existing_vpc_id + vpc_id = data.google_compute_network.existing_vpc.id - ingress_with_self_cidr_range = [var.existing_subnet_cidr] + ingress_with_self_cidr_range = [data.google_compute_subnetwork.exising_subnet.ip_cidr_range] ingress_from_cidr_map = [ { rule = "https-443-tcp" @@ -81,3 +94,29 @@ module "anyscale_firewall" { anyscale_project_id = var.google_project_id } + +resource "helm_release" "ingress_nginx" { + name = "ingress-nginx" + repository = "https://kubernetes.github.io/ingress-nginx" + chart = "ingress-nginx" + version = "4.11.2" + namespace = "ingress-nginx" + + create_namespace = true + wait = false + + set { + name = "controller.service.type" + value = "LoadBalancer" + } + + set { + name = "controller.service.annotations.cloud\\.google\\.com/load-balancer-type" + value = "External" + } + + set { + name = "controller.service.externalTrafficPolicy" + value = "Local" + } +} diff --git a/examples/gcp/gke-existing_cluster/outputs.tf b/examples/gcp/gke-existing_cluster/outputs.tf index ca10ca3..52f22f7 100644 --- a/examples/gcp/gke-existing_cluster/outputs.tf +++ b/examples/gcp/gke-existing_cluster/outputs.tf @@ -12,9 +12,9 @@ output "anyscale_registration_command" { --filestore-location ${module.anyscale_filestore.anyscale_filestore_location} \ --anyscale-service-account-email ${module.anyscale_iam.iam_anyscale_access_service_acct_email} \ --provider-name ${module.anyscale_iam.iam_workload_identity_provider_name} \ - --kubernetes-namespaces + --kubernetes-namespaces \ --kubernetes-ingress-external-address \ --kubernetes-zones \ - --kubernetes-dataplane-identity + --kubernetes-dataplane-identity EOT } diff --git a/examples/gcp/gke-existing_cluster/variables.tf b/examples/gcp/gke-existing_cluster/variables.tf index e456978..23ef8fe 100644 --- a/examples/gcp/gke-existing_cluster/variables.tf +++ b/examples/gcp/gke-existing_cluster/variables.tf @@ -30,13 +30,16 @@ variable "anyscale_org_id" { } } +# ----------------- +# Kubernetes +# ----------------- +variable "gke_endpoint" { + description = "The endpoint for the GKE cluster" + type = string +} -variable "customer_ingress_cidr_ranges" { - description = <<-EOT - The IPv4 CIDR blocks that allows access Anyscale clusters. - These are added to the firewall and allows port 443 (https) and 22 (ssh) access. - ex: `52.1.1.23/32,10.1.0.0/16' - EOT +variable "gke_ca_certificate" { + description = "Base64 encoded PEM certificate for the cluster" type = string } @@ -48,17 +51,20 @@ variable "existing_vpc_name" { type = string } -variable "existing_vpc_id" { - description = "The ID of the existing VPC" +variable "existing_subnet_name" { + description = "The name of the existing Subnet" type = string } -variable "existing_subnet_cidr" { - description = "The CIDR range of the existing subnet" +variable "customer_ingress_cidr_ranges" { + description = <<-EOT + The IPv4 CIDR blocks that allows access Anyscale clusters. + These are added to the firewall and allows port 443 (https) and 22 (ssh) access. + ex: `52.1.1.23/32,10.1.0.0/16' + EOT type = string } - # ------------------------------------------------------------------------------ # OPTIONAL PARAMETERS # These variables have defaults, but may be overridden. @@ -102,7 +108,6 @@ variable "labels" { description = "(Optional) A map of labels to all resources that accept labels." type = map(string) default = { - "test" : true, "environment" : "test" } } diff --git a/examples/gcp/gke-existing_cluster/versions.tf b/examples/gcp/gke-existing_cluster/versions.tf index 1638fc2..0383e89 100644 --- a/examples/gcp/gke-existing_cluster/versions.tf +++ b/examples/gcp/gke-existing_cluster/versions.tf @@ -18,33 +18,31 @@ terraform { } } - -provider "helm" { - kubernetes { - host = module.anyscale_eks_cluster.eks_kubeconfig.endpoint - cluster_ca_certificate = base64decode(module.anyscale_eks_cluster.eks_kubeconfig.cluster_ca_certificate) - - # https://registry.terraform.io/providers/hashicorp/helm/latest/docs#exec-plugins - exec { - api_version = "client.authentication.k8s.io/v1beta1" - args = ["eks", "get-token", "--cluster-name", module.anyscale_eks_cluster.eks_cluster_name] - command = "aws" - } - } +provider "google" { + project = var.google_project_id + region = var.google_region } provider "kubernetes" { - host = module.anyscale_eks_cluster.eks_kubeconfig.endpoint - cluster_ca_certificate = base64decode(module.anyscale_eks_cluster.eks_kubeconfig.cluster_ca_certificate) - + host = "https://${var.gke_endpoint}" + token = data.google_client_config.provider.access_token + cluster_ca_certificate = base64decode(var.gke_ca_certificate) exec { api_version = "client.authentication.k8s.io/v1beta1" - args = ["eks", "get-token", "--cluster-name", module.anyscale_eks_cluster.eks_cluster_name] - command = "aws" + command = "gke-gcloud-auth-plugin" } } -provider "google" { - project = var.google_project_id - region = var.google_region +data "google_client_config" "provider" {} + +provider "helm" { + kubernetes { + host = "https://${var.gke_endpoint}" + token = data.google_client_config.provider.access_token + cluster_ca_certificate = base64decode(var.gke_ca_certificate) + exec { + api_version = "client.authentication.k8s.io/v1beta1" + command = "gke-gcloud-auth-plugin" + } + } } diff --git a/modules/anyscale-k8s-helm/README.md b/modules/anyscale-k8s-helm/README.md index 38f33a3..a053679 100644 --- a/modules/anyscale-k8s-helm/README.md +++ b/modules/anyscale-k8s-helm/README.md @@ -22,10 +22,10 @@ This module creates Kubernetes helm charts for Anyscale applications and workloa | Name | Version | |------|---------| -| [aws](#provider\_aws) | 5.63.0 | +| [aws](#provider\_aws) | 5.68.0 | | [helm](#provider\_helm) | 2.15.0 | | [kubernetes](#provider\_kubernetes) | 2.32.0 | -| [time](#provider\_time) | 0.12.0 | +| [time](#provider\_time) | 0.12.1 | ## Modules diff --git a/modules/anyscale-k8s-namespace/README.md b/modules/anyscale-k8s-namespace/README.md index dfb542a..a4c2f9b 100644 --- a/modules/anyscale-k8s-namespace/README.md +++ b/modules/anyscale-k8s-namespace/README.md @@ -12,6 +12,43 @@ This module creates a Kubernetes Namespace for Anyscale. The Anyscale Namespace can also be created via the Anycsale Helm Chart. +## Requirements + +| Name | Version | +|------|---------| +| [terraform](#requirement\_terraform) | >= 1.0 | +| [kubernetes](#requirement\_kubernetes) | ~> 2.0 | + +## Providers + +| Name | Version | +|------|---------| +| [kubernetes](#provider\_kubernetes) | 2.32.0 | + +## Modules + +No modules. + +## Resources + +| Name | Type | +|------|------| +| [kubernetes_namespace.anyscale](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| [cloud\_provider](#input\_cloud\_provider) | (Required) The cloud provider (aws or gcp)

ex:
cloud_provider = "aws"
| `string` | n/a | yes | +| [anyscale\_kubernetes\_namespace](#input\_anyscale\_kubernetes\_namespace) | (Optional) The name of the Kubernetes namespace.

ex:
anyscale_kubernetes_namespace = "anyscale-k8s"
| `string` | `"anyscale-k8s"` | no | +| [kubernetes\_cluster\_name](#input\_kubernetes\_cluster\_name) | (Optional) The name of the Kubernetes cluster.

ex:
kubernetes_cluster_name = "my-cluster"
| `string` | `null` | no | +| [module\_enabled](#input\_module\_enabled) | (Optional) Determines if this module should create resources.

If set to true, `eks_role_arn`, `anyscale_subnet_ids`, and `anyscale_security_group_id` must be provided.
ex:
module_enabled = true
| `bool` | `true` | no | + +## Outputs + +| Name | Description | +|------|-------------| +| [anyscale\_kubernetes\_namespace\_name](#output\_anyscale\_kubernetes\_namespace\_name) | The name of the Kubernetes namespace. |