Composer v2 - the zone argument is both required and disallowed for node_config

[andykyrala]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.0.5
on darwin_amd64

• provider registry.terraform.io/hashicorp/archive v2.2.0
• provider registry.terraform.io/hashicorp/google v3.82.0
• provider registry.terraform.io/hashicorp/google-beta v3.82.0
• provider registry.terraform.io/pagerduty/pagerduty v1.10.1
• provider registry.terraform.io/terraform-providers/statuscake v1.0.1

Affected Resource(s)

• google_composer_environment

Terraform Configuration Files

resource "google_service_account" "airflow_sa" {
  account_id   = var.airflow_instance_name
  display_name = var.airflow_instance_name
  project      = var.project
}

resource "google_project_iam_member" "airflow_iam" {
  for_each = var.airflow_roles
  member   = "serviceAccount:${google_service_account.airflow_sa.email}"
  project  = var.project
  role     = each.value
}

resource "google_composer_environment" "airflow2" {
  name    = var.airflow_instance_name2
  project = var.project
  region  = var.region
  labels = {
    "team" = "data-engineering"
  }

  config {
    node_config {
      network         = google_compute_network.comp_ntwk.id
      subnetwork      = google_compute_subnetwork.comp_subntwk.id
      service_account = google_service_account.airflow_sa.email
    }

    software_config {
      airflow_config_overrides = {
        core-load_example          = "False"
        core-dagbag_import_timeout = "180"
        core-dags_are_paused_at_creation = "True"
      }
      pypi_packages = {
        apache-airflow-providers-slack = "[http]==4.0.1"
        slack-sdk                      = "==3.10.1"
      }
      image_version = var.airflow_version2
    }
  }
}

resource "google_compute_network" "comp_ntwk" {
  name                    = "${var.airflow_instance_name2}-network"
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "comp_subntwk" {
  name          = "${var.airflow_instance_name2}-subnetwork"
  ip_cidr_range = var.airflow_cidr_range
  region        = var.region
  network       = google_compute_network.comp_ntwk.id
}

Debug Output

[<!---
Please provide a link to a GitHub Gist containing the complete debug output. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

To obtain the debug output, run terraform apply with the environment variable TF_LOG=DEBUG. See the Terraform documentation on debugging for more information.
--->]
(https://gist.github.com/andykyrala/b7f0ffce9008bee8b7a8dc82cf6b4645)

Panic Output

Expected Behavior

This should have spun up a composer v2 instance

Actual Behavior

This errors when Zone is supplied under the node config, and also when Zone is not supplied under the node config

Steps to Reproduce

1. terraform apply

Important Factoids

var.airflow_version2 is set to composer-2.0.0-airflow-2.1.4

References

• Composer deployment fails when we specify Node Zone (required field) .  #10235
  ^ may be related but I couldn't see a resolution

b/301066088

[PeWu]

Duplicate of #15450

[melinath]

Closing as duplicate

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.