Cycle when creating BigQuery Omni Connection

[voycey]

It seems like there is a bug / error in how google_bigquery_connection needs to be setup. Currently the documentation on the Google Cloud website states that gcloud must be used to set this up (as there is a cyclical dependency), however there needs to be the ability to have this setup in Terraform 100% (as our deployments are touch free).

Specifically I can't see a way to define the connection role and the google_bigquery_connection without creating a cycle as it requires an IAM role and that IAM role requires the identity from google_bigquery_connection:

Terraform Version

Terraform v1.1.9
on linux_amd64

Affected Resource(s)

google_bigquery_connection
aws_iam_role

Terraform Configuration Files

resource "google_bigquery_connection" "connection" {
    provider      = google-beta
    connection_id = "bq-connection"
    location      = "aws-ap-southeast-1"
    friendly_name = "👋"
    description   = "BQ Omni Connection"
    aws {
      access_role {
         iam_role_id =  aws_iam_role.bigquery-omni-connection-role.arn
      }
    }
}

resource "aws_iam_role" "bigquery-omni-connection-role" {
    name                 = "bigquery-omni-connection"
    max_session_duration = 43200

    assume_role_policy = <<-EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Federated": "accounts.google.com"
          },
          "Action": "sts:AssumeRoleWithWebIdentity",
          "Condition": {
            "StringEquals": {
              "accounts.google.com:sub": ${google_bigquery_connection.connection.id}
            }
          }
        }
      ]
    }
    EOF
}

Debug Output

Expected Behavior

Some ability to set up the STS role without creating a cycle or to create the IAM role without requiring the output of the STS role.

Actual Behavior

Cycle

References

#11459 (comment)

[edwardmedia]

@voycey It seems there is no perfect solution. aws is optional. Is it possible you can deploy twice?

[voycey]

I think I can do it by creating proxy roles and applying them in multiple steps but its not a clean solution.

I will test it and then post it. The AWS is required here to be able to set the policy to allow BigQuery to access S3

[edwardmedia]

@voycey I knew aws is required for your solution. What I suggested is to add it in the 2nd apply?

[voycey]

I understand - but unfortunately we need to have it in a single apply as we deploy multiple environments off the same code, I will have a crack with the multiple policy attachments tomorrow and report back!

[edwardmedia]

@voycey from the provider's perspective, I can't think of a perfect solution.

[edwardmedia]

@voycey any update?

[voycey]

The only update is workarounds that need to be done to solve this temporarily - I stand by this being a bug that needs to be addressed.

I have got it applying with the following but haven't tested the connectivity yet

resource "aws_iam_policy" "bigquery-omni-s3-policy" {
    name = "bigquery-omni-connection-policy"

    policy = <<-EOF
    {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Sid": "BucketLevelAccess",
              "Effect": "Allow",
              "Action": ["s3:ListBucket"],
              "Resource": ["arn:aws:s3:::${var.meter_bucket_name}"]
          },
          {
              "Sid": "ObjectLevelAccess",
              "Effect": "Allow",
              "Action": ["s3:GetObject"],
              "Resource": ["arn:aws:s3:::${var.meter_bucket_name}"]
          }
      ]
    }
    EOF
}

resource "aws_iam_policy" "bigquery-omni-connection-policy" {
  name        = "bq-omni-policy"
  path        = "/"
  description = "BQ Omni Policy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow",
        Action = "sts:AssumeRoleWithWebIdentity",
        Resource = "*",
        Condition = {
          StringEquals = {
            "accounts.google.com:sub" = "${google_bigquery_connection.connection.id}"
          }
        }
      }
    ]
  })
}


resource "aws_iam_role" "bigquery-omni-connection-role" {
  name = "bigquery-omni-connection"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Sid    = ""
        Principal = {
          Federated = "accounts.google.com"
        }
      },
    ]
  })
}


resource "aws_iam_role_policy_attachment" "bigquery-omni-s3-role-attach" {
  role       = aws_iam_role.bigquery-omni-connection-role.name
  policy_arn = aws_iam_policy.bigquery-omni-s3-policy.arn
}

resource "aws_iam_role_policy_attachment" "bigquery-omni-connection-role-attach" {
  role       = aws_iam_role.bigquery-omni-connection-role.name
  policy_arn = aws_iam_policy.bigquery-omni-connection-policy.arn
}

output "bigquery_omni_role" {
  value = aws_iam_role.bigquery-omni-connection-role.arn
}

[edwardmedia]

@voycey there is not much we can do that the provider level. You may file an issue to Terraform Core or SDK to see if they can help. Closing the issue now

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.