Support for GCP?

[gonzaatcroud]

I have used this tool in the past for a couple of AWS projects (congrats BTW, this is fantastic).

I find myself needing to do the same but with GCP. I'm having trouble trying to find something with a similar scope to this but in the GCP domain.

Are there any plans for supporting GCP in the future? If not, can you offer guidance on how you would fork this project and include GCP support?

Thanks!

[iann0036]

Hi @gonzaatcroud,

Apologies for the very late response.

I've started some initial work on this by beginning the mapping, however this may take some time before it's sufficient enough to be included in the tooling - but it is possible in the future.

[gonzaatcroud]

Hi @gonzaatcroud,

Apologies for the very late response.

I've started some initial work on this by beginning the mapping, however this may take some time before it's sufficient enough to be included in the tooling - but it is possible in the future.

This would be amazing, thanks for sharing an update on it!

[iann0036]

Preview support out today, but lots of mappings missing so use with caution.

Feedback welcomed!

[joemiller]

This is fantastic! thanks for adding this.

It works fine with the gcloud cli after setting the proxy config with gcloud config set as described in the README, however, terraform google provider seems to ignore these settings. Are there environment vars that can be set instead? I'm assuming the tf google provider is using the standard go sdk for gcp

[iann0036]

Hey @joemiller,

I'm surprised that hasn't worked out of the box for you if it's using the SDK under the hood. I do notice an unresolved issue relating to this here: hashicorp/terraform#523

Do none of the configs work?

gcloud config set proxy/type http
gcloud config set proxy/address 127.0.0.1
gcloud config set proxy/port 10080
gcloud config set core/custom_ca_certs_file ~/.iamlive/ca.pem
http_proxy=http://127.0.0.1:10080 HTTP_PROXY=http://127.0.0.1:10080 HTTPS_PROXY=http://127.0.0.1:10080 https_proxy=http://127.0.0.1:10080 terraform plan

If not, I may go hunting in Terraform.

[joemiller]

@iann0036 The gcloud config set pattern did not work. Running tf just ran as normal, no output from iamlive proxy.

I did have a little more luck with the _PROXY env vars. The problem is that it also sends the aws provider thru the proxy, if I recall. I was using it with a rather large stack that had multiple providers (eg. aws, gcp, k8s)

I'm also running tf via terragrunt. I'm not sure yet if that also complicating things. I hope to try again in the next day or two.

EDIT: spent a few minutes on another attempt today. Setting with gcloud config set has no effect. Using *_PROXY env vars does seem like it could work but in my case only if there was a way to ensure those env vars were only set by TF when exec'ing the google provider. I can't find a way w/ terragrunt or terraform to only set env vars for some providers.

EDIT 2: Possible path... gcloud cli respects these vars: CLOUDSDK_PROXY_TYPE=http CLOUDSDK_PROXY_ADDRESS=127.0.0.1 CLOUDSDK_PROXY_PORT=10080 CLOUDSDK_CORE_CUSTOM_CA_CERTS_FILE=/home/joe/.iamlive/ca.pem gcloud projects list but not sure if the google tf provider does. I've tried a few different ways to get them into the process with no luck thus far.

According to this googleapis/google-api-go-client#221 (comment) the gcp go client would respect the standard HTTP_PROXY env vars and that's about it, which is consistent w/ my experiments so far.

My next step is to try a small fork of the provider to add some method to inject specific proxy settings, either via provider config or special env vars like the CLOUDSDK_* vars that gcloud cli recognizes

[joemiller]

SUCCESS! Here's what I had to do:

The key: NO_PROXY env var.

I was able to get a successful terragrunt plan run with output from iamlive using:

1. NO_PROXY to exclude everything except googleapis. This takes some trial and error. See mine below for a starting point.
2. HTTPS_PROXY=http://localhost:10080
3. Add the iamlive.ca.pem to the system trust store since there is no other way that I am aware of to induce the go http stdlib to use alternative CA certs. (on ubuntu/debian: sudo cp ~/.iamlive/ca.pem /usr/local/share/ca-certificates/iamlive.ca.crt && sudo update-ca-certificates. The file must be suffixed .crt for update-ca-certs to pick it up)

Trickiest part here was getting NO_PROXY right and this will be different for everyone, most likely. Here is my final NO_PROXY:

NO_PROXY=tailscale.com,okta.com,githubusercontent.com,github.com,terraform.io,hashicorp.com,amazonaws.com

Most of these were easy to figure out because tf would fail early with an error about not being able to fetch a plugin from *.terraform.io or *.hashicorp.com. A very tricky one was okta.com however. This one was not obvious at first. Terraform was simply hanging forever on a specific resource data.google_project in this case. It was not until I started watching the proxy with tcpdump -i any -n -s0 -A port 10080 that I noticed okta was being requested thru the proxy. This is specific to my setup, most likely, and others might not run into the exact case. But if you see terraform hanging (retrying a resource until timeout) I would reach to tcpdump to figure out what is trying and failing to run thru the proxy.

One thing I found surprising is some of the permissions being reported by iamlive. For example orgpolicy.policy.{get,set}. I'm pretty sure these are not being used as I have not given the GSA used by TF any roles that contain those. And there are actually not that many roles that have those perms, eg:

$ gcp-iam-lookup orgpolicy.policy.set  

==> Searching for roles containing permissions: [orgpolicy.policy.set] ...
roles/assuredworkloads.admin (21)
roles/assuredworkloads.editor (21)
roles/orgpolicy.policyAdmin (16)

I also saw resourcemanager.projects.undelete in the list which seems strange on a terraform plan run. These might be perms that TF is trying to use (probe) and expecting/ignoring failures. Is there a way to get more data on the permissions used, such as the result of the call (succes/failure)?

[iann0036]

Hey @joemiller,

Thanks for the detailed response, that's super insightful.

According to the datasource, only the following methods should produce the orgpolicy.policy.set permission:

• cloudresourcemanager.folders.clearOrgPolicy
• cloudresourcemanager.folders.setOrgPolicy
• cloudresourcemanager.organizations.clearOrgPolicy
• cloudresourcemanager.organizations.setOrgPolicy
• cloudresourcemanager.projects.clearOrgPolicy
• cloudresourcemanager.projects.setOrgPolicy

So I would also find it weird that the Terraform provider would attempt that.

There's an outstanding issue with intercepting responses, however I've cut a new version (1.1.3) which includes a --debug flag which will dump the full request and disable terminal clearing so you can hopefully narrow down the call.

Let me know how this goes.

[joemiller]

@iann0036 I am trying out the latest release with --debug. I can see a series of POST /v1/projects/<REDACTED>:getIamPolicy?alt=json&prettyPrint=false HTTP/1.1 early in the TF run and one of them results in the orgpolicy.policy.set perm being added to the list. I can't tell anything different between the calls as the headers and path are all the same.

EDIT: I also see {"options":{"requestedPolicyVersion":3}} in the body of the getIamPolicy req but not sure how or why that would matter (probably doesn't?)

https://cloud.google.com/resource-manager/reference/rest/Shared.Types/GetIamPolicyRequest#GetPolicyOptions

I added a debug print in handleGCPRequest() and this is what I get when the orgpolicy.policy.set is added:

POST /v1/projects/<REDACTED>:getIamPolicy?alt=json&prettyPrint=false HTTP/1.1
Host: cloudresourcemanager.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.5.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/4.84.0
Content-Length: 41
Accept-Encoding: gzip

{"options":{"requestedPolicyVersion":3}}

2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID:
2023/11/05 20:12:52  API ID: cloudresourcemanager.projects.clearOrgPolicy
[
    "iam.serviceAccounts.getOpenIdToken",
    "orgpolicy.policy.set",
    "resourcemanager.projects.get"
]

[iann0036]

Hey @joemiller,

Thanks for that detail. I've determined the presence of the : character in the matching path is conflicting with the URL matching library I'm currently using.

Will work on a fix shortly.

[iann0036]

Hey @joemiller,

I've updated the pattern matching logic to now uses Regex to avoid the issue with the : character as previously mentioned. An unfortunate but necessary performance hit - shouldn't be too bad for now though. This is released in v1.1.4.

Could you try the same now and let me know how you go?

[joemiller]

@iann0036 looks great! no more orgpolicy.* perms listed. This is a fairly big module that I am testing with and there are a lot of resources, but a quick glance of the resulting perms from iamlive looks much more plausible based on what I know to be in the module. 👍

I also ran 1.1.4 against a much smaller module (basically just https://registry.terraform.io/modules/terraform-google-modules/cloud-storage/google/latest/examples/simple_bucket) and the resulting permission set was much better. I had run 1.1.3 against this module and it did not output any storage.* perms, now it does.

[iann0036]

Awesome feedback, thanks!

Again, not every single permission is mapped out yet - but I'm working through it. Appreciate your time testing things out.

[valorl]

This is great, managed to get it to work using @joemiller's approach. I'm not sure if this is even a thing for other providers, but for me the tool would be much much more useful if it could somehow group the detected permissions by resource (or atleast request). A good example is, there is likely always going to be storage.objects.update in the list due to Terraform state backend, but the service account might not need to storage.objects.update anything else than that.

As a workaround I have to run with --debug and do some text analysis afterwards (diffing the permission output after each request).