when using WLIF getting error : "missing field client_id"

[rojomisin]

I'm running gcs-rsync from an ec2 instance which auth's w/ gcp via workload identity.

How would I set the GOOGLE_CLIENT_ID or client_id?

gcs-rsync -u gs://bucket-name/file.test /tmp

Error: StorageError(GcsTokenError(HttpError(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("metadata.google.internal")), port: None, path: "/computeMetadata/v1/instance/service-accounts/default/token", query: None, fragment: None }, source: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" })) })))

gcs-rsync -u gs://bucket-name/file.test /tmp

Error: StorageError(GcsTokenError(EnvVarError { key: "GOOGLE_APPLICATION_CREDENTIALS", error: NotPresent }))
[ec2-user@ip-10-50-5-146 ~]$ export GOOGLE_APPLICATION_CREDENTIALS=/home/ec2-user/credentials.json
You have new mail in /var/spool/mail/ec2-user

gcs-rsync -u gs://bucket-name/file.test /tmp

Error: StorageError(GcsTokenError(DeserializationError { expected_type: "gcs_rsync::gcp::oauth2::token::AuthorizedUserCredentials", error: Error("missing field `client_id`", line: 14, column: 1) }))

cat credentials.json

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/<my-project-id-number>/locations/global/workloadIdentityPools/amzn/providers/aws-provider",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
    "imdsv2_session_token_url": "http://169.254.169.254/latest/api/token"
  },
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/my-sa@my-project.iam.gserviceaccount.com:generateAccessToken"
}

[cboudereau]

Hi.

Thank you for the quality of that issue.

You can use authorized user or service account json.
Those methods work on CI and local box for instance.

According to gcp doc, it seems that your aws ec2 is running out of gcp.

This is why the Google Metadata api is not available (ec2 is on aws).

Actually, gcp Metadata api, authorized user and service account auth are available. So do not use the -u flag instead use the env var
GOOGLE_APPLICATION_CREDENTIALS=./credentials.json

To create the credentials.json file, you can follow this setup:
https://github.com/cboudereau/gcs-rsync/tree/main/src/gcp/oauth2

In my previous experience, we were using vault or secret manager.

I am gonna check how to use the oauth external_account (from your json files) to plan a new oauth integration in the future.