Jib: seamless and automatic GCR (gcr.io) auth on GCP using Application Default Credentials (ADC): Google Cloud Build (GCB), Google Compute Engine (GCE), Google Kubernetes Engine (GKE), Cloud SDK application-default, service accounts, GOOGLE_APPLICATION_CREDENTIALS, etc

[chanseokoh]

Creating an issue for easy lookup and search as well as for future reference.

On GCP, if no credentials were configured or could be retrieved for GCR access, Jib should try ADC that are available on most GCP environments as a last attempt. This will effectively resolve the case like #1500.

UPDATE: this feature is live and supported for Jib >= 1.5.0

ADC from Cloud SDK

Jib can get ADC set through gcloud auth application-default login. ADC in this case are "end user credentials". (Note that gcloud auth login won't work, as it isn't ADC.)

WARNING: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/.
[DEBUG] Google ADC found
[DEBUG] Using Google Application Default Credentials for gcr.io
...
[INFO] Built and pushed image as gcr.io/<redacted>/test
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

ADC from Service Account

The current implementation for a service account is to generate a short-lived OAuth2 access token from the service account and use it for GCR auth.

$ GOOGLE_APPLICATION_CREDENTIALS=my-service-account-key.json mvn -X jib:build
...
[DEBUG] Google ADC found
[DEBUG] ADC is a service account. Set GCS read-write scope
[DEBUG] Using Google Application Default Credentials for gcr.io
...
[INFO] Built and pushed image as gcr.io/<redacted>/test
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

ADC on Google Cloud Build (GCB)

ADC are available on GCB and have the permission to push to GCR.

[DEBUG] Google ADC found
[DEBUG] Using Google Application Default Credentials for gcr.io
...
[INFO] Built and pushed image as gcr.io/<redacted>/gcb-test
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
...
PUSH
DONE
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

ID                                    CREATE_TIME                DURATION  SOURCE                                                                                             IMAGES  STATUS
0b9e0bb2-3cdb-4e7d-8954-86d5ba0b7e72  2019-08-12T21:08:27+00:00  1M55S     gs://<redacted>_cloudbuild/source/1565644104.23-97f5c662bd70409186821171c3a06db0.tgz  -       SUCCESS

ADC on Google Compute Engine (GCE)

Tested on Linux and Windows:

[DEBUG] Google ADC found
[DEBUG] Using Google Application Default Credentials for gcr.io
...
[INFO] Built and pushed image as gcr.io/<redacted>/adc-gce
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

ADC on Cloud Shell

Confirmed it works. However, Cloud Shell has docker-credential-gcloud, docker-credential-gcr, and ~/.docker/config.json, so it's already working pretty well without this ADC support.

[DEBUG] Google ADC found
[DEBUG] Using Google Application Default Credentials for gcr.io

ADC on Cloud Run

To be tested.

[chanseokoh]

ADC on Google Kubernetes Engine (GKE)

GKE does not follow the same model as other GCP environments regarding the default ADC.

ADC is available on GKE, as the node pool is by default associated with the "GCE default service account" (configurable), the same account used for GCE instances.

Jib can pick up the ADC and pull private images from GCR.

[DEBUG] Google ADC found
[DEBUG] Using Google Application Default Credentials for gcr.io

However, the node pool is created with a limited access scope by default, so Jib won't be able to push to GCR out of the box.

For the ADC to have permissions to push to GCR, you need to add at least the "Storage Read Write" scope when creating the node pool.

However, this may not necessarily be an ideal way to make Jib auth with GCR. Jib has many other ways to retrieve credentials. So, for example, you could

1. generate/download a JSON key file from a service account;
2. save the JSON content as a k8s Secret; and
3. configure the cluster to expose the k8s Secret through the GOOGLE_APPLICATION_CREDENTIALS environment variable. Then Jib will automatically recognize the environment variable as ADC.

See a more comprehensive doc about using a k8s Secret to pull in a service account key this way.

However, the above alternative to use the k8s Secret may not necessarily be the best solution either, since it requires you to download and manage a service account key (user-managed service account key). We are currently exploring another option to make Jib automatically utilize Workload Identity on GKE.

Also note that GCR has multiple authentication methods, which includes directly passing the JSON key content as a password. The username in this case should be the literal _json_key.