Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add impersonate service account docs #323

Merged
merged 5 commits into from
Aug 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,4 @@ Please be aware of the following changes in v0.18.0. While breaking, we do not e
- [list (ls)](commands/list)
- [Redshift profile](redshift-profile#specifying-an-iam-profile)
- [`asset-paths` config](asset-paths) (also updated [dbt_project.yml](dbt_project.yml.md) and the [description](description) docs to match)
- [`impersonate_service_account` in the BigQuery profile configuration](https://docs.getdbt.com/reference/warehouse-profiles/bigquery-profile#service-account-impersonation)
48 changes: 48 additions & 0 deletions website/docs/reference/warehouse-profiles/bigquery-profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,54 @@ Database Error in model debug_table (models/debug_table.sql)
compiled SQL at target/run/bq_project/models/debug_table.sql
```

### Service Account Impersonation
<Changelog>New in v0.18.0</Changelog>
You may want your models to be built using a dedicated service account that has
bbhoss marked this conversation as resolved.
Show resolved Hide resolved
elevated access to read or write data to the specified project or dataset.
Typically, this requires you to create a service account key for running under
development or on your CI server. By specifing the email address of the service
account you want to build models as, you can use [Application Default Credentials](
https://cloud.google.com/sdk/gcloud/reference/auth/application-default) or the
service's configured service account (when running in GCP) to assume the identity
of the service account with elevated permissions.

This allows you to reap the advantages of using federated identity for developers
(via ADC) without needing to grant individual access to read and write data
directly, and without needing to create separate service account and keys for
each user. It also allows you to completely eliminate the need for service
account keys in CI as long as your CI is running on GCP (Cloud Build, Jenkins,
GitLab/Github Runners, etc).

To use this functionality, first create the service account you want to
impersonate. Then grant users that you want to be able to impersonate
this service account the `roles/iam.serviceAccountTokenCreator` role on
the service account resource. Then, you also need to grant the service
account the same role on itself. This allows it to create short-lived
tokens identifying itself, and allows your human users (or other service
accounts) to do the same. More information on this scenario is available
[here](https://cloud.google.com/iam/docs/understanding-service-accounts#directly_impersonating_a_service_account).

Once you've granted the appropriate permissions, you'll need to enable
the [IAM Service Account Credentials API](https://console.cloud.google.com/apis/library/iamcredentials.googleapis.com).
Enabling the API and granting the role are eventually consistent operations,
taking up to 7 minutes to fully complete, but usually fully propagating within 60
seconds. Give it a few minutes, then add the `impersonate_service_account`
option to your BigQuery profile configuration:

```yaml
my-profile:
target: dev
outputs:
dev:
type: bigquery
method: oauth
project: abc-123
dataset: my_dataset
impersonate_service_account: dbt-runner@yourproject.iam.gserviceaccount.com
bbhoss marked this conversation as resolved.
Show resolved Hide resolved
```

For a general overview of this process, see the official docs for [Creating Short-lived Service Account Credentials](https://cloud.google.com/iam/docs/creating-short-lived-service-account-credentials).

## Required permissions

BigQuery's permission model is dissimilar from more conventional databases like Snowflake and Redshift. The following permissions are required for dbt user accounts:
Expand Down