Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support BigQuery Labels #1947

Closed
kconvey opened this issue Nov 21, 2019 · 1 comment
Closed

Support BigQuery Labels #1947

kconvey opened this issue Nov 21, 2019 · 1 comment
Labels
duplicate This issue or pull request already exists

Comments

@kconvey
Copy link
Contributor

kconvey commented Nov 21, 2019

Describe the feature

Allow for tables to be configured in dbt_project.yml with BigQuery labels. Ex.

models:
  fishtown_analytics:
    labels: {town: 'fish', analytics: 'yes'}
  
    events:
      labels:
       town: 'less_fish'
       analytics: ''

Implementing this should look very similar to kms key encryption in #1829

Describe alternatives you've considered

It may be possible to do this as a post-processing step outside of dbt or post-hook macro with some more adapter functionality implemented.

Additional context

Labels can be specified in the 'OPTIONS' parameter of 'create or replace' ddl like OPTIONS(labels=[('town': 'fish')]). 'OPTIONS' requires the labels to be in the form 'ARRAY<STRUCT<STRING, STRING>>', which associates label keys with their values, making a dictionary the logical form for configuring this in dbt_project.yml.

One quirk with needing to configure this as a dictionary is that the project parser seems to think that 'labels' is a model without a matching sql file and will give the warning:
WARNING: Configuration paths exist in your dbt_project.yml file which do not apply to any resources. There are 1 unused configuration paths: - models.labels
which will not prevent this from working in most cases, but can be confusing. It may be necessary to tweak the project parser to avoid this (could use some guidance on the best way to go about this).

Who will this benefit?

Anyone who uses BigQuery and may need to have labels for their models.

@kconvey kconvey added enhancement New feature or request triage labels Nov 21, 2019
@drewbanin drewbanin added duplicate This issue or pull request already exists and removed enhancement New feature or request triage labels Nov 26, 2019
@drewbanin
Copy link
Contributor

Less fish? I don't buy it :)

Your thinking here is spot-on -- we can add labels as a known config for BQ here: https://github.com/fishtown-analytics/dbt/blob/e51c942e91a94936f68f2965963d3b46f1257658/plugins/bigquery/dbt/adapters/bigquery/impl.py#L51-L52

This should avoid the warning you noted above.

I'm going to close this one as a dupe of #1942 but I am very interested in supporting this in the near future!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants