Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] persist_docs not working for seeds #1211

Open
2 tasks done
dpguthrie opened this issue Apr 30, 2024 · 3 comments
Open
2 tasks done

[Bug] persist_docs not working for seeds #1211

dpguthrie opened this issue Apr 30, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@dpguthrie
Copy link

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The descriptions for the seed and columns are not being persisted to BQ when using the appropriate config.

seeds:
  [<resource-path>](https://docs.getdbt.com/reference/resource-configs/resource-path):
    +persist_docs:
      relation: true
      columns: true

Expected Behavior

The descriptions are populated at the object and column level for the seed in my dbt project.

Steps To Reproduce

  1. Use the yaml below to persist_docs for seeds
seeds:
  [<resource-path>](https://docs.getdbt.com/reference/resource-configs/resource-path):
    +persist_docs:
      relation: true
      columns: true
  1. Run dbt seed
  2. View descriptions in BQ for that seed

This may have something to do with the size of the seed though. From @mikealfare "It might also have something to do with the fact that tables are sharded in BQ, and we only pull the column descriptions from the most recent shard (because they can change)."

Relevant log output

No response

Environment

- OS:
- Python:
- dbt-core:
- dbt-bigquery:

Additional Context

No response

@dpguthrie dpguthrie added bug Something isn't working triage labels Apr 30, 2024
@jtcohen6
Copy link
Contributor

It looks like we persist top-level descriptions for seeds, but not column-level descriptions:

{% if config.persist_relation_docs() and 'description' in model %}
{{ adapter.update_table_description(model['database'], model['schema'], model['alias'], model['description']) }}
{% endif %}

I'm not sure why that is - this implementation goes way back:

Screenshot 2024-05-14 at 13 46 57

@dpguthrie
Copy link
Author

@jtcohen6 that's interesting, I was able to persist my column descriptions:

image

@dpguthrie
Copy link
Author

@jtcohen6 @amychen1776 My customer is asking if we're able to prioritize this at all?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants