Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Barbara Gittings] Docs generation fails if a database/schema name is numeric-looking #2206

Closed
1 of 5 tasks
drewbanin opened this issue Mar 17, 2020 · 1 comment
Closed
1 of 5 tasks
Labels
bigquery bug Something isn't working

Comments

@drewbanin
Copy link
Contributor

drewbanin commented Mar 17, 2020

Describe the bug

Tagging this one with the BigQuery label because that's where I've seen it, but it could conceivably happen on any plugin.

If a database or schema name is numeric, then Agate's type inference will make the database_name or schema_name column (unique to a given agate table) numeric during catalog generation. This causes some calls to database_name.lower() or schema_name.lower() to fail with an error:

encountered an error while generating catalog: 'decimal.Decimal' object has no attribute 'lower'

I think I narrowed this down to _relations_filter_schemas in the base impl.py, but it's possible that there are other places where we assume the database name is going to be a string.

Beyond this first issue, Agate will also be mad about unioning together different agate tables that were fetched in parallel if one of those columns is string-y and the other is number-y in nature.

dbt should ensure that all of the known-string columns are returned as strings (and not numbers, if they look numeric) when fetching information from the system catalog.

Steps To Reproduce

Use the profile:

    dev:
      type: bigquery
      method: oauth
      project: '{project name}'
      dataset: "1991"
      threads: 1

Then run:

dbt run
dbt docs generate

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

0.16.0-rc2
@drewbanin drewbanin added bug Something isn't working bigquery labels Mar 17, 2020
@drewbanin drewbanin added this to the Barbara Gittings milestone Mar 17, 2020
@drewbanin drewbanin changed the title [Barbara Gittings] Docs generation fails if a database name is numeric-looking [Barbara Gittings] Docs generation fails if a database/schema name is numeric-looking Mar 17, 2020
beckjake added a commit that referenced this issue Mar 17, 2020
…-catalog

Fix issue with table databases/schemas/names becoming numbers (#2206)
@beckjake
Copy link
Contributor

Fixed in #2208

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bigquery bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants