Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: snapshots should create relations with database=none #95

Merged
merged 1 commit into from
Jun 10, 2020

Conversation

jtcohen6
Copy link
Contributor

@jtcohen6 jtcohen6 commented Jun 10, 2020

Problem

The snapshot materialization twice needs to get or make a relation. Previously, it set the database by grabbing the value of the existing/model relation's database. This resulting in an error:

hologram.ValidationError: Undefined is not of type 'string'

Failed validating 'type' in schema[0]:
    {'type': 'string'}

On instance:
    Undefined

Solution

We should explicitly set database=none, in line with the changes in dbt-spark==0.17.0 that preclude database from having a value.

Note on testing

I caught this in my local end-to-end testing of all current features. This wasn't caught in automated integration testing because:

  • dbt-integration-tests suite does not include a dbt snapshot step
  • snapshots are a Delta-only features, and current integration tests run only on dockerized Apache Spark

The resolution involves:

@jtcohen6 jtcohen6 requested a review from beckjake June 10, 2020 20:05
@jtcohen6 jtcohen6 changed the title Snapshots: create relations w/ db=none Fix: snapshots should create relations with database=none Jun 10, 2020
@jtcohen6 jtcohen6 merged commit b5c73d8 into master Jun 10, 2020
@kwigley kwigley deleted the fix/snapshot-db-none branch March 23, 2021 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants