bug/email-metrics-variable

[fivetran-catfritz]

PR Overview

This PR will address the following Issue/Feature:
#102

This PR will result in the following new package version:

v0.9.1 - This is non-breaking as it will not affect model outputs.

Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:
The models listed below generate metrics by iterating through a list of fields:

• hubspot__contacts
• hubspot__email_campaigns
• int_hubspot__email_metrics__by_contact_list

This list of metrics is set by default, which means that some of the values in this default list may not exist in upstream tables if their sources are disabled. This can result in a run error.

To address this issue, I created a macro that checks for the presence of these extra columns and removes them from the list. Note that the default list has been moved to the macro to remove setting the variable within dbt_project.yml to avoid any potential conflicts with a user's dbt_project.yml.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

• dbt compile
• dbt run --full-refresh
• dbt run
• dbt test
• dbt run --vars '{hubspot_email_event_status_change_enabled: false}'

Before marking this PR as "ready for review" the following have been applied:

• The appropriate issue has been linked and tagged
• You are assigned to the corresponding issue and this PR
• BuildKite integration tests are passing

Detailed Validation

Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":

• You have validated these changes and assure this PR will address the respective Issue/Feature.
• You are reasonably confident these changes will not impact any other components of this package or any dependent packages.
• You have provided details below around the validation steps performed to gain confidence in these changes.

• Reproduce the issue by using the test data that does not include the source email_event_status_change (see internal ticket for details) and setting hubspot_email_event_status_change_enabled: false. You can only reproduce the error when this table is not present.
• After making the code adjustments, the error no longer occurs, and the compiled code does not show the missing columns.
• (Pending customer feedback.)

Standard Updates

Please acknowledge that your PR contains the following standard updates:

• Package versioning has been appropriately indexed in the following locations:
  • indexed within dbt_project.yml
  • indexed within integration_tests/dbt_project.yml
• CHANGELOG has individual entries for each respective change in this PR
• [NA] README updates have been applied (if applicable)
• [NA] DECISIONLOG updates have been updated (if applicable)
• [NA] Appropriate yml documentation has been added (if applicable)

dbt Docs

Please acknowledge that after the above were all completed the below were applied to your branch:

• docs were regenerated (unless this PR does not include any code or yml updates)

If you had to summarize this PR in an emoji, which would it be?

🚜

[fivetran-reneeli]

Tested with the configs you sent and it ran on my end, and the macro makes sense!