Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add pass through macro #22

Merged
merged 5 commits into from
Apr 1, 2021
Merged

Conversation

fivetran-joemarkiewicz
Copy link
Contributor

What change does this PR introduce?

The addition of the add_pass_through_columns macro. This macro allows a user to declare a pass through variable in the following format:

dbt_project.yml

...

vars:

  deal_pass_through_columns:
  
        - name: "property_sales_development_representative"
          alias: "sales_development_representative_owner_id"
  
        - name: "property_energy_consultant"
          alias: "energy_consultant_owner_id"

If this PR introduces a new macro, how did you test the new macro?

This macro was tested within the dbt_hubspot_source package and was originally created by @jamesrayoub within the respective PR

If this PR introduces a modification to an existing macro, which packages is the macro currently present in and what steps were taken to test compatibility across packages?

This will only be used within the dbt_hubspot_source package. However, it can be applied to other packages that accept pass through fields as variables.

Did you update the README to reflect the macro addition/modifications?

  • Yes
  • No (provide further explanation)

@fivetran-joemarkiewicz
Copy link
Contributor Author

An additional macro was added to allow for the code within the staging model to remain dry and dynamic. The fill_pass_through_columns macro was tested and documented.


{% for column in pass_through_var %}

{% do base_columns.append({ "name": column.name, "datatype": column.datatype, "alias": column.alias }) if column.alias else base_columns.append({ "name": column.name, "datatype": column.datatype}) %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we removing the datatype attribute from this part? forgot where we left off

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think it makes the most sense to remove it as it is not needed and adds unnecessary complexity. Removed, thanks!

Copy link
Contributor

@fivetran-jamie fivetran-jamie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! i would just maybe add some more documentation for the macros (like in what types of files they're called in) but the code is all good

@fivetran-joemarkiewicz fivetran-joemarkiewicz merged commit 7c0aa63 into master Apr 1, 2021
@fivetran-joemarkiewicz fivetran-joemarkiewicz deleted the add-passthrough-macro branch April 1, 2021 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants