v0.4.10 dbt_fivetran_utils

Bug Fix

• This release addresses a bug caused by the introduction of the date_spine() macro. Users who employed the Fivetran Utils dispatch recommendations reported a recursion error that was resolved by ensuring that this macro had a different name from dbt_utils.date_spine().
  • Thus, the date_spine() macro has been renamed to fivetran_date_spine() (PR #136).

Contributors

• @nydnarb: Huge shoutout to Brandyn Lee for helping us troubleshoot and resolve this quickly.

v0.4.9 dbt_fivetran_utils

Feature Update

• Added macro extract_url_parameter to create special logic for Databricks instances not supported by dbt_utils.get_url_parameter(). The macro uses dbt_utils.get_url_parameter() for default, non-Databricks targets. See README for more details (PR #130).
• Made the try_cast() macro compatible with SQL Server (PR #131).
• Made the json_parse() macro compatible with SQL Server (PR #131).
• Added a date_spine() macro (PR #131).
  • For non-SQL Server databases, this will simply call dbt_utils.date_spine().
  • For SQL Server targets, this will manually create a spine, with code heavily leveraged from tsql-utils.date_spine() but adjusted for recent changes to dbt_utils.

Bug Fix

• Corrected the name of the default version of try_cast() from safe_cast to try_cast (PR #131).

Full Changelog: v0.4.8...v0.4.9

v0.4.8 dbt_fivetran_utils

PR #127 includes the following updates.

Feature Updates

• Updated the collect_freshness macro for compatibility with dbt versions greater than v1.5.0 to prevent erroneous warnings from occurring during freshness tests.

Full Changelog: v0.4.7...v0.4.8

v0.4.7 dbt_fivetran_utils

PR #118 includes the following updates.

Feature Updates

• Update to the union_data macro to ensure a source connection is established when just one schema is being used with the macro. (code link for reference)
  • The previous build did allow the macro to read data from the correct location; however, it did not leverage the source. Therefore, the models using this macro would result in not having a source connection. This has since been updated in this release and all versions of this macro (whether unioning or not) will attempt to leverage the source function.
• In addition to the above update, a specific code adjustment was provided for the LinkedIn Company Pages and Instagram Business Pages packages as the sources are not named the same as their default schema. This was the core issue that resulted in dbt_fivetran_utils v0.4.5 being rolled back. This has now been addressed and may be expanded in the future for any other packages with mismatching source/default_schema names. (code link for reference)
  • For additional context, this is the approach taken as renaming the sources could result in incompatible versions of base package and fivetran_utils. Particularly, this could result in package users having a different experience based on the version of their base package. Which is not the behavior we would prefer for this new feature.

Under the Hood

• The union_data macro has been adjusted to leverage the identifier variable and the true relation when running on a single schema. This ensures a more accurate use of the macro and the ability for the macro to work as expected within the Fivetran integration test pipeline.
• Adjusted the union_data macro to leverage the true relation obtained from the macro as opposed to leveraging the default variable when defining the single schema relation. (code link for reference)
• The .buildkite folder has been overhauled to ensure full integration test coverage over all Fivetran dbt packages that reference the dbt_fivetran_utils package. The new pipeline updates will ensure the fivetran_utils release version will succeed and not cause errors on any Fivetran dependent package prior to release.
• Cleaned up a variety of integration test configurations that are no longer needed with the new approach to integration tests for Fivetran Utils.

Full Changelog: v0.4.6...v0.4.7

v0.4.6 dbt_fivetran_utils

Bug Fixes

• Rolling back an error within the v0.4.5 release that caused a breaking change for the LinkedIn Pages dbt Package (#113)

Full Changelog: v0.4.5...v0.4.6

v0.4.5 dbt_fivetran_utils

Feature Updates

PR #110 includes the following feature updates:

• Update to the union_data macro to ensure a source connection is established when just one schema is being used with the macro.
  • The previous build did allow the macro to read data from the correct location; however, it did not leverage the source. Therefore, the models using this macro would result in not having a source connection. This has since been updated in this release and all versions of this macro (whether unioning or not) will attempt to leverage the source function.

Full Changelog: v0.4.4...v0.4.5

v0.4.4 dbt_fivetran_utils

Feature Updates

PR #106 introduces the following two new macros:

• drop_schemas_automation
• wrap_in_quotes

v0.4.3...v0.4.4

v0.4.3 dbt_fivetran_utils

Feature Updates

• (PR #100) Expanded the union_data macro to create an empty table if none of the provided schemas or databases contain a source table. If the source table does not exist anywhere, union_data will return a completely empty table (ie limit 0) with just one string column (_dbt_source_relation) and raise a compiler warning message that the respective staging model is empty.
  • The compiler warning can be turned off by the end user by setting the fivetran__remove_empty_table_warnings variable to True.

Under the Hood

• Added dbt compile tests for popular Fivetran packages in integration testing. (PR #101)

v0.4.2...v0.4.3

v0.4.2 dbt_fivetran_utils

Bug Fixes

• Fix broken anchor tags in README. (PR #96)
• Expand spark adapter logic in the fill_staging_columns macro to databricks adapters. (PR #98)

Contributors

• @kylemcnair (PR #96)

dbt_fivetran_utils v0.4.1

Happy Wednesday and international monkey day 🐒

This release of the dbt_fivetran_utils package includes the following updates:

Bug Fixes

• Updates dispatch from {{ dbt_utils.<macro> }} to {{ dbt.<macro> }} for additional cross-db macros missed in the fivetran_utils.union_relations() macro. (PR #91)

Updates

• Updates the pivot_json_extract macro to include additional formats of fields using a new name and alias argument to be pivoted out into columns. Specifically, allowing for fields with . to be replaced with _, for metadata variables to accept dictionaries in addition to strings, and for aliasing of fields. (PR #92)
• Updates the add_pass_through_columns and fill_pass_through_columms macros to be backwards compatible for lists of pass through columns (ie those that do not have attributes like name, alias, or transform_sql) (PR #93).
• packages.yml update for dbt_utils from 1.0.0-b2 to [">=1.0.0", "<2.0.0"]