v0.14.0 dbt_hubspot

🚨 Breaking Changes 🚨

• Within the source package the created_at and closed_at fields in the below mentioned staging models have been renamed to created_date and closed_date respectively to be consistent with the source data. Additionally, this will ensure there are no duplicate column errors when passing through all property_* columns, which could potentially conflict with property_created_at or property_closed_at. (PR #119)
  • stg_hubspot__company
    • Impacts hubspot__companies
  • stg_hubspot__contact
    • Impacts hubspot__contacts
  • stg_hubspot__deal
    • Impacts hubspot__deals
  • stg_hubspot__ticket
    • Impacts hubspot__tickets

New Model Alert 😮

Introducing Service end models! These are disabled by default but can be enabled by setting hubspot_service_enabled to true (PR #123):

• hubspot__tickets - Docs
• hubspot__daily_ticket_history - Docs
  • See additional configurations for the history model in README

Features

• Addition of the following variables to allow the disabling of the *_property_history models if they are not being leveraged. All variables are true by default. (PR #122)
  • hubspot_company_property_history_enabled
  • hubspot_contact_property_history_enabled
  • hubspot_deal_property_history_enabled

Under the Hood

• Updates to the seed files and seed file configurations for the package integration tests to ensure updates are properly tested. (PR #122)

Full Changelog: v0.13.0...v0.14.0

Parallel Source Package Release

v0.13.0 dbt_hubspot

🚨 Breaking Changes 🚨

• This release will be a breaking change due to the removal of below dependencies.

Dependency Updates

Removes the dependencies on dbt-expectations and dbt-date. (#118)

Under the Hood

Specifically we removed the dbt_expectations.expect_column_values_to_be_unique test that was used to validate uniqueness under given conditions for the primary keys among the end models. We will be working to replace this with a similar test.

Full Changelog: v0.12.0...v0.13.0

v0.12.0 dbt_hubspot

🚨 Breaking Changes 🚨

• This release includes breaking changes as a result of upstream changes within the v0.12.0 release of the dbt_hubspot_source package. Please see below for the relevant breaking change release notes from the source package. (PR #120)
• The following models in the dbt_hubspot_source package now use a custom macro to remove the property_hs_ prefix in staging columns, while also preventing duplicates. If de-prefixed columns match existing ones (e.g., property_hs_meeting_outcome vs. meeting_outcome), the macro favors the property_hs_field, aligning with the latest HubSpot API update. (PR #115)
  • stg_hubspot__engagement_call
  • stg_hubspot__engagement_company
  • stg_hubspot__engagement_contact
  • stg_hubspot__engagement_deal
  • stg_hubspot__engagement_email
  • stg_hubspot__engagement_meeting
  • stg_hubspot__engagement_note
  • stg_hubspot__engagement_task
  • stg_hubspot__ticket
  • stg_hubspot__ticket_company
  • stg_hubspot__ticket_contact
  • stg_hubspot__ticket_deal
  • stg_hubspot__ticket_engagement
  • stg_hubspot__ticket_property_history

Full Changelog: v0.11.0...v0.12.0

v0.11.0 dbt_hubspot

PR #114 includes the following updates:

🚨 Breaking Changes 🚨

This change is made breaking in part to updates applied to the upstream dbt_hubspot_source package following upgrades to ensure compatibility with the HubSpot v3 API updates. Please see below for the relevant upstream changes:

• Following the May 2023 connector update the HubSpot connector now syncs the below parent and child tables from the new v3 API. As a result the dependent fields and field names from the downstream staging models have changed depending on the fields available in your HubSpot data. Now the respective staging models will sync the required fields for the dbt_hubspot downstream transformations and all of your property_hs_* fields. Please be aware that the property_hs_* will be truncated from the field name in the staging and downstream models. The impacted sources (and relevant staging models) are below:

  - `ENGAGEMENT`
    - `ENGAGEMENT_CALL`
    - `ENGAGEMENT_COMPANY`
    - `ENGAGEMENT_CONTACT`
    - `ENGAGEMENT_DEAL`
    - `ENGAGEMENT_EMAIL`
    - `ENGAGEMENT_MEETING`
    - `ENGAGEMENT_NOTE`
    - `ENGAGEMENT_TASK`
  - `TICKET`
    - `TICKET_COMPANY`
    - `TICKET_CONTACT`
    - `TICKET_DEAL`
    - `TICKET_ENGAGEMENT`
    - `TICKET_PROPERTY_HISTORY`

• Please note that while these changes are breaking, the package has been updated to ensure backwards compatibility with the pre HubSpot v3 API updates. As a result, you may see some null fields which are artifacts of the pre v3 API HubSpot version. Be sure to inspect the relevant field descriptions for an understanding of which fields remain for backwards compatibility purposes. These fields will be removed once all HubSpot connectors are upgraded to the v3 API.

• The engagements_joined macro has been adjusted to account for the HubSpot v3 API changes. In particular, the fields that used to only be available in the engagements source are now available in the individual engagement_[type] sources. As such, coalesce statements were added to ensure the correct populated fields are used following the join. Further, to avoid ambiguous columns a dbt-utils.star was added to remove the explicitly declared columns within the coalesce statements.

  • The coalesce is only included for backwards compatibility. This will be removed in a future release when all HubSpot connectors are on the latest API version.

Under the Hood

• The base_model argument used for the engagements_joined macro has been updated to be an explicit ref as opposed to the previously used var within the below models:
  • hubspot__engagement_calls
  • hubspot__engagement_emails
  • hubspot__engagement_meetings
  • hubspot__engagement_notes
  • hubspot__engagement_tasks
    • This update was applied as the var result was producing inconsistent results during compile and runtime when leveraging the dbt-utils.star macro. However, the explicit ref always provided consistent results.

Documentation Updates

• As new fields were added in the v3 API updates, and old fields were removed, the documentation was updated to reflect the v3 API consistent fields. Please take note if you are still using the pre v3 API, you will find the following end models no longer have complete field documentation coverage:
  • hubspot__engagement_calls
  • hubspot__engagement_emails
  • hubspot__engagement_meetings
  • hubspot__engagement_notes
  • hubspot__engagement_tasks

Full Changelog: v0.10.1...v0.11.0

v0.10.1 dbt_hubspot

PR #113 includes the following updates:

🪲 Bug Fixes

• Explicitly casts join fields (engagement_id and deal_id) in hubspot__deals as the appropriate data types to avoid potential errors in joining.

Full Changelog: v0.10.0...v0.10.1

v0.10.0 dbt_hubspot

🚨 Breaking Changes 🚨

These changes are made breaking due to changes in the source.

• Columns updated_at and created_at were added to the following sources and their corresponding staging models in the source package:
  • DEAL_PIPELINE
  • DEAL_PIPELINE_STAGE
  • TICKET_PIPELINE
  • TICKET_PIPELINE_STAGE
• As a result, the following columns have been added (#111):
  • Model hubspot__deals:
    • deal_pipeline_created_at
    • deal_pipeline_updated_at
  • Model hubspot__deal_stages:
    • deal_pipeline_stage_created_at
    • deal_pipeline_stage_updated_at
• Documentation has also been updated with these new columns. (#111)

🎉 Feature Updates

• Updated README to include the variables hubspot_ticket_deal_enabled and hubspot_owner_enabled. (#111)

🚘 Under the Hood

• Modified the unnest logic in the merge_contacts macro for Redshift users to reduce runtime of the int_hubspot__contact_merge_adjust model. (#110)
• Updated seed data for testing newly added columns. (#111)

Contributors

• @kcraig-ats (#110)

See upstream hubspot_source release notes here.

Full Changelog: v0.9.1...v0.10.0

v0.9.1 dbt_hubspot

This release of dbt_hubspot includes:

🎉 Feature Updates

• A new variable was added hubspot_using_all_email_events to allow package users to remove filtered email events from the stg_hubspot__email_event staging model as well as the relevant downstream reporting models. This is crucial for HubSpot users who greatly take advantage of marking events as filtered in order to provide accurate reporting. (#104)
  • The hubspot_using_all_email_events variable is true by default. Set the variable to false to filter out specified email events in your staging and downstream models. (#104)

🪲 Bug Fixes

• Introduced new macro adjust_email_metrics to prevent failures that may occur when certain tables are disabled or enabled. This macro removes any metrics that are unavailable from the default email_metrics list, ensuring that the models runs smoothly. It's worth noting that you can still manually set the email_metrics list as described in the README's (Optional) Step 5: Additional configurations. (#105)

🚘 Under the Hood

• The email_event_data.csv seed file was updated to include events that are listed as true for filtered_events. This is to effectively test the above mentioned feature update. (#104)
• Included hubspot_using_all_email_events: false as a variable declared in the final run_models.sh step to ensure our integration tests gain coverage over this new feature and variable. (#104)
  • See the source package CHANGELOG for updates made to the staging layer in dbt_hubspot_source v0.9.1.
• Updated the following models to utilize the adjust_email_metrics macro (#105):
  • hubspot__contacts
  • hubspot__contact_lists
  • hubspot__email_campaigns
  • int_hubspot__email_metrics__by_contact_list
• Incorporated the new fivetran_utils.drop_schemas_automation macro into the end of each Buildkite integration test job. (#103)
• Updated the pull request templates. (#103)

Full Changelog: v0.9.0...v0.9.1

v0.9.0 dbt_hubspot

Happy wednesday!

🚨 Breaking Changes 🚨

In November 2022, the Fivetran Hubspot connector switched to v3 of the Hubspot CRM API, which deprecated the CONTACT_MERGE_AUDIT table and stored merged contacts in a field in the CONTACT table. This has not been rolled out to BigQuery warehouses yet. BigQuery connectors with the CONTACT_MERGE_AUDIT table enabled will continue to sync this table until the new CONTACT.property_hs_calculated_merged_vids field and API version becomes available to them.

This release introduces breaking changes around how contacts are merged in order to align with the above connector changes. It is, however, backwards-compatible.

PR #100 applies the following changes:

• Updates logic around the recently deprecated CONTACT_MERGE_AUDIT table.
  • The package now leverages the new property_hs_calculated_merged_vids field to filter out merged contacts. This is the default behavior for all destinations, including BigQuery (the package will run successfully but not actually merge any contacts). This is achieved by the package's new merge_contacts() macro.
  • Backwards-compatibility: the package will only reference the old CONTACT_MERGE_AUDIT table to merge contacts if hubspot_contact_merge_audit_enabled is explicitly set to true in your root dbt_project.yml file.
  • The int_hubspot__contact_merge_adjust model (where the package performs contact merging) is now materialized as a table by default. This used to be ephemeral, but the reworked merge logic requires this model to be either a table or view.

Under the Hood

PR #100 applies the following changes:

• Updates seed data to test new merging paradigm.

See the source package CHANGELOG for updates made to the staging layer in dbt_hubspot_source v0.9.0.

Full changlog: v0.8.2...v0.9.0

v0.8.2 dbt_hubspot

Happy Monday and international chopstick day 🥢

I personally love to eat all types of foods with chopsticks. My favorite gift I ever received was actually a pair of Light Saber chopsticks. I use them, and the force almost every day!

Bug Fixes

• Following the release of v0.8.0, the end model uniqueness tests were not updated to account for the added flexibility of the inclusion of deleted records. As such the respective end model tests have been adjusted to test uniqueness only on non-deleted records. (#94)

Under the Hood

• Addition of the dbt-expectations package to be used for more robust testing of uniqueness for end models. (#94)

Full Changelog: v0.8.1...v0.8.2

v0.8.1 dbt_hubspot

☕ Monday Morning Accidental Release ☕

This was an accidental release with no changes applied. Please see the full changelog below for evidence of no changes. The subsequent v0.8.2 release contains the intended changes that were committed from this release.

Full Changelog: v0.8.0...v0.8.1