v0.17.0 dbt_hubspot_source

Breaking Change (--full-refresh required after upgrading)

• Introduced a new category column to the below models. This association field differentiates records by either HUBSPOT_DEFINED (default label) or USER_DEFINED (custom label) and was introduced to the Hubspot connector in October 2024. See the connector release notes for more.

  • stg_hubspot__deal_company
  • stg_hubspot__deal_contact
  • stg_hubspot__engagement_company
  • stg_hubspot__engagement_contact
  • stg_hubspot__engagement_deal
  • stg_hubspot__ticket_company
  • stg_hubspot__ticket_contact
  • stg_hubspot__ticket_deal
  • stg_hubspot__ticket_engagement

• Added explicit casting of pipeline_id to string in stg_hubspot__deal_pipeline_stage to ensure successful downstream joins.

Bug Fixes

• Updated the unique key to include the new category field for the following models in order for the uniqueness tests to pass:
  • stg_hubspot__deal_contact
  • stg_hubspot__deal_company

Under the Hood

• Updated the respective seed files, get_[source_table]_column macros, and documentation of the mentioned models to include the category field.
• Removed property_hs_timestamp and property_hs_createdate field casting for the engagement_meeting source table in the integration_tests/dbt_project.yml file since they are not present in the seed data.

Full Changelog: v0.16.0...v0.17.0

v0.16.0 dbt_hubspot_source

PR #129 includes the following updates:

Breaking Changes

• Switched from using the fivetran_utils.remove_prefix_from_columns macro to the hubspot_source.remove_duplicate_and_prefix_from_columns macro for when hubspot__pass_through_all_columns is enabled and you are passing through all columns in the stg_hubspot__company, stg_hubspot__contact, and stg_hubspot__deal models. This also ensures the source fields passed through are all quoted from the onset. This is a breaking change because this macro can remove duplicate fields, resulting in an impact to your schema.

Bug Fixes

• Introduced hubspot-specific version of the fivetran_utils.pass_through_columns macro titled hubspot_add_pass_through_columns, which introduces quoting around the source fields being brought in as passthrough columns. This will ensure that your warehouse reads the sql correctly, particularly if the field contains special characters or syntax. This is now used in the respective get_<>_columns macros of the following models:
  • stg_hubspot__company
  • stg_hubspot__contact
  • stg_hubspot__deal
  • stg_hubspot__ticket

Under the Hood

• Updated seed data to include fields with special syntax in order to test the above changes.

Full Changelog: v0.15.0...v0.16.0

v0.15.0 dbt_hubspot_source

PR #126 includes the following updates:

🚨 Breaking Changes 🚨

• Added field _fivetran_end to macro get_ticket_property_history_columns() to ensure the column is available for use downstream.
  • While this should not affect most users, this will add a new column _fivetran_end to stg_hubspot__ticket_property_history if you do not have _fivetran_end in your source TICKET_PROPERTY_HISTORY table.

Documentation

• Update documentation to include _fivetran_end under model stg_hubspot__ticket_property_history.

Under the hood

• Updated the maintainer PR template to the current format.
• Included auto-releaser GitHub Actions workflow to automate future releases.
• Added integration testing for Databricks SQL Warehouse.

Full Changelog: v0.14.1...v0.15.0

v0.14.1 dbt_hubspot_source

PR #125 includes the following updates:

Features

• Adds the stg_hubspot__merged_deal model. Downstream, this model is used to remove stale deals and aggregate them to the deal they have been merged into. This is disabled by default as not every user has the merged_deal table.
  • To enable this model and filter out merged deals downstream, make sure the following is set to true in your dbt_project.yml: hubspot_sales_enabled, hubspot_deal_enabled, and hubspot_merged_deal_enabled. See Step 4 of the README for more details.

Full Changelog: v0.14.0...v0.14.1

v0.14.0 dbt_hubspot_source

PR #122 includes the following updates:

Features

• Added the following staging models, along with documentation and tests:

  • stg_hubspot__property
  • stg_hubspot__property_option
  • These tables can be disabled by setting hubspot_property_enabled: False in your dbt_project.yml vars. See Step 4 of the README for more details.

• When including a passthrough property_hs_* column, you now have the option to include the corresponding, human-readable label in the staging models.

  • The above-mentioned property tables are required for this feature. If you do not have them and have to disable them, unfortunately you will not be able to use this feature.
  • See the Adding property label section of the README for instructions on how to enable this feature!
  • We recommend being selective with the label columns you add. As you add more label columns, your run time will increase due to the underlying logic requirements.
  • This update applies to models:
    • stg_hubspot__company
    • stg_hubspot__contact
    • stg_hubspot__deal
    • stg_hubspot__ticket

Bug fixes

• Updated macro remove_duplicate_and_prefix_from_columns to accommodate incoming custom column names containing characters such as - or $ that are not permitted. The resulting column name will have these characters removed or replaced in its stg_* model.
• Removed extra comma from stg_hubspot__ticket, which was causing compilation issues when passing through all columns.

Full Changelog: v0.13.0...v0.14.0

v0.13.0 dbt_hubspot_source

🚨 Breaking Changes 🚨

• The created_at and closed_at fields in the below mentioned staging models have been renamed to created_date and closed_date respectively to be consistent with the source data. Additionally, this will ensure there are no duplicate column errors when passing through all property_* columns, which could potentially conflict with property_created_at or property_closed_at (PR #119).
  • stg_hubspot__company
  • stg_hubspot__contact
  • stg_hubspot__deal
  • stg_hubspot__ticket

Features

• Addition of the following variables to allow the disabling of the *_property_history models if they are not being leveraged. All variables are true by default (PR #119).
  • hubspot_company_property_history_enabled
  • hubspot_contact_property_history_enabled
  • hubspot_deal_property_history_enabled

Under the Hood

• All stg_hubspot__*_tmp models have been updated to leverage the dbt_utils.star() macro. This ensures if the source dimension changes there is no potential for a mismatch in columns error that is commonly seen in Snowflake destinations (PR #119).
• Updates to the seed files and seed file configurations for the package integration tests to ensure updates are properly tested (PR #119).
• V3 of the Hubspot Service endpoint contains the field _fivetran_deleted in the TICKET table, whereas V2 has a field called is_deleted. Previously, the package only looked at is_deleted. Now, it will fold in _fivetran_deleted and coalesce it with is_deleted to more fully represent is_ticket_deleted (PR #120).

Full Changelog: v0.12.0...v0.13.0

Parallel Transform Package Release

v0.12.0 dbt_hubspot_source

🚨 Breaking Changes 🚨

• The following models now use a custom macro to remove the property_hs_ prefix in staging columns, while also preventing duplicates. If de-prefixed columns match existing ones (e.g., property_hs_meeting_outcome vs. meeting_outcome), the macro favors the property_hs_field, aligning with the latest HubSpot API update. (PR #115)
  • stg_hubspot__engagement_call
  • stg_hubspot__engagement_company
  • stg_hubspot__engagement_contact
  • stg_hubspot__engagement_deal
  • stg_hubspot__engagement_email
  • stg_hubspot__engagement_meeting
  • stg_hubspot__engagement_note
  • stg_hubspot__engagement_task
  • stg_hubspot__ticket
  • stg_hubspot__ticket_company
  • stg_hubspot__ticket_contact
  • stg_hubspot__ticket_deal
  • stg_hubspot__ticket_engagement
  • stg_hubspot__ticket_property_history

Feature Updates

• A new macro remove_duplicate_and_prefix_from_columns has been included which expands off the fivetran_utils.remove_prefix_columns macro by removing any duplicate columns that result from the prefix removal. (PR #115 and PR #114)

Contributors

• @greg-finley (PR #114)

Full Changelog: v0.11.0...v0.12.0

v0.11.0 dbt_hubspot_source

PR #112 includes the following updates:

🚨 Breaking Changes 🚨

• Following the May 2023 connector update the HubSpot connector now syncs the below parent and child tables from the new v3 API. As a result the dependent fields and field names from the downstream staging models have changed depending on the fields available in your HubSpot data. Now the respective staging models will sync the required fields for the dbt_hubspot downstream transformations and all of your property_hs_* fields. Please be aware that the property_hs_* will be truncated from the field name in the staging and downstream models. The impacted sources (and relevant staging models) are below:

  - `ENGAGEMENT`
    - `ENGAGEMENT_CALL`
    - `ENGAGEMENT_COMPANY`
    - `ENGAGEMENT_CONTACT`
    - `ENGAGEMENT_DEAL`
    - `ENGAGEMENT_EMAIL`
    - `ENGAGEMENT_MEETING`
    - `ENGAGEMENT_NOTE`
    - `ENGAGEMENT_TASK`
  - `TICKET`
    - `TICKET_COMPANY`
    - `TICKET_CONTACT`
    - `TICKET_DEAL`
    - `TICKET_ENGAGEMENT`
    - `TICKET_PROPERTY_HISTORY`

• Please note that while these changes are breaking, the package has been updated to ensure backwards compatibility with the pre HubSpot v3 API updates. As a result, you may see some null fields which are artifacts of the pre v3 API HubSpot version. Be sure to inspect the relevant field descriptions for an understanding of which fields remain for backwards compatibility purposes. These fields will be removed once all HubSpot connectors are upgraded to the v3 API.

Documentation Updates

• As new fields were added in the v3 API updates, and old fields were removed, the documentation was updated to reflect the v3 API consistent fields. Please take note if you are still using the pre v3 API, you will find the above mentioned source (and respective staging) models no longer have complete field documentation coverage.

Full Changelog: v0.10.0...v0.11.0

v0.10.0 dbt_hubspot_source

This release of dbt_hubspot_source includes:

🚨 Breaking Changes 🚨

• In the May 2023 connector update type_id was added to sources DEAL_COMPANY and DEAL_CONTACT as a part of the composite primary key for these tables. This column has been adding to the corresponding staging models. (PR #109)
• Updated tests for these tables with type_id as part of the primary key. (PR #109)
• Also resulting from the connector update, columns updated_at and created_at have been added to the following sources and their corresponding staging models: (PR #109)
  • DEAL_PIPELINE
  • DEAL_PIPELINE_STAGE
  • TICKET_PIPELINE
  • TICKET_PIPELINE_STAGE
• Updated docs with these changes and a little housekeeping. (PR #109)

Feature Updates

• Updated README to include the variable hubspot_owner_enabled. (PR #109)

🚘 Under the Hood

• Updated seed data for testing newly added columns. (PR #109)

Full Changelog: v0.9.1...v0.10.0

v0.9.1 dbt_hubspot_source

This release of dbt_hubspot_source includes:

Feature Updates

• A new variable was added hubspot_using_all_email_events to allow package users to remove filtered email events from the stg_hubspot__email_event staging model as well as the relevant downstream reporting models. This is crucial for HubSpot users who greatly take advantage of marking events as filtered in order to provide accurate reporting. (PR #104)
  • The hubspot_using_all_email_events variable is true by default. Set the variable to false to filter out specified email events in your staging and downstream models.

Under the Hood

• The email_event_data.csv seed file was updated to include events that are listed as true for filtered_events. This is to effectively test the above mentioned feature update. (PR #104)
• Included hubspot_using_all_email_events: false as a variable declared in the final run_models.sh step to ensure our integration tests gain coverage over this new feature and variable. (PR #104)
• Incorporated the new fivetran_utils.drop_schemas_automation macro into the end of each Buildkite integration test job. (PR #103)
• Updated the pull request templates. (PR #103)

Full Changelog: v0.9.0...v0.9.1