Merge pull request #79 from fivetran/feature/disable-ticket-deal

Feature/disable ticket deal and apply DEAL property renaming fix

CHANGELOG.md

@@ -1,5 +1,9 @@
 # dbt_hubspot_source v0.5.8
+## Features
+- Adds `hubspot_ticket_deal_enabled` variable (default value=`False`) to disable modelling and testing of the `ticket_deal` source table. If there are no associations between tickets and deals in your Hubspot environment, this table will not exist ([#79](https://github.com/fivetran/dbt_hubspot_source/pull/79)).
+
 ## Fixes
+- Consistently renames `property_dealname`, `property_closedate`, and `property_createdate` to `deal_name`, `closed_at`, and `created_at`, respectively, in the `deals` staging model. Previously, if `hubspot__pass_through_all_columns = true`, only the prefix `property_` was removed from the names of these fields, while they were completely renamed to `deal_name`, `closed_at`, and `created_at` if `hubspot__pass_through_all_columns = false` ([#79](https://github.com/fivetran/dbt_hubspot_source/pull/79)).
 - Bypass freshness tests for when a source is disabled by adding an enable/disable config to the source yml ([#77](https://github.com/fivetran/dbt_hubspot_source/pull/77))
 
 **Notice**: You must have dbt v1.1.0 or greater for the config to work. 

README.md

@@ -46,7 +46,7 @@ vars:
 
 ### Disabling models
 
-When setting up your Hubspot connection in Fivetran, it is possible that not every table this package expects will be synced. This can occur because you either don't use that functionality in Hubspot or have actively decided to not sync some tables. In order to disable the relevant functionality in the package, you will need to add the relevant variables. By default, all variables are assumed to be `true` (with exception of `hubspot_service_enabled` and `hubspot_contact_merge_audit_enabled`). You only need to add variables for the tables you would like to disable or enable respectively:
+When setting up your Hubspot connection in Fivetran, it is possible that not every table this package expects will be synced. This can occur because you either don't use that functionality in Hubspot or have actively decided to not sync some tables. In order to disable the relevant functionality in the package, you will need to add the relevant variables. By default, all variables are assumed to be `true` (with exception of `hubspot_service_enabled`, `hubspot_ticket_deal_enabled`, and `hubspot_contact_merge_audit_enabled`). You only need to add variables for the tables you would like to disable or enable respectively:
 
 ```yml
 # dbt_project.yml
@@ -97,6 +97,7 @@ vars:
 
   # Service
   hubspot_service_enabled: true                           # Enables all service models
+  hubspot_ticket_deal_enabled: true
 ```
 
 ### Dbt-core Version Requirement for disabling freshness tests

dbt_project.yml

@@ -62,6 +62,7 @@ vars:
 
     #Service Hub Enable Variables
     hubspot_service_enabled: false
+    hubspot_ticket_deal_enabled: false
 
     #Contact Merge Audit Enable Variable
     hubspot_contact_merge_audit_enabled: false

models/src_hubspot.yml

@@ -1335,7 +1335,7 @@ sources:
       - name: ticket_deal
         description: Each record represents a 'link' between a ticket and a deal.
         config:
-          enabled: "{{ var('hubspot_service_enabled', true) }}"
+          enabled: "{{ var('hubspot_service_enabled', false) and var('hubspot_ticket_deal_enabled', false) }}"
         columns:
           - name: _fivetran_synced
             description: '{{ doc("_fivetran_synced") }}'

models/stg_hubspot__deal.sql

@@ -19,13 +19,17 @@ with base as (
 ), fields as (
 
     select
+        property_dealname as deal_name,
+        property_closedate as closed_at,
+        property_createdate as created_at,
 
 {% if var('hubspot__pass_through_all_columns', false) %}
-        -- just pass everything through
+        -- just pass everything else through
         {{ 
             fivetran_utils.remove_prefix_from_columns(
                 columns=adapter.get_columns_in_relation(ref('stg_hubspot__deal_tmp')), 
-                prefix='property_') 
+                prefix='property_',
+                exclude=['property_dealname','property_closedate','property_createdate']) 
         }}
     from base
 
@@ -38,11 +42,8 @@ with base as (
         cast(deal_pipeline_stage_id as {{ dbt_utils.type_string() }}) as deal_pipeline_stage_id,
         owner_id,
         portal_id,
-        property_dealname as deal_name,
         property_description as description,
-        property_amount as amount,
-        property_closedate as closed_at,
-        property_createdate as created_at
+        property_amount as amount
 
         --The below macro adds the fields defined within your hubspot__deal_pass_through_columns variable into the staging model
         {{ fivetran_utils.fill_pass_through_columns('hubspot__deal_pass_through_columns') }}

models/stg_hubspot__ticket_deal.sql

@@ -1,4 +1,4 @@
-{{ config(enabled=var('hubspot_service_enabled', False)) }}
+{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_service_enabled','hubspot_ticket_deal_enabled'])) }}
 
 with base as (
 

models/tmp/stg_hubspot__ticket_deal_tmp.sql

@@ -1,4 +1,4 @@
-{{ config(enabled=var('hubspot_service_enabled', True)) }}
+{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_service_enabled','hubspot_ticket_deal_enabled'])) }}
 
 select *
 from {{ var('ticket_deal') }}