Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

package updates #16

Merged
merged 8 commits into from
Nov 2, 2020
Merged

package updates #16

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
71645c9
using utils enabled_vars
fivetran-jamie Oct 29, 2020
81e3d8a
remove 1-use macro
fivetran-jamie Oct 30, 2020
e972c97
update dbt versions and packages
fivetran-jamie Oct 30, 2020
1c49bdf
readme tweak
fivetran-jamie Oct 30, 2020
7da6ec3
add coalesces
fivetran-jamie Oct 30, 2020
e2c01ee
remove macros
fivetran-jamie Oct 30, 2020
e306f24
fix typo
fivetran-jamie Oct 30, 2020
3614ad5
kristins feedback
fivetran-jamie Nov 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This package contains staging models, designed to work simultaneously with our [

* Remove any rows that are soft-deleted
* Name columns consistently across all packages:
* Boolean fields are prefixed with is_ or has_
* Boolean fields are prefixed with `is_` or `has_`
* Timestamps are appended with `_timestamp`
* ID primary keys are prefixed with the name of the table. For example, the user table's ID column is renamed user_id.

Expand Down
4 changes: 2 additions & 2 deletions dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: 'hubspot_source'
version: '1.0.0'
version: '2.0.0'
fivetran-jamie marked this conversation as resolved.
Show resolved Hide resolved
config-version: 2
require-dbt-version: ">=0.17.0"
require-dbt-version: [">=0.18.0", "<0.19.0"]

vars:
hubspot_source:
Expand Down
13 changes: 0 additions & 13 deletions macros/enabled_vars.sql
View file
Open in desktop

This file was deleted.

5 changes: 0 additions & 5 deletions macros/full_name.sql
View file
Open in desktop

This file was deleted.

12 changes: 0 additions & 12 deletions macros/remove_prefix_from_columns.sql
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions models/stg_hubspot__company.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_company_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_company_enabled'])) }}

{%- set columns = adapter.get_columns_in_relation(ref('stg_hubspot__company_adapter')) -%}

Expand All @@ -12,7 +12,7 @@ with base as (

select
id as company_id,
{{ remove_prefix_from_columns(columns=columns, exclude=['id']) }}
{{ fivetran_utils.remove_prefix_from_columns(columns=columns, exclude=['id']) }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{ fivetran_utils.remove_prefix_from_columns(columns=columns, exclude=['id']) }}
{{ fivetran_utils.remove_prefix_from_columns(columns=columns, prefix='property_', exclude=['id']) }}

Need to define prefix in all remove_prefix_from_columns macros bc of this comment in your other PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added it to all affected files!

from base

)
Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__company_adapter.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_company_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_company_enabled'])) }}

select *
from {{ var('company') }}
2 changes: 1 addition & 1 deletion models/stg_hubspot__company_property_history.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_company_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_company_enabled'])) }}

with base as (

Expand Down
4 changes: 2 additions & 2 deletions models/stg_hubspot__contact.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ with base as (

select *
from {{ ref('stg_hubspot__contact_adapter') }}
where _fivetran_deleted is null
where not coalesce(_fivetran_deleted, false)

), fields as (

select
id as contact_id,
{{ remove_prefix_from_columns(columns=columns, exclude=['id']) }}
{{ fivetran_utils.remove_prefix_from_columns(columns=columns, exclude=['id']) }}
from base

)
Expand Down
4 changes: 2 additions & 2 deletions models/stg_hubspot__contact_list.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled'])) }}

with base as (

select *
from {{ var('contact_list') }}
where _fivetran_deleted is null
where not coalesce(_fivetran_deleted, false)

), fields as (

Expand Down
4 changes: 2 additions & 2 deletions models/stg_hubspot__contact_list_member.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled'])) }}

with base as (

select *
from {{ var('contact_list_member') }}
where _fivetran_deleted is null
where not coalesce(_fivetran_deleted, false)

), fields as (

Expand Down
4 changes: 2 additions & 2 deletions models/stg_hubspot__deal.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}

{%- set columns = adapter.get_columns_in_relation(ref('stg_hubspot__deal_adapter')) -%}

Expand All @@ -11,7 +11,7 @@ with base as (
), fields as (

select
{{ remove_prefix_from_columns(columns=columns) }}
{{ fivetran_utils.remove_prefix_from_columns(columns=columns) }}
from base

)
Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__deal_adapter.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}

select *
from {{ var('deal') }}
4 changes: 2 additions & 2 deletions models/stg_hubspot__deal_pipeline.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}

with base as (

select *
from {{ var('deal_pipeline')}}
where _fivetran_deleted is null
where not coalesce(_fivetran_deleted, false)

), fields as (

Expand Down
4 changes: 2 additions & 2 deletions models/stg_hubspot__deal_pipeline_stage.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}

with base as (

select *
from {{ var('deal_pipeline_stage')}}
where _fivetran_deleted is null
where not coalesce(_fivetran_deleted, false)

), fields as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__deal_property_history.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_deal_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_campaign.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_bounce.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_bounce_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_bounce_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_click.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_click_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_click_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_deferred.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_deferred_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_deferred_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_delivered.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_delivered_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_delivered_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_dropped.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_dropped_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_dropped_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_forward.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_forward_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_forward_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_open.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_open_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_open_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_print.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_print_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_print_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_sent.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_sent_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_sent_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_spam_report.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_spam_report_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_spam_report_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__email_event_status_change.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_status_change_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_marketing_enabled','hubspot_email_event_enabled','hubspot_email_event_status_change_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_call.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_call_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_call_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_company.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_company_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_company_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_contact.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_contact_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_contact_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_deal.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_deal_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_deal_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_email.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_email_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_email_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_meeting.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_email_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_email_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_note.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_note_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_note_enabled'])) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_hubspot__engagement_task.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_task_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_engagement_enabled','hubspot_engagement_task_enabled'])) }}

with base as (

Expand Down
4 changes: 2 additions & 2 deletions models/stg_hubspot__owner.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=enabled_vars(['hubspot_sales_enabled','hubspot_owner_enabled'])) }}
{{ config(enabled=fivetran_utils.enabled_vars(['hubspot_sales_enabled','hubspot_owner_enabled'])) }}

with base as (

Expand All @@ -17,7 +17,7 @@ with base as (
portal_id,
type as owner_type,
updated_at as updated_timestamp,
{{ full_name('first_name','last_name') }}
trim( {{ dbt_utils.concat(['first_name', "' '", 'last_name']) }} ) as full_name
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did this because the macro was only used once and was just one line

from base

)
Expand Down
5 changes: 4 additions & 1 deletion packages.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
packages:
- package: fishtown-analytics/dbt_utils
version: [">=0.4.0"]
version: [">=0.6.0", "<0.7.0"]

- git: "https://github.com/fivetran/dbt_fivetran_utils.git"
warn-unpinned: false