-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update stripe package with running total model #60
Changes from 8 commits
1cbb12c
ebc7ad5
3719dee
27ac68d
bcdc4f3
2feea9c
58d7e84
bb3a6e9
482ac34
e248165
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -25,6 +25,7 @@ with date_spine as ( | |
), final as ( | ||
|
||
select | ||
date_spine.account_id, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Was this added to help with the join? I would add the reason for this addition to the changelog. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. true, added! |
||
date_spine.date_day, | ||
date_spine.date_week, | ||
date_spine.date_month, | ||
|
@@ -46,6 +47,7 @@ with date_spine as ( | |
from date_spine | ||
left join account_rolling_totals | ||
on account_rolling_totals.date_day = date_spine.date_day | ||
and account_rolling_totals.account_id = date_spine.account_id | ||
and account_rolling_totals.source_relation = date_spine.source_relation | ||
) | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,21 @@ | ||
version: 2 | ||
|
||
models: | ||
- name: int_stripe__account_daily | ||
description: Each record represents each day's ending balances per account. | ||
|
||
- name: int_stripe__account_partitions | ||
description: Each record is a group of partitioned account totals updating null values with zeroes to eventually calculate running totals downstream. | ||
|
||
- name: int_stripe__account_rolling_totals | ||
description: Each record represents each day's ending balances per account, in addition to changes over time. | ||
|
||
- name: int_stripe__date_spine | ||
description: Each record represents a day of each calendar year. | ||
|
||
- name: int_stripe__incomplete_charges | ||
description: Each record represents a charge that is incomplete. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This was helpful to understand what the different int models do! |
||
- name: stripe__balance_transactions | ||
description: Each record represents a change to your account balance, enriched with data about the transaction. | ||
tests: | ||
|
@@ -390,6 +405,8 @@ models: | |
- date_day | ||
- source_relation | ||
columns: | ||
- name: account_id | ||
description: The ID of the account tied to the balance. | ||
- name: date_day | ||
description: Day of record, taken from the date of each balance transaction. | ||
- name: source_relation | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,32 +1,46 @@ | ||
{% set total_fields = ['total_daily_sales_amount', 'total_daily_refunds_amount', 'total_daily_adjustments_amount', 'total_daily_other_transactions_amount', 'total_daily_gross_transaction_amount', 'total_daily_net_transactions_amount', 'total_daily_payout_fee_amount', 'total_daily_gross_payout_amount', 'daily_net_activity_amount', 'daily_end_balance_amount', 'total_daily_sales_count', 'total_daily_payouts_count', 'total_daily_adjustments_count', 'total_daily_failed_charge_count', 'total_daily_failed_charge_amount'] %} | ||
{% set rolling_fields = ['rolling_total_daily_sales_amount', 'rolling_total_daily_refunds_amount', 'rolling_total_daily_adjustments_amount', 'rolling_total_daily_other_transactions_amount', 'rolling_total_daily_gross_transaction_amount', 'rolling_total_daily_net_transactions_amount', 'rolling_total_daily_payout_fee_amount', 'rolling_total_daily_gross_payout_amount', 'rolling_daily_net_activity_amount', 'rolling_daily_end_balance_amount', 'rolling_total_daily_sales_count', 'rolling_total_daily_payouts_count', 'rolling_total_daily_adjustments_count', 'rolling_total_daily_failed_charge_count', 'rolling_total_daily_failed_charge_amount'] %} | ||
|
||
with account_partitions as ( | ||
|
||
select * | ||
from {{ ref('int_stripe__account_partitions') }} | ||
), | ||
|
||
), final as ( | ||
final as ( | ||
|
||
select | ||
date_day, | ||
date_week, | ||
date_month, | ||
date_year, | ||
|
||
{% for t in total_fields %} | ||
{{ t }}, | ||
{% endfor %} | ||
account_id, | ||
{{ dbt_utils.generate_surrogate_key(['account_id','date_day']) }} as account_daily_id, | ||
Comment on lines
+12
to
+13
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry I didn't catch this before, but I'm getting a test failure from combination_of_columns:
- date_day
- source_relation Guessing this test needs to be adjusted? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ah, added account_id, forgot I included that field now. Should be good now, thanks for catching! |
||
|
||
date_day, | ||
date_week, | ||
date_month, | ||
date_year, | ||
date_index, | ||
source_relation, | ||
coalesce(total_daily_sales_amount,0) as total_daily_sales_amount, | ||
coalesce(total_daily_refunds_amount,0) as total_daily_refunds_amount, | ||
coalesce(total_daily_adjustments_amount,0) as total_daily_adjustments_amount, | ||
coalesce(total_daily_other_transactions_amount,0) as total_daily_other_transactions_amount, | ||
coalesce(total_daily_gross_transaction_amount,0) as total_daily_gross_transaction_amount, | ||
coalesce(total_daily_net_transactions_amount,0) as total_daily_net_transactions_amount, | ||
coalesce(total_daily_payout_fee_amount,0) as total_daily_payout_fee_amount, | ||
coalesce(total_daily_gross_payout_amount,0) as total_daily_gross_payout_amount, | ||
coalesce(daily_net_activity_amount,0) as daily_net_activity_amount, | ||
coalesce(daily_end_balance_amount,0) as daily_end_balance_amount, | ||
coalesce(total_daily_sales_count,0) as total_daily_sales_count, | ||
coalesce(total_daily_payouts_count,0) as total_daily_payouts_count, | ||
coalesce(total_daily_adjustments_count,0) as total_daily_adjustments_count, | ||
coalesce(total_daily_failed_charge_count,0) as total_daily_failed_charge_count, | ||
coalesce(total_daily_failed_charge_amount,0) as total_daily_failed_charge_amount, | ||
{% for f in rolling_fields %} | ||
coalesce({{ f }}, | ||
first_value({{ f }}) over (partition by {{ f }}_partition order by date_day rows unbounded preceding)) as {{ f }}, | ||
first_value({{ f }}) over (partition by {{ f }}_partition order by date_day rows unbounded preceding)) as {{ f }} | ||
{%- if not loop.last -%},{%- endif -%} | ||
{% endfor %} | ||
|
||
source_relation | ||
|
||
from account_partitions | ||
) | ||
) | ||
|
||
select * | ||
from final |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think some of this applies anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And the README install sections still needs to be adjusted for the breaking change version.