-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First draft of Reporting Source of Truth™ #496
Open
wrridgeway
wants to merge
56
commits into
master
Choose a base branch
from
387-reporting-sot
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
d780d76
First draft of sales script
wrridgeway 00909fd
File renaming
wrridgeway 2ac5982
Cleaner for loop
wrridgeway 2107d2a
First draft taxes and exemptions table
wrridgeway c56aaaf
Wrap assessment_roll
wrridgeway 6c81308
Correct size, count calculations
wrridgeway 1bf9b9c
Wrap sales table
wrridgeway 0a9e1f3
Correct stage grouping, counting
wrridgeway 030a7c5
Fix assessment roll stage grouping
wrridgeway 1c2adae
Clean output before writing
wrridgeway 672bd1e
Begin dbt building
wrridgeway 0c42e23
Merge branch 'master' into 387-reporting-sot
wrridgeway 3f60a77
Attempt to build assessment_roll table
wrridgeway fdff457
Testing build on smaller input
wrridgeway 6abd074
Trying to build on limited sample
wrridgeway fd342b6
Try to build sales table
wrridgeway cccf8e1
Try to build taxes and exemptions table
wrridgeway 3656964
Try to build taxes and exemptions table
wrridgeway 8b0f95f
Try to build taxes table
wrridgeway 9383bdc
Try to build ratio stats table
wrridgeway 08d3bd6
Add assesspy to ratio_stats table
wrridgeway d2cac22
ratio_stats builds in dbt, excluding assesspy funcs
wrridgeway f559753
sot_ratio_stats table building in dbt
wrridgeway 1f8ad1f
Add res_other group
wrridgeway 063591c
Add reassessment year indicator for assessment roll
wrridgeway a9ffc64
Retry assessment_year indicator
wrridgeway 62dd68e
Assessment_roll should run with reassessment year indicator
wrridgeway c185e81
Add schema to assessment_roll table
wrridgeway d08bc3d
Correct output from sales and taxes tables
wrridgeway 4808aa4
Add table schemas
wrridgeway 08c8d53
Fix schemas
wrridgeway 2f8dc3d
Resolve sales table column type issues
wrridgeway 88ce049
Add exe_total to exemptions table
wrridgeway 271576d
Add more ratio stats
wrridgeway c39a2d8
Clean sales table columns
wrridgeway 20c9bd6
Clean taxes table columns
wrridgeway adc16ea
Clean assessment_roll columns
wrridgeway f8b87ab
Fix delta columns
wrridgeway 54ebab8
Clean ratio table columns
wrridgeway d2dddab
Attempt to fix pin_n_tot type error that doesn't trigger locally
wrridgeway 00e790c
Try again to fix pin_n_tot
wrridgeway 408de56
Change ass roll sample to be able to compare across stages
wrridgeway fd95fcb
Add commenting for input tables, try to partion assessment_roll table
wrridgeway f296292
Comment python scripts
wrridgeway a23ff72
Clean up ratio_stats script
wrridgeway 07f6dfe
Back to fixing pin_n_tot
wrridgeway b78a072
Replace nan with None
wrridgeway 337954e
Partition input tables by year
wrridgeway 1031144
Fix year partitioning
wrridgeway 45ea305
Use double for nullable columns
wrridgeway ca139f3
Move data year specification to dbt seed
wrridgeway 788f971
Formatting
wrridgeway 4ea6718
Merge branch 'master' into 387-reporting-sot
wrridgeway 5449d8c
Improve diff and pct_change syntax
wrridgeway c87713f
Simplify reassessment year syntax
wrridgeway d1079f0
More commenting
wrridgeway File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,4 +19,4 @@ venv/ | |
# Ignore most CSVs, except those that are used as dbt seeds | ||
*.csv | ||
!dbt/seeds/**/*.csv | ||
|
||
*.parquet.gzip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -73,5 +73,7 @@ seeds: | |
+schema: location | ||
model: | ||
+schema: model | ||
reporting: | ||
+schema: reporting | ||
spatial: | ||
+schema: spatial |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -30,6 +30,75 @@ Materialized to speed up queries for Tableau. | |
`property_group` | ||
{% enddocs %} | ||
|
||
# sot_assessment_roll | ||
|
||
{% docs table_sot_assessment_roll %} | ||
Table to feed the Python dbt job that creates the | ||
`reporting.sot_assessment_roll` table. Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `stage_name`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_assessment_roll_input | ||
|
||
{% docs table_sot_assessment_roll_input %} | ||
Table to feed the Python dbt job that creates the | ||
`reporting.sot_assessment_roll` table. Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `stage_name`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_ratio_stats | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. issue (blocking): No other dbt models have plural names, let's stick to that convention. So:
etc. |
||
|
||
{% docs table_sot_ratio_stats %} | ||
Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `stage_name`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_ratio_stats_input | ||
|
||
{% docs table_sot_ratio_stats_input %} | ||
Table to feed the Python dbt job that creates the | ||
`reporting.sot_ratio_stats` table. Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `stage_name`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_sales | ||
|
||
{% docs table_sot_sales %} | ||
Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_sales_input | ||
|
||
{% docs table_sot_sales_input %} | ||
Table to feed the Python dbt job that creates the | ||
`reporting.sot_sales` table. Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_taxes_exemptions | ||
|
||
{% docs table_sot_taxes_exemptions %} | ||
Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# sot_taxes_exemptions_input | ||
|
||
{% docs table_sot_taxes_exemptions_input %} | ||
Table to feed the Python dbt job that creates the | ||
`reporting.sot_taxes_exemptions` table. Feeds public reporting assets. | ||
|
||
**Primary Key**: `year`, `geography_id`, `group_id` | ||
{% enddocs %} | ||
|
||
# vw_assessment_roll | ||
|
||
{% docs view_vw_assessment_roll %} | ||
|
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
issue (blocking): This is the same as the description for
sot_assesssment_roll_input
. Let's change the actualsot_
non-input table descriptions to include the table's purpose, structure, and a short description of how to use the table.