-
Notifications
You must be signed in to change notification settings - Fork 981
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add custom granularities to mf timespine #6145
Changes from 2 commits
d33aa6b
cf6500a
07816ee
15e86c2
f1686d4
4b9ebc4
e7c5156
3c0bd33
4a5b15c
0d60378
09e3a3a
1f1e177
89d3d1d
42132b9
405130f
98243aa
f5bd3d4
2921f0e
f626ab4
60eef89
46aabda
e023b96
8ebdccc
beefe57
5e6a7a7
dd8547e
1ad6362
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,13 +13,13 @@ MetricFlow requires you to define a time-spine table as a model-level configurat | |
- [Metric offsets](/docs/build/derived#derived-metric-offset) | ||
- [Conversion metrics](/docs/build/conversion) | ||
- [Slowly Changing Dimensions](/docs/build/dimensions#scd-type-ii) | ||
- [Metrics](/docs/build/metrics-overview) with the `join_to_timespine` configuration set to true | ||
To see the generated SQL for the metric and dimension types that use time-spine joins, refer to the respective documentation or add the `compile=True` flag when querying the Semantic Layer to return the compiled SQL. | ||
- [Metrics](/docs/build/metrics-overview) with the `join_to_timespine` configuration set to `true`. To see the generated SQL for the metric and dimension types that use time-spine joins, refer to the respective documentation or add the `compile=True` flag when querying the Semantic Layer to return the compiled SQL. | ||
|
||
#### Configuring time-spine | ||
### Configuring time-spine | ||
- You only need to configure time-spine models that the Semantic Layer should recognize. | ||
- At a minimum, define a time-spine table for a daily grain. | ||
- You can optionally define a time-spine table for a different granularity, like hourly. | ||
- If your organization uses [custom calendars](#custom-calendar) (like fiscal years) or custom granularities (like `retail_month`), you can configure those as well. Learn how to [add custom calendars and granularities](#add-custom-granularities) further down this page to help support specific business needs. | ||
- Note that if you don’t have a date or calendar model in your project, you'll need to create one. | ||
- If you're looking to specify the grain of a time dimension so that MetricFlow can transform the underlying column to the required granularity, refer to the [Time granularity documentation](/docs/build/dimensions?dimension=time_gran) | ||
|
||
|
@@ -36,13 +36,18 @@ If you don’t have a date dimension table, you can still create one by using th | |
|
||
<Lightbox src="/img/time_spines.png" title="Time spine directory structure" /> | ||
|
||
<VersionBlock firstVersion="1.9"> | ||
<File name="models/_models.yml"> | ||
|
||
```yaml | ||
models: | ||
- name: time_spine_hourly | ||
description: "my favorite time spine" | ||
time_spine: | ||
standard_granularity_column: date_hour # column for the standard grain of your table | ||
custom_granularities: | ||
- name: fiscal_year | ||
column_name: fiscal_year_column | ||
columns: | ||
- name: date_hour | ||
granularity: hour # set granularity at column-level for standard_granularity_column | ||
|
@@ -54,6 +59,29 @@ models: | |
granularity: day # set granularity at column-level for standard_granularity_column | ||
``` | ||
</File> | ||
</VersionBlock> | ||
|
||
<VersionBlock lastVersion="1.8"> | ||
<File name="models/_models.yml"> | ||
|
||
```yaml | ||
models: | ||
- name: time_spine_hourly | ||
description: "my favorite time spine" | ||
time_spine: | ||
standard_granularity_column: date_hour # column for the standard grain of your table | ||
columns: | ||
- name: date_hour | ||
granularity: hour # set granularity at column-level for standard_granularity_column | ||
- name: time_spine_daily | ||
time_spine: | ||
standard_granularity_column: date_day # column for the standard grain of your table | ||
columns: | ||
- name: date_day | ||
granularity: day # set granularity at column-level for standard_granularity_column | ||
``` | ||
</File> | ||
</VersionBlock> | ||
|
||
For an example project, refer to our [Jaffle shop](https://github.com/dbt-labs/jaffle-sl-template/blob/main/models/marts/_models.yml) example. | ||
|
||
|
@@ -230,3 +258,52 @@ where date_day > dateadd(year, -4, current_timestamp()) | |
and date_hour < dateadd(day, 30, current_timestamp()) | ||
``` | ||
</File> | ||
|
||
|
||
## Custom calendar | ||
mirnawong1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
<VersionBlock lastVersion="1.8"> | ||
|
||
Being able to configure custom calendars, such as like a fiscal calendar, is available in [dbt Cloud Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless) or dbt Core [v1.9 and above](/docs/dbt-versions/core). | ||
|
||
To access this feature, [upgrade to Versionless](/docs/dbt-versions/versionless-cloud) or dbt Core v1.9 and above. | ||
</VersionBlock> | ||
|
||
<VersionBlock firstVersion="1.9"> | ||
|
||
If you use a custom calendar in your organization, such as a fiscal calendar, you can configure it in MetricFlow using its date and time operations. | ||
|
||
This is useful for calculating metrics based on a custom calendar, such as fiscal quarters or weeks. Use the `custom_granularities` key to define a non-standard time period for querying data, such as a `retail_month` or `fiscal_week`, instead of standard options like `day`, `month`, or `year`. This feature provides more control over how time-based metrics are calculated. | ||
|
||
|
||
### Why use a custom calendar model? | ||
Custom date transformations can be complex, and organizations often have unique needs that can’t be easily generalized. Creating a custom calendar model allows you to define these transformations in SQL, offering more flexibility than native transformations in MetricFlow. This approach lets you map custom columns back to MetricFlow granularities, ensuring consistency while giving you control over the transformations. | ||
|
||
### Calendar model requirements | ||
|
||
To use a custom calendar model, ensure it meets the following requirements: | ||
|
||
- The model must be at a daily grain and joinable to other date columns using `date_day`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this is a requirement, you could use a dim date table at an hourly grain and define custom granularities for that model. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ++ you can use any of our standard granularities! |
||
- Columns must align with expected data types for a given granularity (for example, `quarter` should be a date). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we update this to be a little more specific: |
||
- Support period-over-period calculations. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This isn't a requirement of the model, it's a requirement of for the functionality we support. We can remove it. |
||
- Support cumulative metric windows using custom granularities (for example, cumulative fiscal year-to-date bookings). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same comment as above, this is a requirement for us, not the underlying model |
||
|
||
### Add custom granularities | ||
|
||
To add custom granularities, the Semantic Layer supports custom calendar configurations that allow users to query data using non-standard time periods like `fiscal_year` or `retail_month`. You can define these custom granularities by modifying your model's YAML configuration like this: | ||
|
||
<File name="models/_models.yml"> | ||
|
||
```yaml | ||
models: | ||
- name: my_time_spine | ||
description: "my favorite time spine" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need to wrap this in quotes |
||
time_spine: | ||
standard_granularity_column: date_day | ||
custom_granularities: | ||
- name: fiscal_year | ||
mirnawong1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
column_name: fiscal_year_column | ||
``` | ||
</File> | ||
|
||
</VersionBlock> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason we use "time-spine" instead of just "time spine"? Curious if @Jstein77 has an opinion on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i was originally using time spine but then saw it used with a dash and thought/assumed i had a wrong. happy with time spine too! @Jstein77 are we good with time spine?