Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Back/fwd compatibility for renamed metrics attributes #5825

Merged
merged 8 commits into from
Sep 16, 2022

Conversation

jtcohen6
Copy link
Contributor

resolves #5807
follow-up to #5775

Description

Define a method, rename_metric_attr, that switches the old metric attribute names + values (sql, type, type: expression) to the new names + values (expression, calculation_method, calculation_method: derived). This is called in two places:

  • During parsing, if the old names are present, raise a deprecation warning and then handle the rename
  • When loading an old manifest (--state), just handle the rename

I also removed some annotations within the metrics parsing code that I believe to be outdated

Pedantic question, not very important: We tend to call these things "properties" within dbt, but am I right in thinking the Pythonic name for them is dataclass attributes?

Checklist

core/dbt/contracts/util.py Outdated Show resolved Hide resolved
core/dbt/deprecations.py Outdated Show resolved Hide resolved
Copy link
Member

@emmyoop emmyoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did the manifest.json files get updated?

@jtcohen6
Copy link
Contributor Author

@emmyoop I added a metric to the project used in TestPreviousVersionState, and re-generated the v4-6 manifests (dbt-core v1.0-1.2) using the new project. The test confirms that dbt can successfully read in older manifest.json, containing the old metric properties names, and detect zero changes in state:modified.

@emmyoop
Copy link
Member

emmyoop commented Sep 14, 2022

Oh, so we just didn't have metrics defined in the previous project we were using to generate the manifest? Makes sense.

AdditionalPropertiesMixin,
Mergeable,
Replaceable,
rename_metric_attr,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for my own understanding, is there any reason (apart from data class vs function) for the different naming conventions here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apart from data class vs function

I think that's it!

Copy link
Contributor

@callum-mcdata callum-mcdata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Only one small comment but that's mainly for my own understanding

@jtcohen6
Copy link
Contributor Author

Thanks for the functional review @callum-mcdata!

@emmyoop I'll wait to merge till you have a chance to review the code

Copy link
Member

@emmyoop emmyoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice temporary help for early adopters! Just a few suggestions.

Comment on lines 257 to 258
if "window" not in metric_content:
metric_content["window"] = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this you should change the ParsedMetric definition to default to None for window, same as it already does for UnparsedMetric. I would expect that to resolve this.

While looking at that bit of code, I noticed ParsedMetric still has timestamp as optional and I know @callum-mcdata updated the UnparsedMetric definition to not be optional in a recent PR. timestamp should probably match.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emmyoop you were totally right! I was able to remove this component by setting the default value of window to be None in ParsedMetric. I also fixed the timestamp field to reflect UnparsedMetric like you suggested - must have missed that on my last PR!

Comment on lines 92 to 99
_description = """\
dbt-core v1.3 renamed attributes for metrics:
\n 'sql' -> 'expression'
\n 'type' -> 'calculation_method'
\n 'type: expression' -> 'calculation_method: derived'
\nThe old metric parameter names will be fully deprecated in v1.4.
\nPlease remove them from the metric definition of metric '{metric_name}'
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment here with a link to the issue that will do this so it doesn't get lost? Love that we're committing to a timeframe here!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue created here: #5849

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emmyoop the link to this GH issue has been added to the deprecation warning!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@callum-mcdata I actually meant a comment in the code, not as part of the deprecation message. Sorry if that was unclear! Unless you think users will find the link to the issue useful?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 woops! Although now that I think about it, if we include the link to the issue then anyone who feels very strongly that this behavior should be retained could go to it easily and let us know there. That way the information isn't spread over multiple channels..... so actually I do think it would be useful!

@jtcohen6
Copy link
Contributor Author

Thanks for the review @emmyoop, and for getting this over the finish line @callum-mcdata!

@jtcohen6 jtcohen6 merged commit f5a94fc into main Sep 16, 2022
@jtcohen6 jtcohen6 deleted the jerco/metric-rename-back-compat branch September 16, 2022 09:21
josephberni pushed a commit to Gousto/dbt-core that referenced this pull request Sep 16, 2022
* Naive handling for metric attr renames

* Add tests for bwd/fwd compatibility

* Add deprecation

* Add changelog entry

* PR feedback

* Small fixups

* emmyoop's suggestions

Co-authored-by: Callum McCann <cmccann51@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-1148] [Bug] Follow-up to metrics field renaming
3 participants