Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openlineage: do not try to redact Proxy objects from deprecated config #33393

Merged
merged 1 commit into from
Aug 15, 2023

Conversation

mobuchowski
Copy link
Contributor

When preparing AirfowRunFacet we're iterating over Context. This access cause warnings:

[2023-08-14, 09:02:53 UTC] {warnings.py:109} WARNING - /usr/local/lib/python3.11/site-packages/airflow/utils/context.py:314: AirflowContextDeprecationWarning: Accessing 'next_ds' from the template is deprecated and will be removed in a future version. Please use '{{ data_interval_end | ds }}' instead.
  warnings.warn(_create_deprecation_warning(k, replacements))
[2023-08-14, 09:02:53 UTC] {secrets_masker.py:277} WARNING - Unable to redact <Proxy at 0x7f07121b14c0 wrapping '2023-08-14' at 0x7f0712137770 with factory functools.partial(<function lazy_mapping_from_context.<locals>._deprecated_proxy_factory at 0x7f071216d260>, 'next_ds', '2023-08-14')>, please report this via <https://github.com/apache/airflow/issues>. Error was: TypeError: ObjectProxy() missing required argument 'wrapped' (pos 1)

This PR removes direct access to those objects, and redacting it when Proxy is encountered. We can't remove iteration over dict, so in this case we're suppressing this error. In case of Proxy we'll skip serializing the object anyway.

Signed-off-by: Maciej Obuchowski <obuchowski.maciej@gmail.com>
@mobuchowski mobuchowski merged commit 8e738cd into main Aug 15, 2023
42 checks passed
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 17, 2023
apache#33393)

Signed-off-by: Maciej Obuchowski <obuchowski.maciej@gmail.com>
ephraimbuddy pushed a commit that referenced this pull request Aug 28, 2023
#33393)

Signed-off-by: Maciej Obuchowski <obuchowski.maciej@gmail.com>
(cherry picked from commit 8e738cd)
@Taragolis Taragolis deleted the openlineage-do-not-redact-proxy branch September 26, 2023 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants