Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dataset alias unique constraint and remove wrong dataset alias removing logic #41097

Merged
merged 4 commits into from
Jul 30, 2024

Conversation

Lee-W
Copy link
Member

@Lee-W Lee-W commented Jul 29, 2024

Why

  1. DatasetAlias should be unique. However, the existing logic does not handle that part.
  2. The original logic only keeps the dataset alias in one dag file.

What

  1. Added the unique constraint to DatasetAliasModel
  2. Remove the deletion logic of DatasetAlias. It now works ok, but the unique dataset aliases will accumulate. I'm thinking of doing something similar to TaskOutletDatasetAliasReference and will create another PR for it.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@Lee-W Lee-W force-pushed the add-dataset-alias-unique-constraint branch from 7abed49 to fa66210 Compare July 29, 2024 14:20
@Lee-W Lee-W force-pushed the add-dataset-alias-unique-constraint branch 2 times, most recently from 0e62e00 to 20ae5aa Compare July 30, 2024 02:01
@Lee-W Lee-W requested a review from ephraimbuddy July 30, 2024 06:35
@Lee-W Lee-W marked this pull request as draft July 30, 2024 07:16
@Lee-W Lee-W marked this pull request as ready for review July 30, 2024 07:34
@Lee-W Lee-W force-pushed the add-dataset-alias-unique-constraint branch from 20ae5aa to b1ee453 Compare July 30, 2024 07:34
@Lee-W Lee-W merged commit b317aa8 into apache:main Jul 30, 2024
48 checks passed
@Lee-W Lee-W deleted the add-dataset-alias-unique-constraint branch July 30, 2024 09:13
@utkarsharma2 utkarsharma2 added the type:improvement Changelog: Improvements label Jul 31, 2024
@utkarsharma2 utkarsharma2 added this to the Airflow 2.10.0 milestone Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:db-migrations PRs with DB migration kind:documentation type:improvement Changelog: Improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants