Skip to content

Conversation

@Lee-W
Copy link
Member

@Lee-W Lee-W commented May 18, 2025

Why

airflow db migrate failed when I migrated an old airflow instance to 2.11.0rc1

What

Following up #49301 and trying to fix some found issues

  • 0093 upgrade, 0100 upgrade, 0060 downgrade, 0064 downgrade: since SQite does not support constant drop, follow how Fix DB migrations from 2.10.5 to 3.0.0 for SQlite #49301 deals with fk for SQlite
  • 0110: backport _drop_fkey_if_exists for handling fk in sqlite
  • 0142: Explicitly list down the column names when recreating dag_run table
  • 0073: add a safeguard to ensure we're not wrongly update the name

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W added 2 commits May 18, 2025 13:59
* 0093: skip pk constant drop as sqlite does not support it
* 0110: backport _drop_fkey_if_exists for handling fk in sqlite
* 0142: explicitly list down the column names when recreating dag_run table
@jscheffl
Copy link
Contributor

Ups, something wrong with the PR and target branch is incorrect.

@Lee-W Lee-W changed the base branch from main to v2-11-test May 18, 2025 09:09
@Lee-W
Copy link
Member Author

Lee-W commented May 18, 2025

Ups, something wrong with the PR and target branch is incorrect.

Ah.. right. Checked out from v2-11-test but forget to change the base branch when pushing. Thanks! Just updated it

Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay as fix, otherwise the if conditions in that old migration code would need to be optimized for readability :-D

@Lee-W
Copy link
Member Author

Lee-W commented May 18, 2025

Okay as fix, otherwise the if conditions in that old migration code would need to be optimized for readability :-D

Just get back to my laptop before bed 🙌 . removed one layer of nested if-else. please check whether it's a bit better 👀

@jscheffl
Copy link
Contributor

Okay as fix, otherwise the if conditions in that old migration code would need to be optimized for readability :-D

Just get back to my laptop before bed 🙌 . removed one layer of nested if-else. please check whether it's a bit better 👀

Better! Thannks for un-rolling the if - but CI is not convinced. So maybe tomorrow after some sleep you will make it happen :-D

@Lee-W
Copy link
Member Author

Lee-W commented May 19, 2025

Okay as fix, otherwise the if conditions in that old migration code would need to be optimized for readability :-D

Just get back to my laptop before bed 🙌 . removed one layer of nested if-else. please check whether it's a bit better 👀

Better! Thannks for un-rolling the if - but CI is not convinced. So maybe tomorrow after some sleep you will make it happen :-D

Diff ruff version 🤦‍♂️ I'm still using the one on main. I extracted that logic to the utils as it was used in 0100 as well.

@Lee-W
Copy link
Member Author

Lee-W commented May 19, 2025

Found a few more issues related. so I extracted the sqlite thing as another function in utils

@Lee-W
Copy link
Member Author

Lee-W commented May 19, 2025

Forget to update the erd. 🤦‍♂️Will update it after dinner

@kaxil kaxil merged commit d93ce5e into apache:v2-11-test May 19, 2025
41 checks passed
@kaxil kaxil deleted the fix-migration branch May 19, 2025 18:34
@kaxil kaxil modified the milestones: Airflow 2.11.0, Airflow 2.11.1 May 19, 2025
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 1, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 1, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 1, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 1, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 7, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 7, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 7, 2025
apache/airflow#50745 will be released in airflow 2.11
Lee-W added a commit to pycontw/pycon-etl that referenced this pull request Jun 8, 2025
apache/airflow#50745 will be released in airflow 2.11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants