Allow downgrading to 2.11 from 3.x #54371
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There were two things blocking this: 1) The revision heads map didn't have any 2.11.x versions in it, so the previous implementation of `_get_version_revision` was only looking within the same <major.minor> pathc version. We change it to rely on the fact that our pre-commit checks ensure this map is ordered, and iterate over the dictionary reversed, and when we find the first thing less than the target revision we use that (direct equal is handled already above) 2) The `ab_*` tables not existing were blocking the migration. Part of this is now fixable manually with apache#54227, but I have decided that since FAB was required and the only option in 2.x, so I have decided to just create the tables if they are missing In order to try and cope with possible future changes I create the tables at the latest version and then downgrade to the oldest known revision. This is all handled in a `reset_to_2_x()` method on the FABDBManager, with a fallback to just blindly create the tables from the ORM for versions of the provider that don't yet have that function.
This never made sense, and wasn't actually called as part of the `airflow db downgrade` CLI calls. The reason it doesn't make sense is that the version you pass is either the Airflow version (but external DB managers are installed and versioned separately) or the migration revision ID for the Airflow Core meta db. For FAB specifically there is the `airflow fab-db` CLI command to manage things, so "checking RunDBManager doesn't run Fab migrations" doesn't make sense as a test now (as the code that _could_ do it is removed), so I've removed the test too.
ashb
added
the
full tests needed
ashb
added
affected_version:3.0
eladkal
approved these changes
Aug 12, 2025
18 tasks
ashb
added a commit
to astronomer/airflow
that referenced
this pull request
Aug 12, 2025
* Allow downgrading to 2.11 from 3.x There were two things blocking this: 1) The revision heads map didn't have any 2.11.x versions in it, so the previous implementation of `_get_version_revision` was only looking within the same <major.minor> pathc version. We change it to rely on the fact that our pre-commit checks ensure this map is ordered, and iterate over the dictionary reversed, and when we find the first thing less than the target revision we use that (direct equal is handled already above) 2) The `ab_*` tables not existing were blocking the migration. Part of this is now fixable manually with apache#54227, but I have decided that since FAB was required and the only option in 2.x, so I have decided to just create the tables if they are missing In order to try and cope with possible future changes I create the tables at the latest version and then downgrade to the oldest known revision. This is all handled in a `reset_to_2_x()` method on the FABDBManager, with a fallback to just blindly create the tables from the ORM for versions of the provider that don't yet have that function. * Remove `downgrade` from the RunDBManager interface This never made sense, and wasn't actually called as part of the `airflow db downgrade` CLI calls. The reason it doesn't make sense is that the version you pass is either the Airflow version (but external DB managers are installed and versioned separately) or the migration revision ID for the Airflow Core meta db. For FAB specifically there is the `airflow fab-db` CLI command to manage things, so "checking RunDBManager doesn't run Fab migrations" doesn't make sense as a test now (as the code that _could_ do it is removed), so I've removed the test too. (cherry picked from commit 1d04f09)
ashb
added a commit
to astronomer/airflow
that referenced
this pull request
Aug 12, 2025
* Allow downgrading to 2.11 from 3.x There were two things blocking this: 1) The revision heads map didn't have any 2.11.x versions in it, so the previous implementation of `_get_version_revision` was only looking within the same <major.minor> pathc version. We change it to rely on the fact that our pre-commit checks ensure this map is ordered, and iterate over the dictionary reversed, and when we find the first thing less than the target revision we use that (direct equal is handled already above) 2) The `ab_*` tables not existing were blocking the migration. Part of this is now fixable manually with apache#54227, but I have decided that since FAB was required and the only option in 2.x, so I have decided to just create the tables if they are missing In order to try and cope with possible future changes I create the tables at the latest version and then downgrade to the oldest known revision. This is all handled in a `reset_to_2_x()` method on the FABDBManager, with a fallback to just blindly create the tables from the ORM for versions of the provider that don't yet have that function. * Remove `downgrade` from the RunDBManager interface This never made sense, and wasn't actually called as part of the `airflow db downgrade` CLI calls. The reason it doesn't make sense is that the version you pass is either the Airflow version (but external DB managers are installed and versioned separately) or the migration revision ID for the Airflow Core meta db. For FAB specifically there is the `airflow fab-db` CLI command to manage things, so "checking RunDBManager doesn't run Fab migrations" doesn't make sense as a test now (as the code that _could_ do it is removed), so I've removed the test too. (cherry picked from commit 1d04f09)
ashb
added a commit
to astronomer/airflow
that referenced
this pull request
Aug 12, 2025
* Allow downgrading to 2.11 from 3.x There were two things blocking this: 1) The revision heads map didn't have any 2.11.x versions in it, so the previous implementation of `_get_version_revision` was only looking within the same <major.minor> pathc version. We change it to rely on the fact that our pre-commit checks ensure this map is ordered, and iterate over the dictionary reversed, and when we find the first thing less than the target revision we use that (direct equal is handled already above) 2) The `ab_*` tables not existing were blocking the migration. Part of this is now fixable manually with apache#54227, but I have decided that since FAB was required and the only option in 2.x, so I have decided to just create the tables if they are missing In order to try and cope with possible future changes I create the tables at the latest version and then downgrade to the oldest known revision. This is all handled in a `reset_to_2_x()` method on the FABDBManager, with a fallback to just blindly create the tables from the ORM for versions of the provider that don't yet have that function. * Remove `downgrade` from the RunDBManager interface This never made sense, and wasn't actually called as part of the `airflow db downgrade` CLI calls. The reason it doesn't make sense is that the version you pass is either the Airflow version (but external DB managers are installed and versioned separately) or the migration revision ID for the Airflow Core meta db. For FAB specifically there is the `airflow fab-db` CLI command to manage things, so "checking RunDBManager doesn't run Fab migrations" doesn't make sense as a test now (as the code that _could_ do it is removed), so I've removed the test too. (cherry picked from commit 1d04f09)
ashb
added a commit
that referenced
this pull request
Aug 13, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(This is a "re-run" of #54231 which got reverted due to failing on Python 3.13)
There were two things blocking this:
The revision heads map didn't have any 2.11.x versions in it, so the
previous implementation of
_get_version_revisionwas only looking withinthe same
<major>.<minor>patch version.We change it to rely on the fact that our pre-commit checks ensure this map
is ordered, and iterate over the dictionary reversed, and when we find the
first thing less than the target revision we use that (direct equal is
handled already above)
The
ab_*tables not existing were blocking the migration. Part of this isnow fixable manually with Create FAB's user/role tables on migration, not only on initdb #54227, but I have decided that since FAB was
required and the only option in 2.x, so I have decided to just create the
tables if they are missing
In order to try and cope with possible future changes I create the tables
at the latest version and then downgrade to the oldest known revision.
This is all handled in a
reset_to_2_x()method on the FABDBManager, witha fallback to just blindly create the tables from the ORM for versions of
the provider that don't yet have that function.