Fix migration issues and tighten the CI upgrade/downgrade test #25869
Conversation
|
I think those updated tests are detecting one more problem with downgrade :) |
ephraimbuddy
force-pushed
the
fix-typo-migration
branch
from
27df218 to
7006ec8
Compare
Yes. A couple more problem. I'm attempting fixes now and tightening the tests further |
ephraimbuddy
force-pushed
the
fix-typo-migration
branch
3 times, most recently
from
b1ca1eb to
2cacff9
Compare
ephraimbuddy
changed the title
ephraimbuddy
force-pushed
the
fix-typo-migration
branch
from
c37f674 to
a908c39
Compare
airflow/migrations/versions/0113_2_4_0_compare_types_between_orm_and_db.py
Outdated
Show resolved
Hide resolved
ephraimbuddy
force-pushed
the
fix-typo-migration
branch
from
74fbf93 to
d8e7710
Compare
|
It looks like the failing tests are not caused by the PR. Have rerun it before and rebased but it keeps failing cc: @potiuk |
The PRs that are changing "build" scripts running in "upgrade to newer deps" mode. The "warnings" issues (caused by new urrllib3 should be handled by #25885 (please approve). I am not sure what PRod image failure is about (lookign into it). |
|
Possible fix to Google failing prod image tests: #25886 (BTW. It's great we have the tests - they are finding some unexpected problems our users would have only see after they got the release in their hands - seems that there is an implicit dependency in google provider's dependencies on protobuf version) |
|
Should be fixed now. please rebase @ephraimbuddy :) |
There was a typo that was not detected in the migration file which prevented upgrades. On fixing the typo, some other migration issues were detected. Apart from postgres, other dbs couldn't upgrade -> downgrade -> upgrade. This was partly because we now create new DB from the ORM which also uses a naming convention to create constraints. I have applied a fix, the side effect is that it breaks offline generation of sql for mysql. This is because it uses the new naming convention to create constraints when we run upgrade through the migration file. In migration file 0093, we dropped unique keys with names dag_id & dag_id_2 while with the naming convention in place, these two now have a unique name across all db that's different from the names we use to drop the keys. Because of that, I have chosen to use sqlalchemy inspector to get the name and drop them.
ephraimbuddy
force-pushed
the
fix-typo-migration
branch
from
d8e7710 to
53b8a39
Compare
Thanks |
ephraimbuddy
added
the
type:misc/internal
There was a typo that was not detected in the migration file which prevented upgrades.
On fixing the typo, some other migration issues were detected.
Apart from postgres, other dbs couldn't upgrade -> downgrade -> upgrade. This was partly
because we now create new DB from the ORM which also uses a naming convention to create constraints.
I have applied a fix, the side effect is that it breaks offline generation of sql for mysql. This is because it uses the new naming convention to create constraints when we run upgrade through the migration file. In
migration file 0093, we dropped unique keys with names dag_id & dag_id_2 while with the naming
convention in place, these two now have a unique name across all db that's different from the names
we use to drop the keys. Because of that, I have chosen to use sqlalchemy inspector to get the name
and drop them.