Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove upper-binding for SQLAlchemy #24819

Merged
merged 1 commit into from
Jul 5, 2022

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Jul 4, 2022

There was a problem with custom classes for SQLAlchemy that
prevented it to work on MySQL. This PR removes the SQLAlchemy
upper binding.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst, in newsfragments.

@potiuk
Copy link
Member Author

potiuk commented Jul 4, 2022

Let's see if the problem has been fixed in the meantime

@potiuk potiuk added the full tests needed We need to run full set of tests for this PR to merge label Jul 4, 2022
@potiuk potiuk force-pushed the remove-upper-binding-on-sqlalchemy branch from f437518 to 239f858 Compare July 4, 2022 07:53
@potiuk potiuk marked this pull request as ready for review July 4, 2022 14:15
@potiuk potiuk force-pushed the remove-upper-binding-on-sqlalchemy branch from 239f858 to 691d1ee Compare July 4, 2022 14:15
@potiuk
Copy link
Member Author

potiuk commented Jul 4, 2022

Hey @ephraimbuddy @dstandish - I got the tests for MSSQL workiig for latest SQLalachemy but I am not sure if the solution to the failing tetst were good.

There were a number tests added recently to those db migration scripts (good!) and apparently when I migrated to latest sqlalchemy the previous problem mentioned in #21272 is solved but this one tests comparing the schema with the ORM information failled. I am not sure if I fixed it correctly, the root cause of this is likely this one:

https://github.com/apache/airflow/blob/main/airflow/migrations/versions/0093_2_2_0_taskinstance_keyed_to_dagrun.py#L278

I believe the CASCADE modification was the problem. Both "diffs" that I ignored had "CASCADE" ondelete, where we are definitely setting NO_ACTION for MsSQL due to the https://docs.microsoft.com/en-us/sql/relational-databases/errors-events/mssqlserver-1785-database-engine-error?view=sql-server-ver15

So I guess for some reason our "diff" script expects CASCADE but gets NO_ACTION,

I just ignored the offending differences - similarly to others tests, but maybe this is an indication of some other problem we have (or maybe our diff is not robust enough to account for differences between the dialects?)

Any insight and comments (and maybe approval if I did it correctly) is appreciated.

There was a problem with custom classes for SQLAlchemy that
prevented it to work on MySQL. This PR removes the SQLAlchemy
upper binding.

This has been added as an issue in:

sqlalchemy/sqlalchemy#7660

But apparently it's been fixed in one of the more recent SQLAlchemy
releases.
@potiuk potiuk force-pushed the remove-upper-binding-on-sqlalchemy branch from 691d1ee to ecdf53f Compare July 4, 2022 14:25
@potiuk potiuk added this to the Airflow 2.3.4 milestone Jul 4, 2022
@potiuk potiuk requested review from ashb and jedcunningham July 4, 2022 14:50
@potiuk
Copy link
Member Author

potiuk commented Jul 4, 2022

All Green.

@potiuk potiuk merged commit 0853d60 into apache:main Jul 5, 2022
@potiuk potiuk deleted the remove-upper-binding-on-sqlalchemy branch July 5, 2022 09:13
@potiuk potiuk modified the milestones: Airflow 2.3.4, Airflow 2.3.3 Jul 5, 2022
potiuk added a commit that referenced this pull request Jul 5, 2022
There was a problem with custom classes for SQLAlchemy that
prevented it to work on MySQL. This PR removes the SQLAlchemy
upper binding.

This has been added as an issue in:

sqlalchemy/sqlalchemy#7660

But apparently it's been fixed in one of the more recent SQLAlchemy
releases.

(cherry picked from commit 0853d60)
@ephraimbuddy ephraimbuddy added the type:misc/internal Changelog: Misc changes that should appear in change log label Jul 5, 2022
ephraimbuddy pushed a commit that referenced this pull request Jul 5, 2022
There was a problem with custom classes for SQLAlchemy that
prevented it to work on MySQL. This PR removes the SQLAlchemy
upper binding.

This has been added as an issue in:

sqlalchemy/sqlalchemy#7660

But apparently it's been fixed in one of the more recent SQLAlchemy
releases.

(cherry picked from commit 0853d60)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
full tests needed We need to run full set of tests for this PR to merge type:misc/internal Changelog: Misc changes that should appear in change log
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants