Skip to content

Conversation

@mohit7705
Copy link
Contributor

Closes #59905

What does this PR do?

Adds SQLAlchemy as an optional dependency for the Postgres provider.

Why is this needed?

This aligns the Postgres provider with task isolation requirements, where SQLAlchemy should not be a mandatory dependency unless explicitly required.

@boring-cyborg
Copy link

boring-cyborg bot commented Dec 30, 2025

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@potiuk
Copy link
Member

potiuk commented Dec 31, 2025

It's quite a bit more than that - see at other PRs in sibling issues (conditional handling of imports and sqlalchemy version difference)

@mohit7705
Copy link
Contributor Author

Thanks for the guidance!
I’ve aligned the Postgres provider changes with the conditional import pattern and optional SQLAlchemy handling used in sibling provider PRs.
Please let me know if you’d like me to adjust anything further.

@mohit7705
Copy link
Contributor Author

Thanks for the review!
I’ve fixed the indentation issue and updated the branch with latest main.
Could you please take another look when you have time?

@jscheffl
Copy link
Contributor

Can you please - without just shooting code as a PR - run basic tests locally via prek? Basic indent is not fitting again and again. All would be fixed with local checks before pushing.

@potiuk
Copy link
Member

potiuk commented Dec 31, 2025

Yes. It adds a lot of overhead if you dont follow basic contributing guides - this makes us loose more time to review and react to it than if we did it ourselves, and you do not seem to be self-reliant on fixing even those issues that are fixable automatically by prek

Not very helpful

Copy link
Contributor

@Srabasti Srabasti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 92 should be referencing "sqlalchemy>=1.4.49" instead of "sqlalchemy>=1.4".
Happy New Year!

@potiuk potiuk requested a review from Srabasti January 4, 2026 00:08
@Srabasti
Copy link
Contributor

Srabasti commented Jan 4, 2026

Thanks for including the changes for Line 92 @mohit7705! However, looks like static checks are failing as below.
image

As Jens and Jarek advised above, have you reproduced locally by running prek run --all-files in your branch?
Prek will fix any formatting errors, and then you can push the commit from your branch.

To run prek as part of git workflow, use prek install to set up git hooks.
image

@mohit7705 mohit7705 force-pushed the fix-postgres-optional-sqlalchemy branch from cf27cf4 to e280e2c Compare January 4, 2026 13:39
@potiuk
Copy link
Member

potiuk commented Jan 4, 2026

Indeed, fixing the other static checks needed

@mohit7705 mohit7705 force-pushed the fix-postgres-optional-sqlalchemy branch from 109bef9 to 07d41e9 Compare January 4, 2026 14:05
from psycopg2.extras import DictCursor, NamedTupleCursor, RealDictCursor, execute_batch
from sqlalchemy.engine import URL

try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type-hinting will be needed of the URL (making it Any).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the type hinting - not setting "Any"

URL: Any baically.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you verify once again @potiuk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification, this makes sense 👍

I see the earlier approach and the alternative with lazy imports you linked. I’m happy to align with the preferred direction — please let me know if you’d like me to update this PR to use lazy imports instead, or if the current approach is acceptable as-is.

@mohit7705 mohit7705 requested review from jscheffl and potiuk January 4, 2026 22:12
from airflow.providers.common.sql.hooks.sql import DbApiHook
from airflow.providers.postgres.dialects.postgres import PostgresDialect

USE_PSYCOPG3: bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actusally deleting it is wrong. The conditional logic should still be used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification — that makes sense 👍
I’ll restore the conditional logic and adjust the SQLAlchemy import to be MyPy-safe while keeping it optional.
I’ll push an updated commit shortly.

@property
def sqlalchemy_url(self) -> URL:
try:
from sqlalchemy.engine import URL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will likely fail with MyPy issues.

@mohit7705 mohit7705 closed this by deleting the head repository Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add proper dependencies for sqlalchemy - postgres

5 participants