Skip to content

Add proper dependencies for sqlalchemy - dependent providers #59895

@potiuk

Description

@potiuk

Some of our providers have direct imports from sqlalchemy:

The sqlalchemy dependency there is usualy for very small part of the functionality of those providers, so it should be "Optional Provider Feature" rather than being effectively "mandatory" dependency. This is not a problem today - but when we have task isolation in place, task-sdk will have no sqlalchemy as dependency, so those providers should have optional sqlalchemy extra + handling importing the hooks when sqlalchemy is not installed (i.e. either raising OptionalProviderFeatureException when used, and handling fallback to import errors when sqlalchemy import fails.

The way how it should be done:

  • for those providers that "need" sqlalchemy badly - sqlalchemy (same specification as in airflow-core now) should be added as "required" dependency

  • for those providers where sqlalchemy is just for some feature, we should make sure that they handle lack of sqlalchemy installed - handing import error, and raising "Optional Provider Feature" exception when a method that needs sqlalchemy is callsed and sqlalchemy is not installed. Sqlalchemy should be added as "extra" sqlalchemy dependency.

Sub-issues

Metadata

Metadata

Assignees

Type

No type

Projects

Status

In progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions