Common SQL provider depends on the `methodtools` package

[tatiana]

Apache Airflow Provider(s)

common-sql

Versions of Apache Airflow Providers

apache-airflow-providers-common-sql==1.23.0 (released on 26 February 2025)

Apache Airflow version

2.9

Operating System

Any

Deployment

Other

Deployment details

apache-airflow==2.9.0
apache-airflow-providers-amazon==8.19.0
apache-airflow-providers-cncf-kubernetes==8.0.1
apache-airflow-providers-common-compat==1.5.0
apache-airflow-providers-common-io==1.3.0
apache-airflow-providers-common-sql==1.23.0
apache-airflow-providers-docker==3.9.2
apache-airflow-providers-fab==1.0.2
apache-airflow-providers-ftp==3.7.0
apache-airflow-providers-google==14.0.0
apache-airflow-providers-http==4.10.0
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-microsoft-azure==12.2.0
apache-airflow-providers-postgres==5.10.2
apache-airflow-providers-smtp==1.6.1
apache-airflow-providers-sqlite==3.7.1

What happened

Since apache-airflow-providers-common-sql==1.23.0 (#41327), the provider seems to depend on methodtools.

Previously, we would not face issues running the following in Cosmos' CI:

from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator

But since the last release of apache-airflow-providers-common-sql, the Cosmos' CI started raising the following error:

_________ ERROR collecting tests/operators/_asynchronous/test_base.py __________
ImportError while importing test module '/home/runner/work/astronomer-cosmos/astronomer-cosmos/tests/operators/_asynchronous/test_base.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.12-2.9/lib/python3.12/site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
../../../.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.12-2.9/lib/python3.12/site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1387: in _gcd_import
    ???
<frozen importlib._bootstrap>:1360: in _find_and_load
    ???
<frozen importlib._bootstrap>:1331: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:935: in _load_unlocked
    ???
../../../.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.12-2.9/lib/python3.12/site-packages/_pytest/assertion/rewrite.py:184: in exec_module
    exec(co, module.__dict__)
tests/operators/_asynchronous/test_base.py:11: in <module>
    from cosmos.operators._asynchronous.bigquery import DbtRunAirflowAsyncBigqueryOperator
cosmos/operators/_asynchronous/bigquery.py:8: in <module>
    from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator
../../../.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.12-2.9/lib/python3.12/site-packages/airflow/providers/google/cloud/operators/bigquery.py:32: in <module>
    from airflow.providers.common.sql.operators.sql import (  # type: ignore[attr-defined] # for _parse_boolean
../../../.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.12-2.9/lib/python3.12/site-packages/airflow/providers/common/sql/operators/sql.py:[29](https://github.com/astronomer/astronomer-cosmos/actions/runs/13544957799/job/37854234771#step:6:30): in <module>
    from airflow.providers.common.sql.hooks.sql import DbApiHook, fetch_all_handler, return_single_query_results
../../../.local/share/hatch/env/virtual/astronomer-cosmos/Za_bFbg4/tests.py3.12-2.9/lib/python3.12/site-packages/airflow/providers/common/sql/hooks/sql.py:37: in <module>
    from methodtools import lru_cache
E   ModuleNotFoundError: No module named 'methodtools'

https://github.com/astronomer/astronomer-cosmos/actions/runs/13544957799/job/37854234771

Which is gone if we install methodtools as part of Cosmos dependencies:
astronomer/astronomer-cosmos@d6613ec

What you think should happen instead

I believe libraries that libraries such as Cosmos that do:

from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator

Should not have to add methodtools as their dependency.

How to reproduce

Install:

apache-airflow==2.9.0
apache-airflow-providers-amazon==8.19.0
apache-airflow-providers-cncf-kubernetes==8.0.1
apache-airflow-providers-common-compat==1.5.0
apache-airflow-providers-common-io==1.3.0
apache-airflow-providers-common-sql==1.23.0
apache-airflow-providers-docker==3.9.2
apache-airflow-providers-fab==1.0.2
apache-airflow-providers-ftp==3.7.0
apache-airflow-providers-google==14.0.0
apache-airflow-providers-http==4.10.0
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-microsoft-azure==12.2.0
apache-airflow-providers-postgres==5.10.2
apache-airflow-providers-smtp==1.6.1
apache-airflow-providers-sqlite==3.7.1

And try to run:

from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator

Anything else

This issue did not happen previously, with apache-airflow-providers-common-sql==1.21.0:

apache-airflow-providers-amazon==8.19.0
apache-airflow-providers-cncf-kubernetes==8.0.1
apache-airflow-providers-common-compat==1.3.0
apache-airflow-providers-common-io==1.3.0
apache-airflow-providers-common-sql==1.21.0
apache-airflow-providers-docker==3.9.2
apache-airflow-providers-fab==1.0.2
apache-airflow-providers-ftp==3.7.0
apache-airflow-providers-google==12.0.0
apache-airflow-providers-http==4.10.0
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-microsoft-azure==12.0.0
apache-airflow-providers-postgres==5.10.2
apache-airflow-providers-smtp==1.6.1
apache-airflow-providers-sqlite==3.7.1

https://github.com/astronomer/astronomer-cosmos/actions/runs/13518937849/job/37773660843

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct