missing pyarrow dependency in google provider?

[saucoide]

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

6.1 but looks it's the same in >10

Apache Airflow version

2.2

Operating System

tested linux, macos

Deployment

Virtualenv installation

Deployment details

linux/macos & uv pip to install the packages

What happened

I dont know if im missing some obvious reason for this, but pyarrow is not specified as a dependency for the google provider, while it definetly depends on it: https://github.com/apache/airflow/blob/main/providers/src/airflow/providers/google/cloud/transfers/sql_to_gcs.py#L29

If i do an install in a venv with:

dependencies = [
	"apache-airflow==2.2",
	"apache-airflow-providers-google==6.3",
	"google-cloud-bigquery>=1"
]

pyarrow won't be installed, and importing from sql_to_gcs will raise a exception

if i remove google-cloud-bigquery, it WILL be installed, i have no idea what causes this behavior since google-cloud-bigquery does list pyarrow as a dependency. But the version is installed is due to pandas-gbq and depending on it

What you think should happen instead

IMO if a package is used directly, then it's a direct dependency and it shouldn't rely on it being available via indirect dependencies

I can just add pyarrow myself and solve my problem, but i think the dependency should be explicitly defined in the provider

How to reproduce

[project]
name = "tests"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
	"apache-airflow==2.2",
	"apache-airflow-providers-google==6.3",
	"google-cloud-bigquery>=1"
]

uv pip install .

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Sure - feel free to add it to "provider.yaml" in "providers/google/..." folder. Marked it as a good first issue to fix. Just add it in the same way as in other provider.yaml files:

 - pyarrow>=14.0.1

[potiuk]

BTW. Temporary workaround - until you fix it and we relase a new provider - is o explicitly add "pyarrow>=14.0.1" in your dependencies.