Skip to content

Conversation

@amoghrajesh
Copy link
Contributor

CI fails with:

  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:194: error: No overload variant of "__setitem__" of "DataFrame" matches argument types "Hashable", "Series[str]" 
  [call-overload]
                      df[col] = df[col].astype(str)
                      ^~~~~~~
  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:194: note: Possible overload variants:
  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:194: note:     def __setitem__(self, int | slice[Any, Any, Any] | ndarray[Any, dtype[integer[Any]]] | Index[Any] | list[int] | Series[int] | tuple[int, int] | tuple[slice[Any, Any, Any] | ndarray[Any, dtype[integer[Any]]] | Index[Any] | list[int] | Series[int], int] | tuple[slice[Any, Any, Any] | ndarray[Any, dtype[integer[Any]]] | Index[Any] | list[int] | Series[int], slice[Any, Any, Any] | ndarray[Any, dtype[integer[Any]]] | Index[Any] | list[int] | Series[int]] | tuple[int, slice[Any, Any, Any] | ndarray[Any, dtype[integer[Any]]] | Index[Any] | list[int] | Series[int]], str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | IndexOpsMixin[Any, Any] | Sequence[str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]] | DataFrame | ndarray[tuple[Any, ...], dtype[Any]] | NAType | NaTType | Mapping[Hashable, str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | NAType | NaTType] | None, /) -> None
  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:194: note:     def [ScalarT: str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]] __setitem__(self, Series[bool] | ndarray[Any, dtype[bool_]] | list[bool] | str | str_ | <9 more items>, str | bytes | date | datetime | timedelta | <20 more items> | None, /) -> None
  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:194: note:     def __setitem__(self, tuple[tuple[IndexOpsMixin[Any, Any] | Series[bool] | ndarray[Any, dtype[bool_]] | list[bool] | str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | Sequence[str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]] | slice[Any, Any, Any], ...], Hashable], str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | NAType | NaTType | ExtensionArray | ndarray[Any, dtype[Any]] | IndexOpsMixin[Any, Any] | Sequence[str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]] | Sequence[Sequence[str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]]] | Mapping[Hashable, str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | NAType | NaTType] | None, /) -> None
  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:194: note:     def __setitem__(self, IndexOpsMixin[Any, Any] | DataFrame, str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | NAType | NaTType | ExtensionArray | ndarray[Any, dtype[Any]] | IndexOpsMixin[Any, Any] | Sequence[str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]] | Sequence[Sequence[str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any]]] | Mapping[Hashable, str | bytes | date | datetime | timedelta | <7 more items> | complex | integer[Any] | floating[Any] | complexfloating[Any, Any] | NAType | NaTType] | None, /) -> None
  providers/amazon/src/airflow/providers/amazon/aws/transfers/sql_to_s3.py:204: error: No overload variant of "__setitem__" of "DataFrame" matches argument types "Hashable", "Series[int]" 
  [call-overload]
                          df[col] = df[col].astype(pd.Int64Dtype())

This seems to be due to pandas-stub version bump and this change: pandas-dev/pandas-stubs@3c9e243

To fix this, adding explicit cast() calls to properly handle pandas-stubs 2.3.3 stricter type checking for DataFrame
column assignments. This ensures type safety while maintaining runtime behavior as well.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@potiuk potiuk merged commit c5e70a9 into apache:main Dec 2, 2025
85 checks passed
@kacpermuda
Copy link
Contributor

I think this fix did not help, still getting the same failures after rebasing: https://github.com/apache/airflow/actions/runs/19857113583/job/56898338352?pr=58933

@potiuk
Copy link
Member

potiuk commented Dec 2, 2025

I think this fix did not help, still getting the same failures after rebasing: https://github.com/apache/airflow/actions/runs/19857113583/job/56898338352?pr=58933

Fixed in a follow-up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants