Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Universal-pathlib 0.2.3 seems to break compatibility with 0.2.2 (at least breaks mypy checks). #41723

Closed
1 task done
potiuk opened this issue Aug 25, 2024 · 4 comments · Fixed by #41921
Closed
1 task done
Labels
area:core area:providers kind:bug This is a clearly a bug kind:meta High-level information important to the community

Comments

@potiuk
Copy link
Member

potiuk commented Aug 25, 2024

Body

Manual MyPy Airflow:

 airflow/io/path.py:201: error: Unexpected keyword argument "overwrite" for
"rename" of "UPath"  [call-arg]
            return self.rename(target, overwrite=True)
                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Manual MyPy Providers:

airflow/providers/common/io/xcom/backend.py:145: error: Argument 1 to
"joinpath" of "UPath" has incompatible type "Optional[str]"; expected
"Union[str, PathLike[str]]"  [arg-type]
                p = base_path.joinpath(dag_id, run_id, task_id, f"{uuid.uu...
                                       ^~~~~~
airflow/providers/common/io/xcom/backend.py:145: error: Argument 2 to
"joinpath" of "UPath" has incompatible type "Optional[str]"; expected
"Union[str, PathLike[str]]"  [arg-type]
                p = base_path.joinpath(dag_id, run_id, task_id, f"{uuid.uu...
                                               ^~~~~~
airflow/providers/common/io/xcom/backend.py:145: error: Argument 3 to
"joinpath" of "UPath" has incompatible type "Optional[str]"; expected
"Union[str, PathLike[str]]"  [arg-type]
    ...          p = base_path.joinpath(dag_id, run_id, task_id, f"{uuid.uuid...
                                                        ^~~~~~~

Should be reviewed and fixed.

cc: @jscheffl @bolkedebruin

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@potiuk
Copy link
Member Author

potiuk commented Aug 28, 2024

Seems that this one might be fixed by fsspec/universal_pathlib#257

@jscheffl
Copy link
Contributor

Seems that this one might be fixed by fsspec/universal_pathlib#257

That fixes Problem #1, the second problem that we call the join with Optional[str] because we allow None values passed for dag_id, run_id and task_id is not fixed. I checked this but the interface is inherited from XCom and I assume that is something we need to fix on our side.

@ap--
Copy link
Contributor

ap-- commented Aug 29, 2024

Ping me once you fix the Optional[str] issue on your side. I can then release a new intermediate version of universal-pathlib with the #257 fix.

@potiuk
Copy link
Member Author

potiuk commented Sep 1, 2024

Ping me once you fix the Optional[str] issue on your side. I can then release a new intermediate version of universal-pathlib with the #257 fix.

I don't think it was "serious" but I protected against it in #41921 @ap-- - I guess it will pass this time, feel free to relase a new version/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core area:providers kind:bug This is a clearly a bug kind:meta High-level information important to the community
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants