Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dependency: Python 3.11.9 breaks Dask #537

Closed

Conversation

amotl
Copy link

@amotl amotl commented Apr 13, 2024

Dear @goodwanghan and @kvnkho,

first things first: Thanks a stack for conceiving and maintaining this excellent library.

A few days ago, we discovered a flaw with PyCaret and Dask, and finally tracked it back to an incompatibility with Python 3.11.9, which was released on April 2, 2024, and is being rolled out on GHA runners by GitHub starting Apr 8, 2024, when it started to become more visible to downstream users like us.

In order to fix the problem, it is advised to update to dask>=2024.4.1, which includes a relevant fix. Those patches bring in corresponding updates on behalf of downstream applications/frameworks, validating that approach after it tripped us.

In the same spirit, this patch intends to fix Fugue on new and fresh installations, otherwise tripping users into this subtle dependency flaw. 1

With kind regards,
Andreas.

Footnotes

  1. Let us know if you absolutely need to retain compatibility with Python 3.8, which may not be taken for granted after applying this patch as is, so we may look into this detail once again.

Python 3.11.9 started to employ a different behaviour when raising
`TypeError`s when reflecting docstrings, which leads to that error when
importing `dask.dataframe`.

  TypeError: descriptor '__call__' for 'type' objects doesn't apply to
  a 'property' object

The solution is either to limit execution to Python <3.11.9, or to
upgrade the `dask` and `distributed` packages to their most recent
versions.

Because Fugue can't control on which Python versions it is used
downstream, this patch uses the option to upgrade Dask.

Because GitHub recently rolled out Python 3.11.9 on their runners, and
other downstream users may also upgrade soon, it is advisable to release
this update on behalf of a bugfix release, to avoid things going south
for users of Fugue.
Python 3.8 will become EOL 2024-10 [1]. When support should be retained,
the other patch a98199c would need to be made more conditional, so
that more recent versions of Dask are only used on more recent versions
of Python.

[1] https://devguide.python.org/versions/
@amotl amotl marked this pull request as ready for review April 13, 2024 20:52
@amotl
Copy link
Author

amotl commented Apr 15, 2024

Hi again,

this improvement made it into downstream PyCaret on behalf of pycaret/pycaret@fefe8d5fbe3c. Thank you very much, @Yard1. It will be so sweet to also see it integrated here, to provide its value to a wider audience.

With kind regards,
Andreas.

@goodwanghan
Copy link
Collaborator

@amotl my apology for the delay. And thank you so much for proposing the change.

We can't make the dask lower bound version to be the latest. Because a lot of users are not using the latest Dask. Plus, Dask (as well as Ray) tends to have very aggressive changes, which could break a lot of things.

That being said, I will incorporate your change into another PR I am working on. I will make sure Fugue + Dask works with the latest python version.

And the correspondent change will be released in Fugue 0.9.0 shortly

@amotl
Copy link
Author

amotl commented Apr 22, 2024

Dear Han,

thanks for your swift reply, and thanks for taking care. Please feel free to close this PR if it is not applicable. We are looking forward to your other updates. Good luck!

With kind regards,
Andreas.

@amotl
Copy link
Author

amotl commented May 8, 2024

Dear Han,

has this issue been addressed with GH-536 already, and published per version 0.9.0?

If so, thanks a stack, and we may want to close this PR again?

With kind regards,
Andreas.

@goodwanghan
Copy link
Collaborator

Closing because it has been fixed by 0.9.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants