Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLN: remove checks for python < 3.8 #44223

Merged
merged 2 commits into from
Oct 29, 2021
Merged

CLN: remove checks for python < 3.8 #44223

merged 2 commits into from
Oct 29, 2021

Conversation

twoertwein
Copy link
Member

No description provided.

Copy link
Member

@MarcoGorelli MarcoGorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@simonjayhawkins simonjayhawkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @twoertwein for the cleanup

(If we wanted to generate stub files in the future instead of making pandas a pytyped library, we may want to generate those stubs from the inline types using stubgen instead of manually curating those files. In #28142 there has been some suggestion that pandas takes on the microsoft pyright types. IMO having pandas maintaining stubs files would allow this to happen much sooner than pandas will be ready to be a pytyped library. If that were to be the situation we may want to keep the types for the deprecated python versions on master around a bit longer. There is quite a few ifs here, so am also happy to merge this.)

@simonjayhawkins simonjayhawkins added Clean Typing type annotations, mypy/pyright type checking labels Oct 29, 2021
@simonjayhawkins simonjayhawkins added this to the 1.4 milestone Oct 29, 2021
@twoertwein
Copy link
Member Author

we may want to generate those stubs from the inline types using stubgen instead of manually curating those files.

I think that would be great (for cython), but are there tools that can do that for cython/c-extensions? If I read #28142 and pyright's recommendations correctly: prefer inline annotations, but fallback to pyi files (for cython and so on).

In #28142 there has been some suggestion that pandas takes on the microsoft pyright types.

#43744 contains the first(?) batch of pyi files from microsoft for cython files in _libs.

IMO having pandas maintaining stubs files would allow this to happen much sooner than pandas will be ready to be a pytyped library. If that were to be the situation we may want to keep the types for the deprecated python versions on master around a bit longer.

I don't think I understand: does pandas plan to release type annotations for previous releases of pandas? Many typing features require 3.8 which is also luckily the oldest supported version of pandas.

@simonjayhawkins
Copy link
Member

we may want to generate those stubs from the inline types using stubgen instead of manually curating those files.

I think that would be great (for cython), but are there tools that can do that for cython/c-extensions? If I read #28142 and pyright's recommendations correctly: prefer inline annotations, but fallback to pyi files (for cython and so on).

Sorry that I was not explicit here. If we created stubs (big if) automatically we would probably want to use stubgen on the python code and then also probably copy the cython stubs into the package since these are already curated. (Rather than have a completely seperate repo) So pandas master would be the source of truth.

And yes, we are currently on the path of eventually making pandas a pytyped library. Which is why I'm happy that this cleanup is merged. (but OTOH there is also no harm keeping these stub files as is for now)

I don't think I understand: does pandas plan to release type annotations for previous releases of pandas? Many typing features require 3.8 which is also luckily the oldest supported version of pandas.

In #28142 there has been some suggestion that pandas takes on the microsoft pyright types.

#43744 contains the first(?) batch of pyi files from microsoft for cython files in _libs.

We have started to integrate the types, but the task is not complete until microsoft stop bundling pandas types with pyright. (I may have my config incorrect, but I cannot use pyright in vs code when developing pandas code since goto definition always goes to the type libraries and not the code that I am working with)

IMO having pandas maintaining stubs files would allow this to happen much sooner than pandas will be ready to be a pytyped library. If that were to be the situation we may want to keep the types for the deprecated python versions on master around a bit longer.

I don't think I understand: does pandas plan to release type annotations for previous releases of pandas? Many typing features require 3.8 which is also luckily the oldest supported version of pandas.

stubs (as opposed to inline types) are not restricted by python versions for type annotation syntax. Type checking tends to happen using the latest version of a type checker which normally supports all type annotations syntax. Stub files tend to contain types annotations for all supported/previous versions of a software package.

When pandas becomes a pytyped library only that version of pandas has type annotations.

If (big if) pandas released stubs, all code using previous versions of pandas could also be checked. (if we decided too of course)

@twoertwein
Copy link
Member Author

We have started to integrate the types, but the task is not complete until microsoft stop bundling pandas types with pyright. (I may have my config incorrect, but I cannot use pyright in vs code when developing pandas code since goto definition always goes to the type libraries and not the code that I am working with)

For my personal projects, I run
python -c "import pandas; import pathlib; (pathlib.Path(pandas.__path__[0]) / 'py.typed').touch()"
to fool mypy and pyright into thinking that pandas is already a py.typed library. They will then both use the inline and pandas's pyi files. Works at least for pyright, not sure about pylance.

@simonjayhawkins
Copy link
Member

For my personal projects, I run
python -c "import pandas; import pathlib; (pathlib.Path(pandas.__path__[0]) / 'py.typed').touch()"
to fool mypy and pyright into thinking that pandas is already a py.typed library. They will then both use the inline and pandas's pyi files. Works at least for pyright, not sure about pylance.

Good tip. (maybe can put that in the docs somewhere)

I was referring to working with pandas itself. So the open folder is pandas. Not sure how to fool mypy and pylance in this case. I just use Jedi for the language server instead.

@twoertwein
Copy link
Member Author

I was referring to working with pandas itself. So the open folder is pandas. Not sure how to fool mypy and pylance in this case. I just use Jedi for the language server instead.

I would hope that python -c "import pathlib; (pathlib.Path('pandas/py.typed').touch()" or simply touch pandas/py.typed does the trick for the git repro. Haven't tested that.

@simonjayhawkins
Copy link
Member

I would hope that python -c "import pathlib; (pathlib.Path('pandas/py.typed').touch()" or simply touch pandas/py.typed does the trick for the git repro. Haven't tested that.

Ahh yes. maybe could add a py.typed file locally and then update (and merge to master) gitignore so that pandas devs could have a pytped file locally that doesn't get accidently merged to master.

@jreback jreback merged commit b2055b1 into pandas-dev:master Oct 29, 2021
@jreback
Copy link
Contributor

jreback commented Oct 29, 2021

thanks @twoertwein

@twoertwein twoertwein deleted the compat branch April 1, 2022 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Typing type annotations, mypy/pyright type checking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants