Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve ambiguous type annotations #22

Merged
merged 10 commits into from
May 21, 2024
Merged

Improve ambiguous type annotations #22

merged 10 commits into from
May 21, 2024

Conversation

aazuspan
Copy link
Contributor

I noticed some ambiguous types being inferred in VS Code during usage, so I spent a little time trying to improve those. That basically boiled down to:

  1. Using @overload to specify return types that depend on input parameters and
  2. Using some PEP 612 magic to prevent type annotations getting lost by the decorators.

To demonstrate, here are the inferred types in grey (activated with the python.analysis.inlayHints.variableTypes setting in VS Code) before the typing changes:
type_annotations_main

And after:
type_annotations_fixed

Aside from adding some code complexity, the downside of these changes that I realized too late is that PEP 612 wasn't implemented until Python 3.10, so I had to temporarily add typing-extensions (an official backport of new typing features) as a dependency so that we can continue supporting 3.9. Once we eventually drop 3.9, we could also drop typing-extensions.

The other alternatives would be to hold off on this PR or drop 3.9 support now. The Numpy support schedule recommended dropping 3.9 support a couple months ago, so that's probably not unreasonable, but it does feel a little restrictive considering we don't have any older releases for compatibility. Let me know if you have any strong feelings either way @grovduck!

- Ditched the type aliases that were causing the TypeVar parsing
  to fail.
- Made the decorated AttrWrapper generic
We now rely on the ParamSpec and Concatenate features of PEP 612
which weren't implemented until Python 3.10. In order to use those
features while supporting Python 3.9, we can temporarily replace
any typing imports with backported features from typing-extensions.
Once we drop 3.9 support, we can switch back to typing and drop the
dependency.
@aazuspan aazuspan added the enhancement New feature or request label May 20, 2024
@aazuspan aazuspan requested a review from grovduck May 20, 2024 20:31
@aazuspan aazuspan self-assigned this May 20, 2024
Copy link
Member

@grovduck grovduck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aazuspan, very cool stuff. I was able to recreate your initial writeup after I activated python.analysis.inlayHints.variableTypes and it worked perfectly.

I think I'd lean toward pushing this now and relying on typing_extensions, but admittedly I don't know the negative consequences of doing that. Like you, it seems to make sense to me to get at least a single release at 3.9 before pulling support for it. That hasn't seemed to impact us too much recently.

Thanks for improving the development experience!

src/sknnr_spatial/utils/estimator.py Show resolved Hide resolved
src/sknnr_spatial/types.py Show resolved Hide resolved
src/sknnr_spatial/estimator.py Outdated Show resolved Hide resolved
src/sknnr_spatial/datasets/_base.py Outdated Show resolved Hide resolved
To infer the correct type when `as_dataset` is unspecified,
we shouldn't give it a default value in the `True` case.
The number of output targets for kneighbors must be n_neighbors when
it's specified.
@aazuspan
Copy link
Contributor Author

Thanks for the review and catching my mistakes @grovduck! I just pushed a few commits that should fix the outstanding issues with the kneighbors signature, as well as a small typing error in load_swo_ecoplots. Assuming everything looks good to you, I think that wraps up this PR.

I think I'd lean toward pushing this now and relying on typing_extensions, but admittedly I don't know the negative consequences of doing that. Like you, it seems to make sense to me to get at least a single release at 3.9 before pulling support for it. That hasn't seemed to impact us too much recently.

Makes sense to me, thanks for weighing in! I think I'm overly cautious about adding dependencies, and there shouldn't be any practical issues with using typing_extensions as a temporary fix.

@grovduck
Copy link
Member

grovduck commented May 21, 2024

I just pushed a few commits that should fix the outstanding issues with the kneighbors signature

Sorry to be dense here, but now that you have n_neighbors and return_distance explicit in the signatures, there aren't any additional keyword arguments that sklearn's kneighbors is expecting, right? Do you still need **kneighbors_kwargs in the signature? Am I missing something?

The change to (and new test for) DaskBackedWrapper.kneighbors looks good to me!

@aazuspan
Copy link
Contributor Author

Do you still need **kneighbors_kwargs in the signature?

Only to support extended kneighbors methods with extra parameters, like our return_dataframe_index 😉

@aazuspan aazuspan merged commit 79bc028 into main May 21, 2024
5 checks passed
@aazuspan aazuspan deleted the type-annotations branch May 21, 2024 19:55
@grovduck
Copy link
Member

Oh yeah, that one! All looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants