Skip to content

Missing Imports #28133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
simonjayhawkins opened this issue Aug 25, 2019 · 5 comments
Closed

Missing Imports #28133

simonjayhawkins opened this issue Aug 25, 2019 · 5 comments
Labels
Closing Candidate May be closeable, needs more eyeballs Typing type annotations, mypy/pyright type checking

Comments

@simonjayhawkins
Copy link
Member

in setup.cfg we currently have

[mypy]
ignore_missing_imports=True

mypy states "We recommend using this approach only as a last resort: it’s equivalent to adding a # type: ignore to all unresolved imports in your codebase." https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports

the number of instances of missing imports resolving to Any is already significant..

$ mypy pandas --disallow-any-unimported
pandas\util\_test_decorators.py:106: error: Return type becomes "Any" due to an unfollowed import
pandas\util\_test_decorators.py:120: error: Return type becomes "Any" due to an unfollowed import
pandas\_typing.py:19: error: Constraint 5 becomes "Any" due to an unfollowed import
pandas\_typing.py:22: error: Constraint 2 becomes "Any" due to an unfollowed import
pandas\_typing.py:23: error: Constraint 1 becomes "Any" due to an unfollowed import
pandas\_typing.py:23: error: Constraint 2 becomes "Any" due to an unfollowed import
pandas\_typing.py:23: error: Constraint 3 becomes "Any" due to an unfollowed import
pandas\io\formats\printing.py:252: error: Base type BaseFormatter becomes "Any" due to an unfollowed import
pandas\core\dtypes\dtypes.py:653: error: Type of variable becomes "Type[Any]" due to an unfollowed import
pandas\core\dtypes\dtypes.py:812: error: Type of variable becomes "Type[Any]" due to an unfollowed import
pandas\core\dtypes\common.py:144: error: Return type becomes "Any" due to an unfollowed import
pandas\core\dtypes\common.py:144: error: Argument 1 to "ensure_int_or_float" becomes "Any" due to an unfollowed import
pandas\core\dtypes\common.py:181: error: Argument 1 to "ensure_python_int" becomes "Union[int, Any]" due to an unfollowed import
pandas\core\indexers.py:39: error: Argument 2 to "is_empty_indexer" becomes "Any" due to an unfollowed import
pandas\core\indexers.py:115: error: Argument 1 to "validate_indices" becomes "Any" due to an unfollowed import
pandas\core\dtypes\cast.py:765: error: Argument 1 to "maybe_convert_objects" becomes "Any" due to an unfollowed import
pandas\core\dtypes\cast.py:814: error: Argument 1 to "soft_convert_objects" becomes "Any" due to an unfollowed import
pandas\core\nanops.py:194: error: Return type becomes "Optional[Any]" due to an unfollowed import
pandas\core\nanops.py:194: error: Argument 1 to "_maybe_get_mask" becomes "Any" due to an unfollowed import
pandas\core\nanops.py:194: error: Argument 3 to "_maybe_get_mask" becomes "Optional[Any]" due to an unfollowed import
pandas\core\nanops.py:237: error: Return type becomes "Tuple[Any, Optional[Any], Any, Any, Any]" due to an unfollowed import
pandas\core\nanops.py:237: error: Argument 1 to "_get_values" becomes "Any" due to an unfollowed import
pandas\core\nanops.py:237: error: Argument 5 to "_get_values" becomes "Optional[Any]" due to an unfollowed import
pandas\core\nanops.py:634: error: Return type becomes "Tuple[Union[int, Any], Union[int, Any]]" due to an unfollowed import
pandas\core\nanops.py:634: error: Argument 2 to "_get_counts_nanvar" becomes "Optional[Any]" due to an unfollowed import
pandas\core\nanops.py:672: error: Type of variable becomes "Any" due to an unfollowed import
pandas\core\nanops.py:1117: error: Return type becomes "Union[Any, int]" due to an unfollowed import
pandas\core\nanops.py:1117: error: Argument 1 to "_maybe_arg_null_out" becomes "Any" due to an unfollowed import
pandas\core\nanops.py:1117: error: Argument 3 to "_maybe_arg_null_out" becomes "Optional[Any]" due to an unfollowed import
pandas\core\nanops.py:1141: error: Return type becomes "Union[int, Any]" due to an unfollowed import
pandas\core\nanops.py:1141: error: Argument 2 to "_get_counts" becomes "Optional[Any]" due to an unfollowed import
pandas\core\nanops.py:1185: error: Return type becomes "Any" due to an unfollowed import
pandas\core\nanops.py:1185: error: Argument 1 to "_maybe_null_out" becomes "Any" due to an unfollowed import
pandas\core\nanops.py:1185: error: Argument 3 to "_maybe_null_out" becomes "Optional[Any]" due to an unfollowed import
pandas\core\construction.py:51: error: Return type becomes "Any" due to an unfollowed import
pandas\core\construction.py:51: error: Argument 2 to "array" becomes "Union[str, Any, ExtensionDtype, None]" due to an unfollowed import
pandas\core\tools\datetimes.py:71: error: Argument 1 to "should_cache" becomes "Union[List[Any], Tuple[Any, ...], Any, Any]" due to an unfollowed import
pandas\core\tools\datetimes.py:161: error: Return type becomes "Any" due to an unfollowed import
pandas\core\tools\datetimes.py:161: error: Argument 1 to "_box_as_indexlike" becomes "Any" due to an unfollowed import
pandas\core\tools\datetimes.py:191: error: Return type becomes "Union[Any, Any]" due to an unfollowed import
pandas\core\tools\datetimes.py:191: error: Argument 1 to "_convert_and_box_cache" becomes "Union[Any, List[Any], Tuple[Any, ...], Any, Any]" due to an unfollowed import
pandas\core\tools\datetimes.py:191: error: Argument 2 to "_convert_and_box_cache" becomes "Any" due to an unfollowed import
pandas\tseries\offsets.py:157: error: Base type BaseOffset becomes "Any" due to an unfollowed import
pandas\tseries\offsets.py:2513: error: Base type _Tick becomes "Any" due to an unfollowed import
pandas\core\arrays\_ranges.py:15: error: Return type becomes "Tuple[Any, str]" due to an unfollowed import
pandas\core\arrays\_ranges.py:15: error: Argument 1 to "generate_regular_range" becomes "Any" due to an unfollowed import
pandas\core\arrays\_ranges.py:15: error: Argument 2 to "generate_regular_range" becomes "Any" due to an unfollowed import
pandas\core\ops\__init__.py:398: error: Argument 1 to "should_extension_dispatch" becomes "Any" due to an unfollowed import
pandas\core\ops\__init__.py:1152: error: Argument 1 to "maybe_dispatch_ufunc_to_dunder_op" becomes "Any" due to an unfollowed import
pandas\core\ops\__init__.py:1152: error: Argument 4 to "maybe_dispatch_ufunc_to_dunder_op" becomes "Any" due to an unfollowed import
pandas\core\arrays\period.py:230: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\period.py:230: error: Argument 2 to "_from_sequence" becomes "Sequence[Optional[Any]]" due to an unfollowed import
pandas\core\arrays\period.py:301: error: Argument 2 to "_unbox_scalar" becomes "Union[Any, Any]" due to an unfollowed import
pandas\core\arrays\period.py:313: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\period.py:655: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\period.py:655: error: Argument 2 to "_addsub_int_array" becomes "Union[Any, Any, Any, Any]" due to an unfollowed import
pandas\core\arrays\period.py:833: error: Argument 1 to "period_array" becomes "Sequence[Optional[Any]]" due to an unfollowed import
pandas\core\arrays\datetimes.py:330: error: Type of variable becomes "Union[Any, DatetimeTZDtype]" due to an unfollowed import
pandas\core\arrays\datetimes.py:604: error: Return type becomes "Union[Any, DatetimeTZDtype]" due to an unfollowed import
pandas\core\arrays\base.py:267: error: Argument 2 to "__setitem__" becomes "Union[int, Any]" due to an unfollowed import
pandas\core\arrays\base.py:392: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:414: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:525: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:637: error: Return type becomes "Tuple[Any, Any]" due to an unfollowed import
pandas\core\arrays\base.py:661: error: Return type becomes "Tuple[Any, Any]" due to an unfollowed import
pandas\core\arrays\base.py:764: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:855: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:865: error: Return type becomes "Union[Any, Any]" due to an unfollowed import
pandas\core\arrays\base.py:936: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:956: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\base.py:956: error: Argument 2 to "_concat_same_type" becomes "Sequence[Any]" due to an unfollowed import
pandas\core\arrays\base.py:980: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\numpy_.py:89: error: Base type NDArrayOperatorsMixin becomes "Any" due to an unfollowed import
pandas\core\arrays\datetimelike.py:58: error: Type of variable becomes "Any" due to an unfollowed import
pandas\core\arrays\datetimelike.py:65: error: Return type becomes "Type[Any]" due to an unfollowed import
pandas\core\arrays\datetimelike.py:74: error: Return type becomes "Union[Any, Any, Any, Any]" due to an unfollowed import
pandas\core\arrays\datetimelike.py:96: error: Argument 2 to "_unbox_scalar" becomes "Union[Any, Any, Any, Any]" due to an unfollowed import
pandas\core\arrays\datetimelike.py:115: error: Argument 2 to "_check_compatible_with" becomes "Union[Any, Any, Any, Any]" due to an unfollowed import
pandas\core\arrays\datetimelike.py:348: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\datetimelike.py:453: error: Argument 3 to "__setitem__" becomes "Union[Any, Any, Sequence[Any]]" due to an unfollowed import
pandas\core\base.py:990: error: Return type becomes "Any" due to an unfollowed import
pandas\core\indexes\frozen.py:123: error: Base type ndarray becomes "Any" due to an unfollowed import
pandas\core\indexes\base.py:3871: error: Return type becomes "Union[ExtensionArray, Any, Any]" due to an unfollowed import
pandas\core\arrays\sparse.py:109: error: Argument 2 to "__init__" becomes "Union[str, Any, ExtensionDtype]" due to an unfollowed import
pandas\core\arrays\sparse.py:393: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\sparse.py:393: error: Argument 1 to "_get_fill" becomes "Any" due to an unfollowed import
pandas\core\arrays\sparse.py:417: error: Argument 1 to "_sparse_array_op" becomes "Any" due to an unfollowed import
pandas\core\arrays\sparse.py:417: error: Argument 2 to "_sparse_array_op" becomes "Any" due to an unfollowed import
pandas\core\arrays\sparse.py:706: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\sparse.py:706: error: Argument 2 to "_simple_new" becomes "Any" due to an unfollowed import
pandas\core\arrays\sparse.py:706: error: Argument 3 to "_simple_new" becomes "Any" due to an unfollowed import
pandas\core\arrays\interval.py:143: error: Base type IntervalMixin becomes "Any" due to an unfollowed import
pandas\core\arrays\integer.py:519: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\integer.py:576: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\categorical.py:479: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\categorical.py:502: error: Argument 2 to "astype" becomes "Union[str, Any, ExtensionDtype]" due to an unfollowed import
pandas\core\arrays\categorical.py:502: error: Return type becomes "Any" due to an unfollowed import
pandas\core\arrays\categorical.py:2652: error: Argument 1 to "_recode_for_categories" becomes "Any" due to an unfollowed import
pandas\core\internals\blocks.py:1858: error: Argument 3 to "shift" becomes "Any" due to an unfollowed import
pandas\core\indexes\multi.py:64: error: Base type BaseMultiIndexCodesEngine becomes "Any" due to an unfollowed import
pandas\core\indexes\multi.py:64: error: Base type UInt64Engine becomes "Any" due to an unfollowed import
pandas\core\indexes\multi.py:103: error: Base type BaseMultiIndexCodesEngine becomes "Any" due to an unfollowed import
pandas\core\indexes\multi.py:103: error: Base type ObjectEngine becomes "Any" due to an unfollowed import
pandas\core\indexes\category.py:525: error: Argument 2 to "get_value" becomes "Any" due to an unfollowed import
pandas\core\indexes\range.py:79: error: Type of variable becomes "Any" due to an unfollowed import
pandas\core\indexes\range.py:363: error: Argument 2 to "__contains__" becomes "Union[int, Any]" due to an unfollowed import
pandas\core\indexes\interval.py:201: error: Base type IntervalMixin becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:629: error: Argument 2 to "_can_reindex" becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:790: error: Return type becomes "Union[int, slice, Any]" due to an unfollowed import
pandas\core\indexes\interval.py:861: error: Return type becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:861: error: Argument 2 to "get_indexer" becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:931: error: Return type becomes "Tuple[Any, Any]" due to an unfollowed import
pandas\core\indexes\interval.py:931: error: Argument 2 to "get_indexer_non_unique" becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:972: error: Return type becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:972: error: Argument 2 to "get_indexer_for" becomes "Any" due to an unfollowed import
pandas\core\indexes\interval.py:989: error: Argument 2 to "get_value" becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:123: error: Base type ConversionInterface becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:150: error: Base type Formatter becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:193: error: Base type DateConverter becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:252: error: Base type DateConverter becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:325: error: Base type AutoDateFormatter becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:330: error: Base type AutoDateLocator becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:355: error: Base type DateLocator becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:936: error: Base type Locator becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:1030: error: Base type Formatter becomes "Any" due to an unfollowed import
pandas\plotting\_matplotlib\converter.py:1098: error: Base type Formatter becomes "Any" due to an unfollowed import
pandas\io\formats\format.py:1294: error: Return type becomes "Any" due to an unfollowed import
pandas\io\formats\format.py:1400: error: Argument 2 to "__init__" becomes "Union[Any, Series, DatetimeIndex, DatetimeArray]" due to an unfollowed import
pandas\io\formats\format.py:1457: error: Argument 1 to "format_percentiles" becomes "Union[Any, List[Union[int, float]], List[float], List[Union[str, float]]]" due to an unfollowed import
pandas\io\formats\format.py:1527: error: Argument 1 to "_is_dates_only" becomes "Union[Any, DatetimeArray, Index, DatetimeIndex]" due to an unfollowed import
pandas\io\formats\format.py:1548: error: Argument 1 to "_format_datetime64" becomes "Union[Any, Any]" due to an unfollowed import
pandas\io\formats\format.py:1548: error: Argument 2 to "_format_datetime64" becomes "Union[Any, tzutc, None]" due to an unfollowed import
pandas\io\formats\format.py:1565: error: Argument 1 to "_format_datetime64_dateonly" becomes "Union[Any, Any]" due to an unfollowed import
pandas\io\formats\format.py:1592: error: Argument 1 to "_get_format_datetime64_from_values" becomes "Union[Any, DatetimeArray, DatetimeIndex]" due to an unfollowed import
pandas\io\formats\format.py:1623: error: Argument 2 to "__init__" becomes "Union[Any, TimedeltaIndex]" due to an unfollowed import
pandas\io\formats\format.py:1641: error: Argument 1 to "_get_format_timedelta64" becomes "Union[Any, TimedeltaIndex, TimedeltaArray]" due to an unfollowed import
pandas\io\formats\format.py:1719: error: Argument 1 to "_trim_zeros_complex" becomes "Any" due to an unfollowed import
pandas\io\formats\format.py:1730: error: Argument 1 to "_trim_zeros_float" becomes "Union[Any, List[str]]" due to an unfollowed import
pandas\io\formats\format.py:1872: error: Argument 1 to "_binify" becomes "List[Any]" due to an unfollowed import
pandas\io\formats\format.py:1872: error: Argument 2 to "_binify" becomes "Union[Any, int]" due to an unfollowed import
pandas\core\indexing.py:102: error: Base type _NDFrameIndexerBase becomes "Any" due to an unfollowed import
pandas\core\indexing.py:650: error: Argument 3 to "_align_series" becomes "Any" due to an unfollowed import
pandas\core\indexing.py:742: error: Argument 3 to "_align_frame" becomes "Any" due to an unfollowed import
pandas\core\indexing.py:2162: error: Base type _NDFrameIndexerBase becomes "Any" due to an unfollowed import
pandas\core\indexing.py:2371: error: Return type becomes "Any" due to an unfollowed import
pandas\core\internals\managers.py:1874: error: Return type becomes "Union[Any, ExtensionDtype, None]" due to an unfollowed import
pandas\core\generic.py:164: error: Argument 5 to "__init__" becomes "Union[str, Any, ExtensionDtype, None]" due to an unfollowed import
pandas\core\frame.py:6535: error: Return type becomes "Union[Series, Any]" due to an unfollowed import
pandas\core\frame.py:6535: error: Argument 4 to "_gotitem" becomes "Union[Series, Any, None]" due to an unfollowed import
pandas\core\groupby\groupby.py:1087: error: Return type becomes "Tuple[Any, Type[Any]]" due to an unfollowed import
pandas\core\groupby\groupby.py:1087: error: Argument 1 to "objs_to_bool" becomes "Any" due to an unfollowed import
pandas\core\groupby\groupby.py:1095: error: Return type becomes "Any" due to an unfollowed import
pandas\core\groupby\groupby.py:1095: error: Argument 1 to "result_to_bool" becomes "Any" due to an unfollowed import
pandas\core\groupby\groupby.py:1879: error: Return type becomes "Tuple[Any, Optional[Type[Any]]]" due to an unfollowed import
pandas\core\groupby\groupby.py:1879: error: Argument 1 to "pre_processor" becomes "Any" due to an unfollowed import
pandas\core\groupby\groupby.py:1894: error: Return type becomes "Any" due to an unfollowed import
pandas\core\groupby\groupby.py:1894: error: Argument 1 to "post_processor" becomes "Any" due to an unfollowed import
pandas\io\packers.py:800: error: Base type _Packer becomes "Any" due to an unfollowed import
pandas\io\packers.py:820: error: Base type _Unpacker becomes "Any" due to an unfollowed import
pandas\core\window\rolling.py:209: error: Return type becomes "Optional[Any]" due to an unfollowed import
pandas\core\window\rolling.py:222: error: Return type becomes "Any" due to an unfollowed import
pandas\core\window\rolling.py:222: error: Argument 2 to "_prep_values" becomes "Optional[Any]" due to an unfollowed import
pandas\core\window\rolling.py:335: error: Return type becomes "Any" due to an unfollowed import
pandas\core\window\rolling.py:352: error: Argument 4 to "_get_roll_func" becomes "Any" due to an unfollowed import
pandas\core\window\rolling.py:788: error: Return type becomes "Any" due to an unfollowed import
pandas\core\window\rolling.py:844: error: Argument 4 to "_get_roll_func" becomes "Any" due to an unfollowed import

Ideas for keeping this manageable?

cc @WillAyd

@simonjayhawkins simonjayhawkins added the Typing type annotations, mypy/pyright type checking label Aug 25, 2019
@WillAyd
Copy link
Member

WillAyd commented Aug 25, 2019

Yea I started along this path in #26645 . The hang up on that was some of the internal imports which (maybe?) require stubs. Never went far beyond that so if you want to dive deeper I think still worth exploring

@simonjayhawkins
Copy link
Member Author

I think we should create stubs for pyx files.

It would be nice if this could be done automatically (@mrocklin wrote http://matthewrocklin.com/blog/work/2017/11/05/cython-everywhere which mentioned use of cython in regular .py files instead of creating .pyx files)

if we create the stubs manually, i suspect there would not be an issue keeping them in sync, since changes to the code for .pyx file changes would likely cause mypy to report an error if the stub wasn't updated as well.

@WillAyd
Copy link
Member

WillAyd commented Aug 25, 2019

Cool yea I agree with everything you've mentioned above. Article is certainly interesting as well - not sure what limitations there are to that approach but worth exploring if you've got time and interest I think

@rhshadrach
Copy link
Member

I believe we will never be able to set ignore_missing_imports=False unless all dependencies (optional or not) publish stubs or add a py.typed marker. If this happens - great, but otherwise, I think there is nothing we can do here. Marking as a closing candidate.

@rhshadrach rhshadrach added the Closing Candidate May be closeable, needs more eyeballs label Nov 5, 2024
@simonjayhawkins
Copy link
Member Author

In order to close this issue appropriately, I propose that we move forward by setting ignore_missing_imports=False in our configuration. This change means that for every import from a library that lacks published stubs or a py.typed marker, we’ll need to explicitly add a # type: ignore comment. Although this might increase the number of ignore comments in our codebase, it aligns with our long-term goal of comprehensive type safety.

When this issue was first raised, the choice of ignore_missing_imports=True was driven by the concern that enforcing explicit annotations from untyped libraries would force us to sprinkle many # type: ignore statements—a practice that was regarded as undesirable by some maintainers at the time. However, as our ecosystem evolves and more dependencies adopt proper type support, targeted ignore comments can serve as a pragmatic, transitional measure. Any untyped imports become clear candidates for refinement once the libraries catch up with type annotations.

In summary, while a fully strict type checking environment (i.e., ignore_missing_imports=False without any ignores) would require every dependency to ship with stubs or a py.typed marker, the practical interim solution is to adopt ignore_missing_imports=False and manage exceptions on a case-by-case basis.

This approach makes our intention explicit and establishes a clear path forward while still accounting for external constraints. This issue was to discuss and manage that transition and probably should not yet be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closing Candidate May be closeable, needs more eyeballs Typing type annotations, mypy/pyright type checking
Projects
None yet
Development

No branches or pull requests

4 participants