Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The argument 'date_format' of the pandas read_csv function is not present. #636

Closed
SjorsLockhorst opened this issue Apr 6, 2023 · 2 comments · Fixed by #650
Closed

The argument 'date_format' of the pandas read_csv function is not present. #636

SjorsLockhorst opened this issue Apr 6, 2023 · 2 comments · Fixed by #650
Labels

Comments

@SjorsLockhorst
Copy link
Contributor

SjorsLockhorst commented Apr 6, 2023

Describe the bug
Pandas 2.0.0 added data_format parameter to the read_csv function.
The latest version of pandas-stubs doesn't seem to have a type hint for this.

To Reproduce

  1. pd.read_csv("foo.csv", parse_dates=True, date_format="%Y-%m-%d")
  2. mypy
  3. storage_interface/util.py:12: error: Module "pandas.io.parsers.readers" does not explicitly export attribute "ReadCsvBuffer" [attr-defined]
    error: No overload variant of "read_csv" matches argument types "str", "bool", "str" [call-overload]
    note: Possible overload variants:
    def read_csv(filepath_or_buffer: Union[Union[str, PathLike[str]], ReadCsvBuffer[bytes], ReadCsvBuffer[str]], *, sep: Optional[str] = ..., delimiter: Optional[str] = ..., header: Union[int, Sequence[int], Literal['infer'], None] = ..., names: Optional[List[str]] = ..., index_col: Union[int, str, Sequence[Union[str, int]], Literal[False], None] = ..., usecols: Union[List[str], Tuple[str, ...], Sequence[int], Series[Any], Index, ndarray[Any, dtype[Any]], Callable[[str], bool], None] = ..., dtype: Union[Union[Union[ExtensionDtype, Union[str, dtype[generic], Type[str], Type[complex], Type[bool], Type[object]]], Dict[Any, Union[ExtensionDtype, Union[str, dtype[generic], Type[str], Type[complex], Type[bool], Type[object]]]]], defaultdict[Any, Any], None] = ..., engine: Optional[Literal['c', 'python', 'pyarrow', 'python-fwf']] = ..., converters: Union[Mapping[Union[int, str], Callable[[str], Any]], Mapping[int, Callable[[str], Any]], Mapping[str, Callable[[str], Any]], None] = ..., true_values: List[str] = ..., false_values: List[str] = ..., skipinitialspace: bool = ..., skiprows: Union[int, Sequence[int], Callable[[int], bool]] = ..., skipfooter: int = ..., nrows: Optional[int] = ..., na_values: Union[Sequence[str], Mapping[str, Sequence[str]]] = ..., keep_default_na: bool = ..., na_filter: bool = ..., verbose: bool = ..., skip_blank_lines: bool = ..., parse_dates: Union[bool, List[int], List[str], Sequence[Sequence[int]], Mapping[str, Sequence[Union[int, str]]]] = ..., infer_datetime_format: bool = ..., keep_date_col: bool = ..., date_parser: Callable[..., Any] = ..., dayfirst: bool = ..., cache_dates: bool = ..., iterator: Literal[True], chunksize: Optional[int] = ..., compression: Union[None, Dict[str, Any], Literal['infer', 'gzip', 'bz2', 'zip', 'xz', 'zstd']] = ..., thousands: Optional[str] = ..., decimal: str = ..., lineterminator: Optional[str] = ..., quotechar: str = ..., quoting: Literal[0, 1, 2, 3] = ..., doublequote: bool = ..., escapechar: Optional[str] = ..., comment: Optional[str] = ..., encoding: Optional[str] = ..., encoding_errors: Optional[str] = ..., dialect: Union[str, Dialect] = ..., on_bad_lines: Union[Callable[[List[str]], Optional[List[str]]], Literal['error', 'warn', 'skip']] = ..., delim_whitespace: bool = ..., low_memory: bool = ..., memory_map: bool = ..., float_precision: Optional[Literal['high', 'legacy', 'round_trip']] = ..., storage_options: Optional[Optional[Dict[str, Any]]] = ...) -> TextFileReader
    storage_interface/util.py:172: note: def read_csv(filepath_or_buffer: Union[Union[str, PathLike[str]], ReadCsvBuffer[bytes], ReadCsvBuffer[str]], *, sep: Optional[str] = ..., delimiter: Optional[str] = ..., header: Union[int, Sequence[int], Literal['infer'], None] = ..., names: Optional[List[str]] = ..., index_col: Union[int, str, Sequence[Union[str, int]], Literal[False], None] = ..., usecols: Union[List[str], Tuple[str, ...], Sequence[int], Series[Any], Index, ndarray[Any, dtype[Any]], Callable[[str], bool], None] = ..., dtype: Union[Union[Union[ExtensionDtype, Union[str, dtype[generic], Type[str], Type[complex], Type[bool], Type[object]]], Dict[Any, Union[ExtensionDtype, Union[str, dtype[generic], Type[str], Type[complex], Type[bool], Type[object]]]]], defaultdict[Any, Any], None] = ..., engine: Optional[Literal['c', 'python', 'pyarrow', 'python-fwf']] = ..., converters: Union[Mapping[Union[int, str], Callable[[str], Any]], Mapping[int, Callable[[str], Any]], Mapping[str, Callable[[str], Any]], None] = ..., true_values: List[str] = ..., false_values: List[str] = ..., skipinitialspace: bool = ..., skiprows: Union[int, Sequence[int], Callable[[int], bool]] = ..., skipfooter: int = ..., nrows: Optional[int] = ..., na_values: Union[Sequence[str], Mapping[str, Sequence[str]]] = ..., keep_default_na: bool = ..., na_filter: bool = ..., verbose: bool = ..., skip_blank_lines: bool = ..., parse_dates: Union[bool, List[int], List[str], Sequence[Sequence[int]], Mapping[str, Sequence[Union[int, str]]]] = ..., infer_datetime_format: bool = ..., keep_date_col: bool = ..., date_parser: Callable[..., Any] = ..., dayfirst: bool = ..., cache_dates: bool = ..., iterator: bool = ..., chunksize: int, compression: Union[None, Dict[str, Any], Literal['infer', 'gzip', 'bz2', 'zip', 'xz', 'zstd']] = ..., thousands: Optional[str] = ..., decimal: str = ..., lineterminator: Optional[str] = ..., quotechar: str = ..., quoting: Literal[0, 1, 2, 3] = ..., doublequote: bool = ..., escapechar: Optional[str] = ..., comment: Optional[str] = ..., encoding: Optional[str] = ..., encoding_errors: Optional[str] = ..., dialect: Union[str, Dialect] = ..., on_bad_lines: Union[Callable[[List[str]], Optional[List[str]]], Literal['error', 'warn', 'skip']] = ..., delim_whitespace: bool = ..., low_memory: bool = ..., memory_map: bool = ..., float_precision: Optional[Literal['high', 'legacy', 'round_trip']] = ..., storage_options: Optional[Optional[Dict[str, Any]]] = ...) -> TextFileReader
    storage_interface/util.py:172: note: def read_csv(filepath_or_buffer: Union[Union[str, PathLike[str]], ReadCsvBuffer[bytes], ReadCsvBuffer[str]], *, sep: Optional[str] = ..., delimiter: Optional[str] = ..., header: Union[int, Sequence[int], Literal['infer'], None] = ..., names: Optional[List[str]] = ..., index_col: Union[int, str, Sequence[Union[str, int]], Literal[False], None] = ..., usecols: Union[List[str], Tuple[str, ...], Sequence[int], Series[Any], Index, ndarray[Any, dtype[Any]], Callable[[str], bool], None] = ..., dtype: Union[Union[Union[ExtensionDtype, Union[str, dtype[generic], Type[str], Type[complex], Type[bool], Type[object]]], Dict[Any, Union[ExtensionDtype, Union[str, dtype[generic], Type[str], Type[complex], Type[bool], Type[object]]]]], defaultdict[Any, Any], None] = ..., engine: Optional[Literal['c', 'python', 'pyarrow', 'python-fwf']] = ..., converters: Union[Mapping[Union[int, str], Callable[[str], Any]], Mapping[int, Callable[[str], Any]], Mapping[str, Callable[[str], Any]], None] = ..., true_values: List[str] = ..., false_values: List[str] = ..., skipinitialspace: bool = ..., skiprows: Union[int, Sequence[int], Callable[[int], bool]] = ..., skipfooter: int = ..., nrows: Optional[int] = ..., na_values: Union[Sequence[str], Mapping[str, Sequence[str]]] = ..., keep_default_na: bool = ..., na_filter: bool = ..., verbose: bool = ..., skip_blank_lines: bool = ..., parse_dates: Union[bool, List[int], List[str], Sequence[Sequence[int]], Mapping[str, Sequence[Union[int, str]]]] = ..., infer_datetime_format: bool = ..., keep_date_col: bool = ..., date_parser: Callable[..., Any] = ..., dayfirst: bool = ..., cache_dates: bool = ..., iterator: Literal[False] = ..., chunksize: None = ..., compression: Union[None, Dict[str, Any], Literal['infer', 'gzip', 'bz2', 'zip', 'xz', 'zstd']] = ..., thousands: Optional[str] = ..., decimal: str = ..., lineterminator: Optional[str] = ..., quotechar: str = ..., quoting: Literal[0, 1, 2, 3] = ..., doublequote: bool = ..., escapechar: Optional[str] = ..., comment: Optional[str] = ..., encoding: Optional[str] = ..., encoding_errors: Optional[str] = ..., dialect: Union[str, Dialect] = ..., on_bad_lines: Union[Callable[[List[str]], Optional[List[str]]], Literal['error', 'warn', 'skip']] = ..., delim_whitespace: bool = ..., low_memory: bool = ..., memory_map: bool = ..., float_precision: Optional[Literal['high', 'legacy', 'round_trip']] = ..., storage_options: Optional[Optional[Dict[str, Any]]] = ...) -> DataFrame

Please complete the following information:

  • OS: Debian GNU/Linux
  • OS Version: 10 (buster) on Windows 10 x86_64
  • python 3.10.8
  • mypy 1.1.1
  • pandas-stubs 1.5.3.230321

Additional context

@Dr-Irv
Copy link
Collaborator

Dr-Irv commented Apr 6, 2023

We are tracking changes for 2.0 here: #624

Once we have made those changes, we'll do a 2.0.0.yymmdd release.

I will keep this open in case anybody wants to submit a PR.

@Dr-Irv Dr-Irv added good first issue IO CSV read_csv, to_csv labels Apr 6, 2023
@Dr-Irv
Copy link
Collaborator

Dr-Irv commented Apr 12, 2023

Once we have made those changes, we'll do a 2.0.0.yymmdd release.

Note: We just released 2.0.0.230412 . Doesn't support all the new features of 2.0, including the one in this issue. PR's welcome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants