You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have confirmed this bug exists on the latest version of pandera.
(optional) I have confirmed this bug exists on the main branch of pandera.
Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Code Sample, a copy-pastable example
importpicklefrompathlibimportPathimportpandasaspdimportpanderaaspafrompandera.typingimportDataFrame, Seriespa.__version__# Define a schema using panderaclassSchema(pa.DataFrameModel):
column1: Series[int]
column2: Series[float]
column3: Series[str]
# Use the check_types decorator to validate input and output@pa.check_typesdefprocess_data_and_check_types(df: DataFrame[Schema]) ->DataFrame[Schema]:
# Example processing: add a new columndf["column4"] =df["column1"] +df["column2"]
returndf# Create a sample DataFramedata= {"column1": [1, 2, 3], "column2": [0.1, 0.2, 0.3], "column3": ["a", "b", "c"]}
df=pd.DataFrame(data)
# Process the DataFrameprocessed_df=process_data_and_check_types(df)
# Same function without checking typesdefprocess_data(df: DataFrame[Schema]) ->DataFrame[Schema]:
# Example processing: add a new columndf["column4"] =df["column1"] +df["column2"]
returndf# Can only serialize function without decoratorpath=Path("/tmp/tmp.pkl")
withpath.open("wb") asf:
pickle.dump(process_data, f)
# This will raise an errorwithpath.open("wb") asf:
pickle.dump(process_data_and_check_types, f)
Expected behavior
Ability to pickle a function decorated with @pa.check_types.
Desktop (please complete the following information):
OS: macOS 15.1
Version: pandera 0.20.4
Screenshots
Error from above code:
{
"name": "NotImplementedError",
"message": "object proxy must define __reduce_ex__()",
"stack": "---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
Cell In[6], line 2
1 with path.open(\"wb\") as f:
----> 2 pickle.dump(process_data_and_check_types, f)
NotImplementedError: object proxy must define __reduce_ex__()"
}
Additional context
This functionality is required to use a variety of ML frameworks (such as MLFlow and Metaflow) that pickle artifacts under the hood in order to log them.
The text was updated successfully, but these errors were encountered:
Describe the bug
Functions cannot be pickled if they are decorated with
@pa.check_types
.Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Code Sample, a copy-pastable example
Expected behavior
Ability to pickle a function decorated with
@pa.check_types
.Desktop (please complete the following information):
Screenshots
Error from above code:
Additional context
This functionality is required to use a variety of ML frameworks (such as MLFlow and Metaflow) that pickle artifacts under the hood in order to log them.
The text was updated successfully, but these errors were encountered: