Looking for best approach to validate uniqueness across multiple columns

[TerrySnow1963]

FYI, I raised a question in Stake Overflow (thought it might help generate more awareness of pandera, as well as maybe help me use pandera more effectively).

The stack overflow link is here

The jist of the design issue that I was looking at is I want to add a uniqueness validation check to a combination of columns.

For context, the columns are X, Y Coordinates, and for the data set there should only be one row for each combination of X and Y coordinates. Any pointers on how to approve the approach that I have used is appreciated.

[cosmicBboy]

Thanks @TerrySnow1963, just posted an answer: https://stackoverflow.com/a/65721438/3205067

[TerrySnow1963]

great answer, maybe the answer could go into documentation , say new Examples section.

[jeffzi]

Documentation can be part of #383

A built-in pa.Check.unique_columns() would be useful for many people.

[cosmicBboy]

I think actually uniqueness should be a keyword option: #390, renaming the allow_duplicates option currently implemented in Column components and extending it to all schemas and schema components.

[cosmicBboy]

Also, maybe we can add a pandera tag to the stackoverflow post to add some extra metadata for exposure :)

[TerrySnow1963]

Would love to do that for you, however new tags in stack exchange can only be added by users with more than 1500 reputation points.

[cosmicBboy]

hah! shows how much I know about SO 😅

[michaelbestokra]

I stumbled upon this answer, and thought it might be good to mention that now a DataFrameSchema has a unique parameter: https://pandera.readthedocs.io/en/stable/dataframe_schemas.html#validating-the-joint-uniqueness-of-columns.
This means you can do the following:

schema = pa.DataFrameSchema(
    columns={col: pa.Column(int) for col in ["a", "b", "c"]},
    unique=["a", "c"],
)
df = pd.DataFrame.from_records([
    {"a": 1, "b": 2, "c": 3},
    {"a": 1, "b": 2, "c": 3},
])
schema.validate(df)