Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create custom constraint with primary key #1514

Closed
amontanez24 opened this issue Jul 27, 2023 · 0 comments · Fixed by #1523
Closed

Cannot create custom constraint with primary key #1514

amontanez24 opened this issue Jul 27, 2023 · 0 comments · Fixed by #1523
Assignees
Labels
bug Something isn't working
Milestone

Comments

@amontanez24
Copy link
Contributor

Environment Details

Please indicate the following details about the environment in which you found the bug:

  • SDV version: 1.2.1
  • Python version: Any
  • Operating System: Any

Error Description

If I create a custom constraint that involves a primary key, I get a KeyError during sampling

Steps to reproduce

from sdv.single_table import GaussianCopulaSynthesizer
from sdv.metadata import SingleTableMetadata
from sdv.constraints import create_custom_constraint_class
import pandas as pd

# Add constraints
def is_valid(column_names, data):
    return data['key'] == data['letter'] + '_' + data['number']

def transform(column_names, data):
    new_data = data.drop(['letter', 'number'], axis=1)
    return new_data

def reverse_transform(column_names, data):
    columns = data['key'].str.split('_', expand=True)
    data['letter'] = columns[0]
    data['number'] = columns[1]
    return data

custom_constraint = create_custom_constraint_class(
    is_valid_fn=is_valid,
    transform_fn=transform,
    reverse_transform_fn=reverse_transform
)

data = pd.DataFrame({
    'key': ['a_1', 'b_2', 'c_3'],
    'letter': ['a', 'b', 'c'],
    'number': ['1', '2', '3'],
    'other': [7, 8, 9]
})
metadata = SingleTableMetadata()
metadata.detect_from_dataframe(data)
metadata.update_column('key', sdtype='id', regex_format='\w_\d')
metadata.set_primary_key('key')
synth = GaussianCopulaSynthesizer(metadata)
synth.add_custom_constraint_class(custom_constraint, 'custom')


id_must_match = {
    'constraint_class': 'custom',
    'constraint_parameters': {
        'column_names': ['letter', 'number'],
    }
}

synth.add_constraints([id_must_match])
synth.fit(data)
sampled = synth.sample(100)

Additional context

  • This is because we are not adding the key columns to the data that gets reverse transformed by constraints in the data processor. In the following lines, the keys need to be added to reversed_data
    if self._keys and num_rows:
    generated_keys = self.generate_keys(num_rows, reset_keys)
    sampled_columns.extend(self._keys)
    for constraint in reversed(self._constraints_to_reverse):
    reversed_data = constraint.reverse_transform(reversed_data)
@amontanez24 amontanez24 added bug Something isn't working new Automatic label applied to new issues and removed new Automatic label applied to new issues labels Jul 27, 2023
@amontanez24 amontanez24 added this to the 1.2.2 milestone Jul 27, 2023
@amontanez24 amontanez24 self-assigned this Aug 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant