Add evaluation methods to synthesizer

[amontanez24]

Problem Description

As a user, it would be useful to evaluate the synthetic data generated against the original data.

Acceptance criteria

• Add a evaluation module and two submodules within it: single_table and multi_table

• Add the following methods to the evaluation.single_table module

  • evaluate_quality(real_data, synthetic_data, metadata, verbose) - Wrapper around the initialization and evaluation of this class.
  • run_diagnostic(real_data, synthetic_data, metadata, verbose) - Wrapper around the initialization and evaluation of this class.
  • get_column_plot(real_data, synthetic_data, metadata, column_name) - Wrapper around sdmetrics.reports.utils.get_column_plot
  • get_column_pair_plot(real_data, synthetic_data, metadata, column_names) - Wrapper around sdmetrics.reports.utils.get_column_pair_plot

• Add the following methods to the evaluation.multi_table module

  • run_diagnostic(real_data ,synthetic_data, metadata, verbose) - Wrapper around the initialization and evaluation of this class.
  • evaluate_quality(real_data, synthetic_data, metadata, verbose) - Wrapper around the initialization and evaluation of this class.
  • synthesizer.get_column_plot(real_data, synthetic_data, metadata, table_name, column_name) - Wraps the same method as the single table case but requires the table name.
  • synthesizer.get_column_pair_plot(real_data, synthetic_data, metadata, table_name, column_names) - Wraps the same method as the single table case but requires the table name.

Expected behavior

# Single table cases

quality_report = evaluate_quality(
  real_data=real_data, # DataFrame
  synthetic_data=synthetic_data, # DataFrame
  metadata=my_metadata, # SingleTableMetadata
  verbose=True
)
diagnostic_report = run_diagnostic(
  real_data=real_data, # DataFrame
  synthetic_data=synthetic_data, # DataFrame
  metadata=my_metadata, # SingleTableMetadata
  verbose=True
)
fig = get_column_plot(
  real_data=real_data, # DataFrame
  synthetic_data=synthetic_data, # DataFrame
  metadata=my_metadata, # SingleTableMetadata
  column_name='age'
)
fig = get_column_plot(
  real_data=real_data, # DataFrame
  synthetic_data=synthetic_data, # DataFrame
  metadata=my_metadata, # SingleTableMetadata
  column_names=['age', 'weight']
)

# Multi-table cases
quality_report = evaluate_quality(
  real_data=real_data, # dictionary
  synthetic_data=synthetic_data, # dictionary
  metadata=my_metadata, # MultiTableMetadata
  verbose=True
)
diagnostic_report = run_diagnostic(
  real_data=real_data, # dictionary
  synthetic_data=synthetic_data, # dictionary
  metadata=my_metadata, # MultiTableMetadata
  verbose=True
)
# Plot the 1D marginal distribution
fig = get_column_plot(
  real_data=real_data, # dictionary
  synthetic_data=synthetic_data, # dictionary
  metadata=my_metadata, # MultiTableMetadta
  table_name='users',
  column_name='age'
)
# Plot the 2D bivariate distribution
fig = get_column_plot(
  real_data=real_data, # dictionary
  synthetic_data=synthetic_data, # dictionary
  metadata=my_metadata, # MultitableMetadata
  table_name='users',
  column_names=['age', 'weight']
)

[npatki]

@amontanez24 @fealho I'm testing this out. It seems like evaluate_quality is actually returning the score.

The spec is to return the actual QualityReport object from SDMetrics. Shall we re-open this?

[fealho]

@npatki Oh, I thought it was supposed to be the score. Yes, this should be reopened and patched.