Cohort Method report updates - Generalizability tab

[azimov]

From @schuemie

CohortMethod also has a pending update to the results model. Here are the changes to the model:
Deprecated the attrition_fraction and attrition_diagnostic fields from the cm_diagnostics_summary table.
Added the target_estimator field to the cm_result and cm_interaction_result tables.
Added the generalizability_max_sdm and generalizabiltiy_diagnostic fields to the cm_diagnostics_summary table.
Added the mean_before, mean_after, target_std_diff, comparator_std_diff, and target_comparator_std_diff fields to both the cm_covariate_balance and cm_shared_covariate_balance tables.

Reasons for the change:
In the past, we used attrition fraction (the percent of people lost due to PS matching etc.) as a way to estimate generalizability of the findings to the original cohort(s). However, that is not a good metric (if your comparator is small you could lose a lot of people, but still be fine on generalizability). Instead, we’ll now use standardized difference of mean (SDM) as a metric (I also presented this at last year’s OHDSI symposium). Unlike our balance metric (which also uses SDM), which is computing between target and comparator, the generalizability metric is computed between the original cohort (as extracted from the database) and the ‘analytical cohort’, the cohort that went into the Cox regression.

We do have to decide which cohort to compare before and after adjustment. The target cohort only? The union of T and C? This depends on which estimand we’re going for. If we do PS matching, we’re targeting the ATT (average treatment effect in the treated), and we should compute generalizability for the target only. Stratification estimates ATE (average treatment effect), so requires T + C. IPTW can actually target either ATT or ATE. Which estimand we’re targeting is now captured in the ‘target_estimator` field in both the cm_result and cm_interaction_result tables. We should probably show this new field somewhere.

The cm_covariate_balance and cm_shared_covariate_balance tables now have additional fields that can be used to inform the user on generalizability. Depending on the cm_result.target_estimator field, you’ll need to use these:

cm_result.target_estimator = “att”: target_mean_before, target_mean_after, and target_std_diff

cm_result.target_estimator = “atu”: comparator_mean_before, comparator_mean_after, and comparator_std_diff
(‘atu’ stands for ‘treatment effect in the untreated’)

else: mean_before, mean_after, and target_comparator_std_diff

The way to show these in the Shiny app is probably to add a ‘Generalizability’ tab with a table showing these 3 columns for all covariates from the cm_shared_covariate_balance table, sorted so the highest std_diff is first. (if that is too much data we can also just take the top 20).