Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook #1297

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

nparent1
Copy link
Contributor

@nparent1 nparent1 commented Jan 19, 2025

Adds support for the generalized adjustment estimands at the estimation stage. The PR primarily does two things:

  • Abstracts the get_backdoor() method of the IdentifiedEstimand class to instead be a get_adjustment_set() method. Then if the identifier_method is specified as 'general_adjustment', the get_adjustment_set() method will return the general adjustment set (rather than explicitly a backdoor set). Then throughout the repo where get_backdoor_set() was called for estimation purposes, we instead call get_adjustment_set(). Note that because we have get_adjustment_set() default to the backdoor set, the overall behavior is the same as before, unless a user explicitly sets general_adjustment as the identifier_method.
  • Adds various tests and test utils. In particular, adds support for specifying explicit graphs in the estimation tests, for more specific tests of the general_adjustment criterion.

An example notebook is also added, dowhy_generalized_covariate_adjustment_estimation_example.ipynb

This is a follow-on PR to #1292

Issue Ref: #402

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 changed the title [FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook [WIP][FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook Jan 19, 2025
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1
Copy link
Contributor Author

This will need to be rebased once #1292 merges

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
estimator_ate.fit(data["df"])
true_ate = data["ate"]
ate_estimate = estimator_ate.estimate_effect(data["df"])
error = ate_estimate.value - true_ate
error = abs(ate_estimate.value - true_ate)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor bug fix here - taking the absolute value, both here and a few lines down.

Previously the test util would return true if (error < true_ate * self._error_tolerance) which only makes sense if the error and true_ate are in absolute value (otherwise a negative error would always be accepted no matter how far from zero)

@@ -377,3 +394,22 @@ def randomize_column_values(df, columns):
assert error_3 > error_tolerance
except NotImplementedError:
pass # Expected, for many Estimators


class TestGraphObject(object):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is modeled after the graph objects used in the identification test suites (see base.by in the identification test folder).

I added this so that more specific graphs could be tested as needed (which is particularly relevant for testing some of the more involved graphs which really differentiate the general adjustment criterion from the backdoor criterion)

key = self.identifier_method
self.general_adjustment_variables[key] = variables_arr

def get_adjustment_set(self, key: Optional[str] = None):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the core change. IdentifiedEstimand now has a generic get_adjustment_set method which routes to one of get_backdoor_variables or get_general_adjustment_variables.

The wrapper method get_adjustment_set should thus be called instead of get_adjustment_set or get_general_adjustment_variables throughout the codebase.

@nparent1 nparent1 changed the title [WIP][FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook [FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook Jan 27, 2025
@nparent1 nparent1 marked this pull request as ready for review January 27, 2025 04:14
@nparent1 nparent1 marked this pull request as draft January 27, 2025 04:18
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 force-pushed the np/gen_adjustment_estimation branch from 5e10aff to 710b045 Compare January 28, 2025 01:30
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 marked this pull request as ready for review January 29, 2025 03:59
@nparent1
Copy link
Contributor Author

@amit-sharma - this is a follow on to #1292 to add support for general adjustment for estimation. It is ready for review

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant