Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kgao/federal learning #818

Merged
merged 39 commits into from
Nov 2, 2023
Merged

Kgao/federal learning #818

merged 39 commits into from
Nov 2, 2023

Conversation

kgao
Copy link
Collaborator

@kgao kgao commented Oct 4, 2023

Draft PR, please review the documents first.

@kgao kgao requested a review from kbattocchi October 4, 2023 19:20
kbattocchi
kbattocchi requested changes Oct 6, 2023
View reviewed changes
Copy link
Collaborator

@kbattocchi kbattocchi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good, I've suggested a few minor changes to the code. You also need to fix the linting and documentation building CI failures. I didn't review the .rst file yet, since I'll wait until the docs are generated and look at the resulting HTML

econml/sklearn_extensions/federated_learning.py Outdated Show resolved Hide resolved
econml/sklearn_extensions/federated_learning.py Outdated Show resolved Hide resolved
econml/sklearn_extensions/local_test_combined.py Outdated Show resolved Hide resolved
econml/sklearn_extensions/local_test_splitted.py Outdated Show resolved Hide resolved
econml/tests/test_federated_learning.py Show resolved Hide resolved
kbattocchi and others added 12 commits October 6, 2023 18:04
@kbattocchi @kgao
Fix lint issues uncovered by pycodestyle 2.11
4b1e3db 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
init
96c8507 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
update class name
5b08000 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
update method
7c23d5e
@kgao
update matrix A, B calling reference
f0940aa 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
cleanup wrapper class
edd9e94 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
add local test files
cfb3e34 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kbattocchi @kgao
Added federated learner test
0dfa1e3 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kbattocchi @kgao
Fix aggregation logic
d97793d 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kgao
update the doc
fab3531 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
update doc for federated learning
e652b08 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
cleanup branch: test, file structure, docstring and linting
6c7e485 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
@kgao
cleanup federated learning doc
909eb5c 
Signed-off-by: kgao <kevin.leo.gao@gmail.com>
kbattocchi
kbattocchi requested changes Oct 18, 2023
View reviewed changes
Copy link
Collaborator

@kbattocchi kbattocchi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please clean up the documentation, removing all of the generic content and mainly focusing on the actual classes that perform the work in our library. Explaining that this can be used with Spark is helpful, but should not go into a full intro to Spark (and one benefit of how we've implemented this functionality is that it could also be used with any other distributed computing platform).

Also, please merge the latest changes from main into your branch.

doc/spec/federated_learning.rst Outdated Show resolved Hide resolved
kgao and others added 13 commits October 20, 2023 16:34
@kgao
cleaning the doc for federated learning
e187f13
@kbattocchi
Rename moments
5eb64a5 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Make FederatedEstimator a CateEstimator
e00cdd2 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@fverac @kbattocchi
add arg to allow missing values in W and sometimes X (#791)
f943966 
* enable nans in W

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* linting

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* add tests for ests that handle missing in W

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* allow missing in X for some ortholearner subclasses

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* refactor keyword arg to be bool only, add more tests

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* linting

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* enable missing for metalearners and orf, fix dowhywrapped discretetreat dmlorf

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* update arg name to allow_missing, add docstrings

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* add warning when missing values detected

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* dummy commit

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

* dummy commit revert

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>

---------

Signed-off-by: Fabio Vera <fabiovera@microsoft.com>
@kbattocchi
Drop support for sklearn<1.0
1c0bcf8 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Support direct covariance fitting in DRIV
99cc11a 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Enable sklearn 1.3
d90972d 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Ensure groups work with DRIV, DMLIV
c46eac8 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Update __init__.py to reflect current structure
53a4b2a 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Make changes to support dowhy 0.10.1 in tests
9dbf21d 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Allow newer shap, matlab, and seaborn versions
bef9b4c 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Make minor CI improvements
db606ee 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Save notebook outputs during CI
de0c645 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
kbattocchi and others added 4 commits October 31, 2023 12:56
@kbattocchi
Remove legacy assertWarns hack
a6691ff 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@v-shaal @kbattocchi
Scaling ortholearners using Ray (#800)
6291f4a 
Added Implementation of ray-based distributed parallelization to crossfit.

---------

Signed-off-by: Vishal Verma <vishalmverma27@gmail.com>
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
Co-authored-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Improve FederatedEstimator docs
7935df5 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Merge branch 'main' into kgao/federal-learning
ef04ce7 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi kbattocchi marked this pull request as ready for review October 31, 2023 17:28
kbattocchi
kbattocchi approved these changes Oct 31, 2023
View reviewed changes
Copy link
Collaborator

@kbattocchi kbattocchi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now looks good, will merge once all checks pass.

@kbattocchi kbattocchi enabled auto-merge (squash) October 31, 2023 17:33
@kbattocchi
Fixup docs
650f28a 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Fixup docs
85c8458 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Fixup federated learning doctest
a91d8d2 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Reduce featurization degree
1e1f67d 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Add tests for DRLearner
d9444ba 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Fixup tests
16b8860 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Fix test failures
c04df65 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Fix test
e4415c8 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
Fix tests
5205791 
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi kbattocchi merged commit f170bab into main Nov 2, 2023
71 of 72 checks passed
@kbattocchi kbattocchi deleted the kgao/federal-learning branch November 2, 2023 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kbattocchi kbattocchi kbattocchi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@kgao @kbattocchi @fverac @v-shaal