Awesome Federated Machine Learning

Federated Learning (FL) is a new machine learning framework, which enables multiple devices collaboratively to train a shared model without compromising data privacy and security.

This repository aims to keep tracking the latest research advancements of federated learning, including but not limited to research papers, books, codes, tutorials, and videos.

Table of Contents

• Top Machine Learning Conferences
  • ICML ICLR NeurIPS
• Top Computer Vision Conferences
  • CVPR ICCV ECCV
• Top Artificial Intelligence and Data Mining Conferences
  • AAAI AISTATS KDD
• Books
• Papers (Research directions)
  • Model Aggregation
  • Personalization  
  • Recommender system
  • Security  
  • Survey  
  • Efficiency  
  • Optimization  
  • Fairness
    
  • Application  
  • Boosting  
  • Incentive mechanism
  • Unsupervised Learning  
  • Heterogeneity  
  • Client Selection
  • Graph Neural Networks
  • Other Machine Learning Paradigm
  • Trade-off
• Google FL Workshops
• Videos and Lectures
• Tutorials and Blogs
• Open-Sources
  • Enterprise Grade
  • Research Purpose

 

Top Machine Learning Conferences

In this section, we will summarize Federated Learning papers accepted by top machine learning conference, Including NeurIPS, ICML, ICLR.

ICML

Years	Title	Affiliations	Materials
ICML 2022	Deep Neural Network Fusion via Graph Matching with Applications to Model Ensemble and Federated Learning	Shanghai Jiao Tong University	code
	Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization	KAIST
	FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learning	University of Oulu	code
	FedNL: Making Newton-Type Methods Applicable to Federated Learning	KAUST	video
	Federated Minimax Optimization: Improved Convergence Analyses and Algorithms	Carnegie Mellon University
	FedNest: Federated Bilevel, Minimax, and Compositional Optimization	University of Michigan	code
	Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification	University of Maryland	code
	DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training	University of Science and Technology of China	code
	Federated Learning with Positive and Unlabeled Data	Xi’an Jiaotong University
	Neurotoxin: Durable Backdoors in Federated Learning	Southeast University; Princeton University	code
	DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning	University of Cambridge
	Neural Tangent Kernel Empowered Federated Learning	NC State University	code
	EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning	VMware Research	code
	Architecture Agnostic Federated Learning for Neural Networks	The University of Texas at Austin
	Fast Composite Optimization and Statistical Recovery in Federated Learning	Shanghai Jiao Tong University
	Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning	New York University
	Communication-Efficient Adaptive Federated Learning	Pennsylvania State University
	Personalized Federated Learning via Variational Bayesian Inference	Chinese Academy of Sciences
	QSFL: A Two-Level Uplink Communication Optimization Framework for Federated Learning	Nankai University	code
	Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy	University of Minnesota
	The Poisson Binomial Mechanism for Unbiased Federated Learning with Secure Aggregation	Stanford University; Google Research
	The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning	Stanford University; Google Research	code
	Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring	University of Science and Technology of China
	Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling	Geogia Institute of Technology
	Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering	University of Michigan	code
	Resilient and Communication Efficient Learning for Heterogeneous Federated Systems	Michigan State University
	Accelerated Federated Learning with Decoupled Adaptive Optimization	Auburn University
	Proximal and Federated Random Reshuffling	KAUST	code
	Personalized Federated Learning through Local Memorization	Inria	code
	Federated Learning with Partial Model Personalization	University of Washington	code
	ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training	CISPA Helmholz Center for Information Security	code
	Federated Learning with Label Distribution Skew via Logits Calibration	Zhejiang University
	Anarchic Federated Learning	The Ohio State University
	Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning	Hong Kong Baptist University	code
	Generalized Federated Learning via Sharpness Aware Minimization	University of South Florida
	FedScale: Benchmarking Model and System Performance of Federated Learning at Scale	University of Michigan	code
	Multi-Level Branched Regularization for Federated Learning	Seoul National University	HomePage
ICML 2021	Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix	Harvard University	video code
	FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Analysis	Peking University; Princeton University	video
	Personalized Federated Learning using Hypernetworks	Bar-Ilan University; NVIDIA	code HomePage video
	Federated Composite Optimization	Stanford University; Google	code video slides
	Exploiting Shared Representations for Personalized Federated Learning	University of Texas at Austin; University of Pennsylvania	code video
	Data-Free Knowledge Distillation for Heterogeneous Federated Learning	Michigan State University	code video
	Federated Continual Learning with Weighted Inter-client Transfer	KAIST	code video
	Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	The University of Iowa	video
	Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning	The University of Tokyo	video
	Federated Learning of User Verification Models Without Sharing Embeddings	Qualcomm	video
	Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning	Accenture	code video
	Ditto: Fair and Robust Federated Learning Through Personalization	CMU; Facebook AI	code video
	Heterogeneity for the Win: One-Shot Federated Clustering	CMU	video
	The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation	Google	video
	Debiasing Model Updates for Improving Personalized Federated Training	Boston University; Arm	video
	One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning	Toyota; Berkeley; Cornell University	code video
	CRFL: Certifiably Robust Federated Learning against Backdoor Attacks	UIUC; IBM	code video
	Federated Learning under Arbitrary Communication Patterns	Indiana University; Amazon	video
ICML 2020	FedBoost: A Communication-Efficient Algorithm for Federated Learning	Google	Video
	FetchSGD: Communication-Efficient Federated Learning with Sketching	UC Berkeley; Johns Hopkins University; Amazon	Video Code
	SCAFFOLD: Stochastic Controlled Averaging for Federated Learning	EPFL; Google	Video
	Federated Learning with Only Positive Labels	Google	Video
	From Local SGD to Local Fixed-Point Methods for Federated Learning	Moscow Institute of Physics and Technology; KAUST	Slide Video
	Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization	KAUST	Slide Video
ICML 2019	Bayesian Nonparametric Federated Learning of Neural Networks	IBM	Code
	Analyzing Federated Learning through an Adversarial Lens	Princeton University; IBM	Code
	Agnostic Federated Learning	Google

ICLR

Years	Title	Affiliation	Materials
ICLR 2022	Bayesian Framework for Gradient Leakage	ETH Zurich	Code
	Federated Learning from only unlabeled data with class-conditional-sharing clients	The University of Tokyo; The Chinese University of Hong Kong	Code
	FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning	Carnegie Mellon University; University of Illinois at Urbana-Champaign; University of Washington
	Acceleration of Federated Learning with Alleviated Forgetting in Local Training	Tsinghua University	Code
	FedPara: Low-rank Hadamard Product for Communication-Efficient Federated Learning	POSTECH	Code
	An Agnostic Approach to Federated Learning with Class Imbalance	University of Pennsylvania	Code
	Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization	Michigan State University; The University of Texas at Austin	code
	Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models	University of Maryland; New York University	code (Minimum) code (Comprehensive)
	ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity	University of Cambridge; University of Oxford
	Diverse Client Selection for Federated Learning via Submodular Maximization	Intel; Carnegie Mellon University	code
	Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?	Purdue University	code
	Diurnal or Nocturnal? Federated Learning of Multi-branch Networks from Periodically Shifting Distributions	University of Maryland; Google	code
	Towards Model Agnostic Federated Learning Using Knowledge Distillation	EPFL
	Divergence-aware Federated Self-Supervised Learning	Nanyang Technological University; SenseTime
	What Do We Mean by Generalization in Federated Learning?	Stanford University; Google	code
	FedBABU: Toward Enhanced Representation for Federated Image Classification	KAIST	code
	Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing	EPFL	code
	Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters	Aibee	code
	Hybrid Local SGD for Federated Learning with Heterogeneous Communications	University of Texas; Pennsylvania State University
	On Bridging Generic and Personalized Federated Learning for Image Classification	The Ohio State University	code
	Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond	KAIST; MIT
ICLR 2021	Federated Learning Based on Dynamic Regularization	Boston University; ARM
	Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning	The Ohio State University
	HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients	Duke University	code
	FedMix: Approximation of Mixup under Mean Augmented Federated Learning	KAIST
	Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms	CMU; Google	code
	Adaptive Federated Optimization	Google	code
	Personalized Federated Learning with First Order Model Optimization	Stanford University; NVIDIA
	FedBN: Federated Learning on Non-IID Features via Local Batch Normalization	Princeton University	code
	FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning	The Ohio State University
	Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning	KAIST	code
ICLR 2020	Federated Adversarial Domain Adaptation	Boston University; Columbia University; Rutgers University
	DBA: Distributed Backdoor Attacks against Federated Learning	Zhejiang University; IBM Research	Code
	Fair Resource Allocation in Federated Learning	CMU; Facebook AI	Code
	Federated Learning with Matched Averaging	University of Wisconsin-Madison; IBM Research	Code
	Differentially Private Meta-Learning	CMU
	Generative Models for Effective ML on Private, Decentralized Datasets	Google	Code
	On the Convergence of FedAvg on Non-IID Data	Peking University	Code

NeurIPS

Years	Title	Affiliation	Materials
NeurIPS 2022	Federated Learning from Pre-Trained Models: A Contrastive Learning Approach	University of Technology Sydney
	CalFAT: Calibrated Federated Adversarial Training with Label Skewness	Zhejiang University
	DENSE: Data-Free One-Shot Federated Learning	Zhejiang University
	Federated Submodel Optimization for Hot and Cold Data Features	Shanghai Jiao Tong University	code
	SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression	CMU	code
	Factorized-FL: Personalized Federated Learning with Parameter Factorization & Similarity Matching	KAIST
	FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction	Michigan State University	code
	A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits	UCLA	code
	Byzantine-tolerant federated Gaussian process regression for streaming data	Pennsylvania State University
	Preservation of the Global Knowledge by Not-True Distillation in Federated Learning	KAIST
	TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels	UC Berkeley	code
	A Unified Analysis of Federated Learning with Arbitrary Client Participation	IBM	code
	SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training	Duke University	code
	A Communication-efficient Algorithm with Linear Convergence for Federated Minimax Learning	Northwestern University
	Resource-Adaptive Federated Learning with All-In-One Neural Composition	Johns Hopkins University
	Fairness in Federated Learning via Core-Stability	University of Illinois at Urbana Champaign	code
	FedSR: A Simple and Effective Domain Generalization Method for Federated Learning	University of Oxford	code
	On Sample Optimality in Personalized Collaborative and Federated Learning	Inria
	Global Convergence of Federated Learning for Mixed Regression	Northeastern University
	DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing	Hong Kong University of Science and Technology
	SAGDA: Achieving Communication Complexity in Federated Min-Max Learning	The Ohio State University
	SAGDA: Achieving Communication Complexity in Federated Min-Max Learning	The Ohio State University
	FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning	Tsinghua University	code
	FedPop: A Bayesian Approach for Personalised Federated Learning	Skolkovo Institute of Science and Technology	code
	Self-Aware Personalized Federated Learning	Amazon
	Recovering Private Text in Federated Learning of Language Models	Princeton University	code
	Communication Efficient Federated Learning for Generalized Linear Bandits	University of Virginia	code
	A Coupled Design of Exploiting Record Similarity for Practical Vertical Federated Learning	National University of Singapore	code
	On Privacy and Personalization in Cross-Silo Federated Learning	CMU	code
	Personalized Online Federated Learning with Multiple Kernels	University of California Irvine	code
	Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with an Inexact Prox	KAUST
	Learning to Attack Federated Learning: A Model-based Reinforcement Learning Attack Framework	Tulane University	code
	Coresets for Vertical Federated Learning: Regularized Linear Regression and K-Means Clustering	Nanjing University	code
	An Adaptive Kernel Approach to Federated Learning of Heterogeneous Causal Effects	National University of Singapore	code
	LAMP: Extracting Text from Gradients with Language Model Priors	ETH Zurich	code
	SecureFedYJ: a safe feature Gaussianization protocol for Federated Learning	Owkin Inc
	VF-PS: How to Select Important Participants in Vertical Federated Learning, Efficiently and Securely?	Wuhan University
	Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning	EPFL
	Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning	KAUST
	Taming Fat-Tailed (“Heavier-Tailed” with Potentially Infinite Variance) Noise in Federated Learning	The Ohio State University
	FedAvg with Fine Tuning: Local Updates Lead to Representation Learning	The University of Texas at Austin
	Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness	Peking University	code
	On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond	Baidu Research
	Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams	University of Wisconsin-Madison
	Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks	HKUST
	Asymptotic Behaviors of Projected Stochastic Approximation: A Jump Diffusion Perspective	Peking University
	Subspace Recovery from Heterogeneous Data with Non-isotropic Noise	Stanford University
	EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization	KAUST
	On-Demand Sampling: Learning Optimally from Multiple Distributions	University of California, Berkeley	code
	Improved Utility Analysis of Private CountSketch	University of Copenhagen	code
	Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning	Huawei Technologies France	code
	Decentralized Local Stochastic Extra-Gradient for Variational Inequalities	Yandex	code
	BEER: Fast Rate for Decentralized Nonconvex Optimization with Communication Compression	Princeton University	code
	Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning	NTT DATA Mathematical Systems Inc
	Near-Optimal Collaborative Learning in Bandits	Université Paris Cité	code
	Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees	Yandex
	Towards Optimal Communication Complexity in Distributed Non-Convex Optimization	TTIC	code
NeurIPS 2021	Sageflow: Robust Federated Learning against Both Stragglers and Adversaries	KAIST	HomePage
	CAFE: Catastrophic Data Leakage in Vertical Federated Learning	Rensselaer Polytechnic Institute; IBM Research	code HomePage
	Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee	NUS	code HomePage
	Optimality and Stability in Federated Learning: A Game-theoretic Approach	Cornell University	code HomePage
	QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning	UCLA	HomePage
	The Skellam Mechanism for Differentially Private Federated Learning	Google Research; CMU	HomePage
	No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data	NUS; Huawei	HomePage
	STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning	University of Minnesota	HomePage
	Subgraph Federated Learning with Missing Neighbor Generation	Emory University; University of British Columbia; Lehigh University	HomePage
	Evaluating Gradient Inversion Attacks and Defenses in Federated Learning	Princeton University	Code HomePage
	Personalized Federated Learning With Gaussian Processes	Bar-Ilan University	code HomePage
	Differentially Private Federated Bayesian Optimization with Distributed Exploration	MIT; NUS	code HomePage
	Parameterized Knowledge Transfer for Personalized Federated Learning	Hong Kong Polytechnic University;	HomePage
	Federated Reconstruction: Partially Local Federated Learning	Google Research	HomePage
	Fast Federated Learning in the Presence of Arbitrary Device Unavailability	Tsinghua University; Princeton University; MIT	code HomePage
	FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective	Duke University; Accenture Labs	code HomePage
	FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout	KAUST; Samsung AI Center	HomePage
	Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients	University of Pennsylvania	HomePage
	Federated Multi-Task Learning under a Mixture of Distributions	INRIA; Accenture Labs	code HomePage
	Federated Graph Classification over Non-IID Graphs	Emory University	HomePage
	Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing	CMU; Hewlett Packard Enterprise	code HomePage
	On Large-Cohort Training for Federated Learning	Google; CMU	code HomePage
	DeepReduce: A Sparse-tensor Communication Framework for Federated Deep Learning	KAUST; Columbia University; University of Central Florida	code HomePage
	PartialFed: Cross-Domain Personalized Federated Learning via Partial Initialization	Huawei	HomePage
	Federated Split Task-Agnostic Vision Transformer for COVID-19 CXR Diagnosis	KAIST	HomePage
	Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning	Tsinghua University; Alibaba; Weill Cornell Medicine	code HomePage
	Federated Linear Contextual Bandits	The Pennsylvania State University; Facebook; University of Virginia	HomePage
	Few-Round Learning for Federated Learning	KAIST	HomePage
	Breaking the centralized barrier for cross-device federated learning	EPFL; Google Research	code HomePage
	Federated-EM with heterogeneity mitigation and variance reduction	Ecole Polytechnique; Google Research	HomePage
	Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning	MIT; Amazon; Google	HomePage
	FedDR – Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization	University of North Carolina at Chapel Hill; IBM Research	code HomePage
	Gradient Inversion with Generative Image Prior	Pohang University of Science and Technology; University of Wisconsin-Madison; University of Washington	code HomePage
NeurIPS 2020	Differentially-Private Federated Linear Bandits	MIT	code
	Federated Principal Component Analysis	University of Cambridge; Quine Technologies	code
	FedSplit: an algorithmic framework for fast federated optimization	UC Berkeley
	Federated Bayesian Optimization via Thompson Sampling	NUS; MIT
	Lower Bounds and Optimal Algorithms for Personalized Federated Learning	KAUST
	Robust Federated Learning: The Case of Affine Distribution Shifts	UC Santa Barbara; MIT
	An Efficient Framework for Clustered Federated Learning	UC Berkeley; DeepMind	Code
	Distributionally Robust Federated Averaging	Pennsylvania State University	Code
	Personalized Federated Learning with Moreau Envelopes	The University of Sydney	code
	Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach	MIT; UT Austin
	Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge	University of Southern California	code
	Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization	CMU; Princeton University
	Attack of the Tails: Yes, You Really Can Backdoor Federated Learning	University of Wisconsin-Madison
	Federated Accelerated Stochastic Gradient Descent	Stanford University	code
	Inverting Gradients - How easy is it to break privacy in federated learning?	University of Siegen	code
	Ensemble Distillation for Robust Model Fusion in Federated Learning	EPFL
	Throughput-Optimal Topology Design for Cross-Silo Federated Learning	INRIA	code
NeurIPS 2018	cpSGD: Communication-efficient and differentially-private distributed SGD	Princeton University; Google
NeurIPS 2017	Federated Multi-Task Learning	Stanford; USC; CMU	code

 

Top Computer Vision Conferences

In this section, we will summarize Federated Learning papers accepted by top computer vision conference, Including CVPR, ICCV, ECCV.

CVPR

Years	Title	Affiliation	Materials
CVPR 2022	FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction	National University of Defense Technology	code
	Federated Class-Incremental Learning	Chinese Academy of Sciences; Northwestern University; University of Technology Sydney	code
	Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning	Peking University; JD Explore Academy; The University of Sydney
	Differentially Private Federated Learning with Local Regularization and Sparsification	Chinese Academy of Sciences
	Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage	University of Tennessee; Oak Ridge National Laboratory; Google Research	code video
	FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learning	Duke University
	Learn from Others and Be Yourself in Heterogeneous Federated Learning	Wuhan University	code video
	ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning	Arizona State University	code
	Robust Federated Learning with Noisy and Heterogeneous Clients	Wuhan University	code
	Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning	Stanford University	video code
	CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning	Shanghai Jiao Tong University
	FedCorr: Multi-Stage Federated Learning for Label Noise Correction	Singapore University of Technology and Design	video code
	ATPFL: Automatic Trajectory Prediction Model Design under Federated Learning Framework	Harbin Institute of Technology
	Federated Learning with Position-Aware Neurons	Nanjing University
	RSCFed: Random Sampling Consensus Federated Semi-supervised Learning	The Hong Kong University of Science and Technology	code
	Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation	Univ. of Pittsburgh; NVIDIA	code
	Layer-wised Model Aggregation for Personalized Federated Learning	The Hong Kong Polytechnic University
	Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning	University of Central Florida	code
CVPR 2021	Multi-Institutional Collaborations for Improving Deep Learning-Based Magnetic Resonance Image Reconstruction Using Federated Learning	Johns Hopkins University	code
	Model-Contrastive Federated Learning	National University of Singapore; UC Berkeley	code
	FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space	The Chinese University of Hong Kong	code
	Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective	Duke University	code

ECCV

Years	Title	Affiliation	Materials
ECCV 2020	Federated Visual Classification with Real-World Data Distribution	MIT; Google	Video

ICCV

Years	Title	Affiliation	Materials
ICCV 2021	Federated Learning for Non-IID Data via Unified Feature Learning and Optimization Objective Alignment	Peking University
	Ensemble Attention Distillation for Privacy-Preserving Federated Learning	University at Buffalo
	Collaborative Unsupervised Visual Representation Learning from Decentralized Data	Nanyang Technological University; SenseTime

 

Top Artificial Intelligence and Data Mining Conferences

In this section, we will summarize Federated Learning papers accepted by top AI and DM conference, Including AAAI, AISTATS, KDD.

AAAI

Years	Title	Affiliation	Materials
AAAI 2022	HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images	The Chinese University of Hong Kong; Beihang University	code
	Cross-Modal Federated Human Activity Recognition via Modality-Agnostic and Modality-Specific Representation Learning	Chinese Academy of Sciences
	FedInv: Byzantine-Robust Federated Learning by Inversing Local Model Updates	Nanjing University of Aeronautics and Astronautics
	Learning Advanced Client Selection Strategy for Federated Learning	Harvard University
	Federated Learning for Face Recognition with Gradient Correction	Beijing University of Posts and Telecommunications
	SpreadGNN: Decentralized Multi-Task Federated Learning for Graph Neural Networks on Molecular Data	university of Southern California	code
	SmartIdx: Reducing Communication Cost in Federated Learning by Exploiting the CNNs Structures	Harbin Institute of Technology; Peng Cheng Laboratory
	Is Your Data Relevant?: Dynamic Selection of Relevant Data for Federated Learning	IIT Bombay
	Seizing Critical Learning Periods in Federated Learning	SUNY-Binghamton University; Louisiana State University
	Coordinating Momenta for Cross-silo Federated Learning	University of Pittsburgh
	FedProto: Federated Prototype Learning over Heterogeneous Devices	University of Technology Sydney; University of Washington	code
	FedSoft: Soft Clustered Federated Learning with Proximal Local Updating	Carnegie Mellon University
	Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better	The University of Texas at Austin	code
	FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition	National Taiwan University	code
	SplitFed: When Federated Learning Meets Split Learning	CSIRO; Lehigh University	code
	Efficient Device Scheduling with Multi-Job Federated Learning	Soochow University; Baidu
	Implicit Gradient Alignment in Distributed and Federated Learning	IIT Kanpur; EPFL
	Federated Nearest Neighbor Classification with a Colony of Fruit-Flies	IBM Research; Wichita State University	code
AAAI 2021	Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating	Xidian University; JD Tech	video
	FedRec++: Lossless Federated Recommendation with Explicit Feedback	Shenzhen University	video
	Federated Multi-Armed Bandits	University of Virginia	code video
	On the Convergence of Communication-Efficient Local SGD for Federated Learning	Temple University; University of Pittsburgh	video
	FLAME: Differentially Private Federated Learning in the Shuffle Model	Renmin University of China; Kyoto University	video code
	Toward Understanding the Influence of Individual Clients in Federated Learning	Shanghai Jiao Tong University; The University of Texas at Dallas	video
	Provably Secure Federated Learning against Malicious Clients	Duke University	video slides
	Personalized Cross-Silo Federated Learning on Non-IID Data	Simon Fraser University; McMaster University	video
	Model-Sharing Games: Analyzing Federated Learning under Voluntary Participation	Cornell University	code video
	Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning	University of Nevada; IBM Research	video
	Game of Gradients: Mitigating Irrelevant Clients in Federated Learning	IIT Bombay; IBM Research	video Supplementary
	Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models	The Chinese University of Hong Kong; Arizona State University	video code
	Adressing Class Imbalance in Federated Learning	Northwestern University	video code
	Defending against Backdoors in Federated Learning with Robust Learning Rate	The University of Texas at Dallas	video code
AAAI 2020	Practical Federated Gradient Boosting Decision Trees	National University of Singapore; The University of Western Australia	code
	Federated Learning for Vision-and-Language Grounding Problems	Peking University; Tencent
	Federated Latent Dirichlet Allocation: A Local Differential Privacy Based Framework	Beihang University
	Federated Patient Hashing	Cornell University
	Robust Federated Learning via Collaborative Machine Teaching	Symantec Research Labs; KAUST

AISTATS

Years	Title	Affiliation	Materials
AISTATS 2022	Federated Reinforcement Learning with Environment Heterogeneity	Peking University	code
	Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning	KAUST
	Federated Learning with Buffered Asynchronous Aggregation	Meta AI	video
	Federated Myopic Community Detection with One-shot Communication	Purdue University
	QLSD: Quantised Langevin Stochastic Dynamics for Bayesian Federated Learning	Criteo AI Lab	video code
	Asynchronous Upper Confidence Bound Algorithms for Federated Linear Bandits	University of Virginia	code
	SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification	Princeton University	video code
	Federated Functional Gradient Boosting	University of Pennsylvania	code
	Towards Federated Bayesian Network Structure Learning with Continuous Optimization	Carnegie Mellon University	code
	Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective	Stanford University	code
	Differentially Private Federated Learning on Heterogeneous Data	Stanford University	code
	Towards Understanding Biased Client Selection in Federated Learning	Carnegie Mellon University	code
	FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning	KAUST	code
AISTATS 2021	Free-rider Attacks on Model Aggregation in Federated Learning	Accenture Labs	video Supplementary
	Federated f-differential privacy	University of Pennsylvania	code video Supplementary
	Federated learning with compression: Unified analysis and sharp guarantees	The Pennsylvania State University; The University of Texas at Austin	code video Supplementary
	Shuffled Model of Differential Privacy in Federated Learning	UCLA; Google	video Supplementary
	Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning	Google	video Supplementary
	Federated Multi-armed Bandits with Personalization	University of Virginia; The Pennsylvania State University	code video Supplementary
	Towards Flexible Device Participation in Federated Learning	CMU; Sun Yat-Sen University	video Supplementary
AISTATS 2020	FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization	UC Santa Barbara; UT Austin	video Supplementary
	How To Backdoor Federated Learning	Cornell Tech	video code Supplementary
	Federated Heavy Hitters Discovery with Differential Privacy	RPI; Google	video Supplementary

KDD

Years	Sessions	Title	Affiliation	Materials
KDD 2022	Research Track	Collaboration Equilibrium in Federated Learning	Tsinghua University; Alibaba Group	code
		Connected Low-Loss Subspace Learning for a Personalization in Federated Learning	Ulsan National Institute of Science and Technology & Kakao Enterprise	code
		Communication-Efficient Robust Federated Learning with Noisy Labels	University of Pittsburgh
		FedMSplit: Correlation-Adaptive Federated Multi-Task Learning across Multimodal Split Networks
	Application Track	Fed-LTD: Towards Cross-Platform Ride Hailing via Federated Learning to Dispatch	Beihang University
		No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices	Renmin University of China
		EasyFGL: Towards a Unified, Comprehensive and Efficient Platform for Federated Graph Learning	Alibaba Group	code
		FedAttack: Effective and Covert Poisoning Attack on Federated Recommendation via Hard Sampling	Tsinghua University	code
KDD 2021	Research Track	Fed2: Feature-Aligned Federated Learning	George Mason University; Microsoft; University of Maryland
		FedRS: Federated Learning with Restricted Softmax for Label Distribution Non-IID Data	Nanjing University	code
		Federated Adversarial Debiasing for Fair and Trasnferable Representations	Michigan State University	HomePage
		Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling	University of Southern California	code
	Application Track	AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization	JD Tech
		FLOP: Federated Learning on Medical Datasets using Partial Networks	Duke University	code
KDD 2020	Research Track	FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems	University College Dublin	video
	Application Track	Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data	JD Tech	video

 

Books

• 联邦学习（Federated Learning）

  Chinese Version

  English Version

• 联邦学习实战（Practicing Federated Learning）

  Chinese Version

  Github

• Federated Learning - A Comprehensive Overview of Methods and Applications

  English Version  

Papers

1. Model Aggregation

Model Aggregation (or Model Fusion) refers to how to combine local models into a shared global model.

Papers	Abbreviation	Conferences/Affiliations	Materials
Communication-Efficient Learning of Deep Networks from Decentralized Data	FedAvg	ASTATS 2017
Bayesian Nonparametric Federated Learning of Neural Networks	PFNM	ICML 2019	code
Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent	Krum	NeurIPS 2017
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates	median; trimmed mean	ICML 2018
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms	median; mean	NeurIPS 2020
The hidden vulnerability of distributed learning in byzantium	Bulyan	ICML 2018
Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance	Zeno	ICML 2019	code
Statistical Model Aggregation via Parameter Matching	SPAHM	NeurIPS 2019	code
Fed+: A Unified Approach to Robust Personalized Federated Learning	Fed+
FEDERATED OPTIMIZATION IN HETEROGENEOUS NETWORKS	FedProx	MLSys 2020	code
Separation of Powers in Federated Learning	Truda
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning	FedBE	ICLR 2021 (The Ohio State University)
Federated Learning with Matched Averaging	FedMA	ICLR 2020 (University of Wisconsin-Madison; IBM)	Code
FedSim: Similarity Guided Model Aggregation for Federated Learning	FedSim	Neurocomputing 2022 (Vol 483) (Robert Gordon University, UK)	Code

 

2. Personalization

Personalized federated learning refers to train a model for each client, based on the client’s own dataset and the datasets of other clients. There are two major motivations for personalized federated learning：

• Due to statistical heterogeneity across clients, a single global model would not be a good choice for all clients. Sometimes, the local models trained solely on their private data perform better than the global shared model.
• Different clients need models specifically customized to their own environment. As an example of model heterogeneity, consider the sentence: “I live in .....”. The next-word prediction task applied on this sentence needs to predict a different answer customized for each user. Different clients may assign different labels to the same data.

Personalized federated learning Survey paper:

• Survey of Personalization Techniques for Federated Learning

• Three Approaches for Personalization with Applications to Federated Learning

• Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Framework

Methodology	Papers	Conferences/Affiliations	Materials
Multi-Task Learning	Federated Multi-Task Learning	NeurIPS 2017 (Stanford; USC; CMU)	code
	Decentralized Collaborative Learning of Personalized Models over Networks	AISTATS 2017 (INRIA)
	Variational Federated Multi-Task Learning	ETH Zurich
	Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs	AISTATS 2020 (INRIA)	video
	Personalized Cross-Silo Federated Learning on Non-IID Data	AAAI 2021 (Simon Fraser University; McMaster University; Huawei Technologies Canada)	video
	Ditto: Fair and Robust Federated Learning Through Personalization	ICML 2021 (CMU; Facebook AI)	code video
	Federated Multi-Task Learning under a Mixture of Distributions	NeurIPS 2021 (Inria; Accenture Labs)	code
Meta Learning	Personalized Federated Learning: A Meta-Learning Approach	MIT
	Debiasing Model Updates for Improving Personalized Federated Training	ICML 2021 (Boston University; Arm)	video
	Improving Federated Learning Personalization via Model Agnostic Meta Learning	University of Washington; Google
	Adaptive Gradient-Based Meta-Learning Methods	CMU
	Federated Meta-Learning with Fast Convergence and Efficient Communication	Huawei Noah’s Ark Lab
Mixture of Global and Local Models	Federated Learning of a Mixture of Global and Local Models	KAUST
	Federated User Representation Learning	University of Michigan Facebook
	Adaptive Personalized Federated Learning	The Pennsylvania State University
Personalization Layers	Federated Learning with Personalization Layers	Adobe Research Indian Institute of Technology
	Think Locally, Act Globally: Federated Learning with Local and Global Representations	CMU University of Tokyo Columbia University
	Exploiting Shared Representations for Personalized Federated Learning	ICML 2021 (University of Texas at Austin; University of Pennsylvania)	code video
Transfer Learning	Federated evaluation of on-device personalization	Google
	Salvaging Federated Learning by Local Adaptation	Cornell University
	Private Federated Learning with Domain Adaptation	Oracle Labs
	QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning	NeurIPS 2021 (UCLA)	HomePage
Clustering	Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints	Fraunhofer Heinrich Hertz Institute	Code
	An Efficient Framework for Clustered Federated Learning	UC Berkeley DeepMind	Code
	Robust Federated Learning in a Heterogeneous Environment	UC Berkeley
	Personalized Federated Learning with First Order Model Optimization	ICLR 2021 (Stanford University; NVIDIA)
Hypernetwork	Personalized Federated Learning using Hypernetworks	Bar-Ilan University; NVIDIA	code HomePage video

 

3. Recommender system

Recommender system (RecSys) is widely used to solve information overload. In general, the more data RecSys use, the better the recommendation performance we can obtain.

Traditionally, RecSys requires the data that are distributed across multiple devices to be uploaded to the central database for model training. However, due to privacy and security concerns, such directly sharing user data strategies are no longer appropriate.

The incorporation of federated learning and RecSys is a promising approach, which can alleviate the risk of privacy leakage.

More federated recommendation papers can be found in this repository: FedRecPapers

Methodology	Papers	Conferences/Affiliations	Materials
Matrix Factorization	Secure federated matrix factorization	IEEE Intelligent Systems
	Federated Multi-view Matrix Factorization for Personalized Recommendations	ECML-PKDD 2020	video
	Decentralized Recommendation Based on Matrix Factorization: A Comparison of Gossip and Federated Learning	ECML-PKDD 2019
	Towards Privacy-preserving Mobile Applications with Federated Learning: The Case of Matrix Factorization	MobiSys 2019
	Meta Matrix Factorization for Federated Rating Predictions	ACM SIGIR 2020	code
	Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System	Arxiv
GNN	FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation	Arxiv
	Federated Social Recommendation with Graph Neural Network	ACM TIST
	Personalized Federated Recommendation via Joint Representation Learning, User Clustering, and Model Adaptation	CIKM 2022
	Federated Meta Embedding Concept Stock Recommendation	IEEE Big Data 2022
	Federated Learning-Based Cross-Enterprise Recommendation With Graph Neural Networks	IEEE Transactions on Industrial Informatics

 

4. Security

4.1. Attack

Methodology	Papers	Conferences/Affiliations	Materials
Backdoor Attack	How To Backdoor Federated Learning	AISTATS 2020	code
	Can You Really Backdoor Federated Learning?	Arxiv
	Attack of the Tails: Yes, You Really Can Backdoor Federated Learning	NeurIPS 2020	code
	DBA: Distributed Backdoor Attacks against Federated Learning	ICLR 2020	code
Gradients Attack	Deep Leakage from Gradients	NeurIPS 2020	HomePage code
	Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix	ICML 2021	code
	Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning	ACM CCS 2017	video
	iDLG: Improved Deep Leakage from Gradients	Arxiv	code
	See through Gradients: Image Batch Recovery via GradInversion	CVPR 2021
	Inverting Gradients - How easy is it to break Privacy in Federated Learning?	NeurIPS 2020	code
	CAFE: Catastrophic Data Leakage in Vertical Federated Learning	NeurIPS 2021 (Rensselaer Polytechnic Institute; IBM)	code HomePage
	Evaluating Gradient Inversion Attacks and Defenses in Federated Learning	NeurIPS 2021 (Princeton University)
	R-GAP: RECURSIVE GRADIENT ATTACK ON PRIVACY	ICLR 2021 (KU Leuven, Belgium)	Code
	Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage	CVPR 2022 (University of Tennessee;Oak Ridge National Laboratory)	Code
Model Poison Attack	Analyzing Federated Learning through an Adversarial Lens	ICML 2019 (Princeton University; IBM)	Code

 

4.2. Defense

Methodology	Papers	Conferences/Affiliations	Materials
FL+DP	Federated Learning With Differential Privacy: Algorithms and Performance Analysis	IEEE Transactions on Information Forensics and Security
	Differentially Private Federated Learning: A Client Level Perspective	Arxiv	code
	Learning Differentially Private Recurrent Language Models	ICLR 2018
	The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation	ICML 2021 (Google)	video
	The Skellam Mechanism for Differentially Private Federated Learning	NeurIPS 2021 (Google Research; CMU)	HomePage
FL+HE	Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption	Arxiv
	BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning	USENIX 2020	code
FL+TEE	PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments	ACM MobiSys 2021
	Darknetz: towards model privacy at the edge using trusted execution environments.	ACM MobiSys 2020	code video
Algorithm	A Little Is Enough: Circumventing Defenses For Distributed Learning	NeurIPS 2019
	CRFL: Certifiably Robust Federated Learning against Backdoor Attacks	ICML 2021 (UIUC; IBM)	code video
	Evaluating Gradient Inversion Attacks and Defenses in Federated Learning	NeurIPS 2021 (Princeton University)	Code HomePage

 

5. Survey

Category	Papers
General	Federated machine learning: Concept and applications
	A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
	Federated Learning in Mobile Edge Networks: A Comprehensive Survey
	Advances and Open Problems in Federated Learning
	Federated Learning: Challenges, Methods, and Future Directions
	The Internet of Federated Things (IoFT)
Security	A survey on security and privacy of federated learning
	Threats to Federated Learning: A Survey
	Vulnerabilities in Federated Learning
Personalization	Survey of Personalization Techniques for Federated Learning
	Towards Personalized Federated Learning
Aggregation	Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
Incentive	A Comprehensive Survey of Incentive Mechanism for Federated Learning
	A Survey of Incentive Mechanism Design for Federated Learning
Applications	A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
Fairness	A Survey of Fairness-Aware Federated Learning
Graph	FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks
	Federated Graph Learning - A Position Paper
	Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applications
	Federated Graph Neural Networks: Overview, Techniques and Challenges
System	Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies

 

7. Efficiency

Communication-Based: Improving efficiency by reducing model parameters transmission.

Hardware-Based: Improving efficiency by hardware acceleration (GPU, FPGA, etc.)

Algorithm-Based: Improving efficiency by accelerating model convergence rate (local training, model aggregation, client selection, etc.)

Taxonomy	Papers	Techniques	Materials
Communication-Based	Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients	Quantization
	Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning	Quantization
	Communication Efficient Federated Learning with Adaptive Quantization	Quantization
	QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding	Quantization
	SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communications	Sparsity
	RPN: A Residual Pooling Network for Efficient Federated Learning	Sparsity
	FedMD: Heterogenous Federated Learning via Model Distillation	Knowledge Distillation	code
	Ensemble distillation for robust model fusion in federated learning	Knowledge Distillation	code
Hardware-Based	FPGA-Based Hardware Accelerator of Homomorphic Encryption for Efficient Federated Learning	FPGA-Based Acceleration
	Hardware Accelerated Learning at the Edge	GPU-Based Acceleration
	HAFLO: GPU-Based Acceleration for Federated Logistic Regression	GPU-Based Acceleration
Algorithm	FetchSGD: Communication-Efficient Federated Learning with Sketching	Optimization	Video Code
	Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization	Optimization	Slide Video
	FedBoost: A Communication-Efficient Algorithm for Federated Learning	Optimization	Video
	Federated Learning: Strategies for Improving Communication Efficiency	Optimization
	One-Shot Federated Learning	Model Aggregation

 

8. Optimization

Papers	Application Scenarios	Conferences/Affiliations	Materials
Federated Composite Optimization	loss function contains a non-smooth regularizer	ICML 2021(Google)	code
Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	Federated Deep AUC Maximization	ICML 2021 (The University of Iowa)	video
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning	Non-Convex Objective optimization	ICML 2021 (The University of Tokyo)	video
From Local SGD to Local Fixed-Point Methods for Federated Learning	fixed-point algorithms optimization	ICML 2020 (Moscow Institute of Physics and Technology; KAUST)	Slide Video
Federated Learning Based on Dynamic Regularization	In each round, the objective function for each device dynamically updates its regularizer	ICLR 2021 (Boston University; ARM)
Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms	Formulate federated learning optimization as a posterior inference problem	ICLR 2021 (CMU; Google)	code
Adaptive Federated Optimization	Federated versions of adaptive optimizers	ICLR 2021 (Google)	code
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization	How to uses local batch normalization to alleviate the feature shift before averaging models.	ICLR 2021 (Princeton University)	code
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning	SFederated versions of emi-Supervised Learning	ICLR 2021 (KAIST)	code
Sageflow: Robust Federated Learning against Both Stragglers and Adversaries	Handle both stragglers (slow devices) and adversaries simultaneously	NeurIPS 2021 (KAIST)	HomePage
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning	Distributed stochastic non-convex optimization	University of Minnesota	HomePage
Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating	Vertical Federated Learning Optimization	AAAI 2021 (Xidian University; JD Tech)	video

 

9. Fairness

Taxonomy	Papers	Conferences/Affiliations	Materials
Performance Fairness	Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning	NeurIPS 2021 (Tsinghua University; Alibaba)	code HomePage
	Fairness-aware Agnostic Federated Learning	SDM 2021 (University of Arkansas)
	Fair Resource Allocation in Federated Learning	ICLR 2020 (CMU; Facebook AI)	Code
	Agnostic Federated Learning	ICML 2019 (Google)
	Mitigating Bias in Federated Learning	arXiv (IBM)
	Ditto: Fair and Robust Federated Learning Through Personalization	ICML 2021 (CMU; Facebook AI)	code video
Client Selection Fairness	An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee	IEEE Transactions on Parallel and Distributed Systems
	Stochastic Client Selection for Federated Learning with Volatile Clients	IEEE Internet of Things Journal
	Federated learning with class imbalance reduction	European Signal Processing Conference
	Reputation-Based Federated Learning for Secure Wireless Networks	” IEEE Internet of Things Journal
Contribution Fairness	Profit Allocation for Federated Learning	2019 IEEE International Conference on Big Data
	Stochastic Client Selection for Federated Learning with Volatile Clients	IEEE Internet of Things Journal
	Federated learning with class imbalance reduction	European Signal Processing Conference
	Reputation-Based Federated Learning for Secure Wireless Networks	” IEEE Internet of Things Journal

 

10. Applications

Applications	Papers	Conferences/Affiliations	Materials
Computer Vision	FedVision: An Online Visual Object Detection Platform Powered by Federated Learning	WeBank (AAAI 2020)	code
Nature Language Processing	Federated learning for emoji prediction in a mobile keyboard	Google
	Federated Learning for Mobile Keyboard Prediction	Google
	Applied federated learning: Improving google keyboard query suggestions	Google
	Federated Learning Of Out-Of-Vocabulary Words	Google
Automatic Speech Recognition	A Federated Approach in Training Acoustic Models	MicroSoft (INTERSPEECH 2020)	Video
	Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?	INRIA (INTERSPEECH 2019)
	Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework	Google (ICASSP 2021)	Google Assistant Help
	Federated Evaluation and Tuning for On-Device Personalization: System Design \& Applications	Apple	Report
Healthcare	Privacy-preserving Federated Brain Tumour Segmentation	NVIDIA (MICCAI MLMI 2019)
	Advancing health research with Google Health Studies	Google	Blog
	Multi-institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation	Intel	Blog
Blockchain	FedCoin: A Peer-to-Peer Payment System for Federated Learning	Arxiv
	Blockchained On-Device Federated Learning	IEEE Communications Letters 2019

 

11. Boosting

Category	Papers	Conferences/Affiliations	Materials
Tree-Base Boosting	Practical Federated Gradient Boosting Decision Trees	AAAI 2020 (NUS)	code
	Secureboost: A lossless federated learning framework	IEEE Intelligent Systems 2021 (WeBank; HKUST)
	Large-scale Secure XGB for Vertical Federated Learning	CIKM 2021 (Ant Group)	video

 

12. Incentive mechanism

Typically, the incentive mechanism consists of the following two steps:

• How to evaluate the contribution of each participant (Shapley value)

• How to allocate profits based on contributions

Steps	Papers	Conferences/Affiliations	Materials
1. Contribution Evaluation	Data Shapley: Equitable Valuation of Data for Machine Learning	ICML 2019 (Stanford University)	code
	A principled approach to data valuation for federated learning	Arxiv (Harvard University)
	Measure contribution of participants in federated learning	IEEE Big Data (Swiss Re)
2. Profit Allocation	GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning	IAAI 2021 (NTU)
	Profit allocation for federated learning	Arxiv (Beihang University)
	Fedcoin: A peer-to-peer payment system for federated learning	NTU

 

13. Unsupervised Learning

Category	Papers	Conferences/Affiliations	Materials
Clustering	Heterogeneity for the Win: One-Shot Federated Clustering	ICML 2021 (CMU)	video
Representations Learning	Exploiting Shared Representations for Personalized Federated Learning	ICML 2021 (University of Texas at Austin; University of Pennsylvania)	code video
	Towards Federated Unsupervised Representation Learning	EdgeSys '20 (Eindhoven University of Technology)
	Federated Unsupervised Representation Learning	Zhejiang University
	Collaborative Unsupervised Visual Representation Learning from Decentralized Data	(ICCV 2021) (Nanyang Technological University;SenseTime)
	Divergence-aware Federated Self-Supervised Learning	(ICLR 2022) (Nanyang Technological University)
	Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering	(ICML 2022) (University of Michigan;University of Cambridge)	code

 

14. Heterogeneity

Category	Papers	Conferences/Affiliations	Materials
Data Heterogeneity (NON-IID)	Data-Free Knowledge Distillation for Heterogeneous Federated Learning	ICML 2021 (Michigan State University)	code video
	Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	ICML 2021 (The University of Iowa)	video
	Exploiting Shared Representations for Personalized Federated Learning	ICML 2021 (University of Texas at Austin; University of Pennsylvania)	code video
	SCAFFOLD: Stochastic Controlled Averaging for Federated Learning	ICML 2020 (EPFL; Google)	Video
	Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning	ICLR 2021 (The Ohio State University)
	HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients	ICLR 2021 (Duke University)	code
	FedMix: Approximation of Mixup under Mean Augmented Federated Learning	ICLR 2021 (KAIST)
	On the Convergence of FedAvg on Non-IID Data	ICLR 2020 (Peking University)	Code
	No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data	NeurIPS 2021 (NUS; Huawei)	HomePage
Model Heterogeneity	Personalized Federated Learning with Moreau Envelopes	NeurIPS 2020 (The University of Sydney)	code
	Federated Learning of a Mixture of Global and Local Models	KAUST
	Salvaging Federated Learning by Local Adaptation	Cornell University
Device Heterogeneity	Sageflow: Robust Federated Learning against Both Stragglers and Adversaries	NeurIPS 2021 (KAIST)	HomePage
	Towards Flexible Device Participation in Federated Learning	AISTATS 2021 (CMU; Sun Yat-Sen University)	video Supplementary
	Asynchronous Federated Optimization	12th Annual Workshop on Optimization for Machine Learning

 

15. Client Selection

Papers	Conferences/Affiliations	Materials
Diverse Client Selection for Federated Learning via Submodular Maximization	ICLR 2022 (Intel; CMU)	code
TiFL: A Tier-based Federated Learning System	HPDC 2020 (George Mason University)
Communication-Efficient Federated Learning via Optimal Client Sampling	University of Texas at Austin
Oort: Efficient Federated Learning via Guided Participant Selection	OSDI 2021 (University of Michigan)	code
Optimizing federated learning on non-iid data with reinforcement learning	IEEE INFOCOM 2020 (University of Toronto)
Learning Advanced Client Selection Strategy for Federated Learning	AAAI 2022 (Harvard University)
Towards understanding biased client selection in federated learning	AISTATS 2022 (Carnegie Mellon University)
Stochastic Client Selection for Federated Learning with Volatile Clients	IEEE Internet of Things Journal 2022 (South China University of Technology)
Oort: Efficient Federated Learning via Guided Participant Selection	OSDI 2021 (University of Michigan)	code video
Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning	ICML 2021 (Accenture)	code video
Federated Multi-Armed Bandits	AAAI 2021 (University of Virginia)	code video
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization	AISTATS 2020 (UC Santa Barbara; UT Austin)	video Supplementary

 

16. Graph Neural Networks

Category	Papers	Conferences/Affiliations	Materials
Graph-Level	Federated Graph Classification over Non-IID Graphs	NeurIPS 2021 (Emory University)
	Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting	arXiv (University of Rochester)
	FedGCN: Federated Learning-Based Graph Convolutional Networks for Non-Euclidean Spatial Data	Mathematics 2022 (Nanjing University of Information Science and Technology)
SubGraph-Level	FedGraph: Federated Graph Learning with Intelligent Sampling	IEEE TPDS (University of Aizu)
	Glint: Decentralized Federated Graph Learning with Traffic Throttling and Flow Scheduling	IWQOS 2021 (University of Aizu)
	PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN Training Method	Arxiv (Xi’an Jiaotong University)
	Subgraph Federated Learning with Missing Neighbor Generation	NeurIPS 2021 (Emory University;)	HomePage
	FedNI: Federated Graph Learning with Network Inpainting for Population-Based Disease Prediction	Arxiv ( University of Electronic Science and Technology of China)
	FedGL: Federated Graph Learning Framework with Global Self-Supervision	Arxiv (Sun Yat-sen University)
Node-Level	Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling	KDD 2021 (University of Southern California)
	Personalized Federated Learning With Graph	IJCAI 2022 (Beihang University)	code
	BiG-Fed: Bilevel Optimization Enhanced Graph-Aided Federated Learning	FL-ICML Workshop 2021 (Nanyang Technological University)
	Decentralized Federated Graph Neural Networks	FL-IJCAI Workshop 2021 (Blue Elephant Tec)
	Peer-to-Peer Federated Learning on Graphs	Arxiv (University of California)
	A Graph Federated Architecture with Privacy Preserving Learning	Arxiv (EPFL)
	Decentralized Federated Learning for Electronic Health Records	Arxiv (IBM)
	Learn Electronic Health Records by Fully Decentralized Federated Learning	FL-NeurIPS 2019 (University of Minnesota)
	Decentralized Federated Learning via SGD over Wireless D2D Networks	Arxiv (Shenzhen University)
	SpreadGNN: Decentralized Multi-Task Federated Learning for Graph Neural Networks on Molecular Data	AAAI 2022 (university of Southern California)	code
	A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularization	Arxiv (The University of Sydney)	code
	Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds	Arxiv ( Purdue University)	code

 

18. Other Machine Learning Paradigm

Taxonomy	Papers	Conferences/Affiliations	Materials
Neural Architecture Search	Federated Neural Architecture Search	Beijing University of Posts and Telecommunications
	FedNAS: Federated Deep Learning via Neural Architecture Search	CVPR 2020 workshop (University of Southern California)
	Self-supervised cross-silo federated neural architecture search	WeBank
Active Learning	Active Federated Learning	University of Michigan
	Active learning based federated learning for waste and natural disaster image classification	Hamad Bin Khalifa University
	Federated Active Learning	Harvard Medical School
Continual Learning	Federated Continual Learning with Weighted Inter-client Transfer	KAIST (ICML 2021)	code
	Partitioned variational inference: A unified framework encompassing federated and continual learning	University of Sydney
	A distillation-based approach integrating continual learning and federated learning for pervasive services	Inria
	FedSpeech: Federated Text-to-Speech with Continual Learning	Zhejiang University

 

19. Trade off

Privacy, utility, and efficiency are the three key concepts of trustworthy federated learning. We point out that there is no security mechanism that can achieve optimality in terms of privacy leakage, utility loss, and efficiency loss simultaneously.

• No Free Lunch Theorem for Security and Utility in Federated Learning
• Trading Off Privacy, Utility and Efficiency in Federated Learning

 

Google FL Workshops

• Google Workshop on Federated Learning and Analytics 2021
• Google Workshop on Federated Learning and Analytics 2020
• Google Federated Learning workshop 2019

 

Videos and Lectures

• TensorFlow Federated Tutorial Session

• TensorFlow Federated (TFF): Machine Learning on Decentralized Data - Google, TF Dev Summit ‘19 2019

• Federated Learning: Machine Learning on Decentralized Data - Google, Google I/O 2019

• Federated Learning - Cloudera Fast Forward Labs, DataWorks Summit 2019

• GDPR, Data Shortage and AI - Qiang Yang, AAAI 2019 Invited Talk

• Code Tutorial: From Centralized to Federated - Flower Summit 2021

 

Tutorials and Blogs

• What is Federated Learning - Nvidia 2019
• Online Federated Learning Comic - Google 2019
• Federated Learning: Collaborative Machine Learning without Centralized Training Data - Google AI Blog 2017
• Go Federated with OpenFL - Intel 2021

 

Open-Sources

Developing a federated learning framework from scratch is very time-consuming, especially in industrial. An excellent FL framework can facilitate engineers and researchers to train, research and deploy the FL model in practice. In this section, we summarize some commonly used open-source FL frameworks from both industrial and academia perspectives.

Enterprise Grade

Platform	Papers	Affiliations/HomePage
FATE	FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection	WeBank
FedML	FedML: A Research Library and Benchmark for Federated Machine Learning	fedml.ai
OpenFL	OpenFL: An open-source framework for Federated Learning	Intel
NVFlare		NVIDIA
IBM Federated Learning	IBM Federated Learning: an Enterprise Framework White Paper	IBM
Fedlearner		Bytedance
PaddleFL		Baidu
Sherpa.ai Federated Learning	Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy	Sherpa.ai
FederatedScope	FederatedScope: A Flexible Federated Learning Platform for Heterogeneity	Alibaba
secretflow		Antgroup

 

Research Purpose

Platform	Papers	Affiliations/HomePage
Tensorflow-Federated	Towards Federated Learning at Scale: System Design	Google
FedJAX	FEDJAX: Federated learning simulation with JAX	Google
Flower	Flower: A Friendly Federated Learning Research Framework	flower.dev
FLUTE	FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations	Microsoft
FLSim		Meta
PySyft	A generic framework for privacy preserving deep learning	OpenMined
PyVertical	PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN	OpenMined
LEAF	LEAF: A Benchmark for Federated Settings	CMU
FedScale	FedScale: Benchmarking Model and System Performance of Federated Learning at Scale	SymbioticLab(University of Michigan)
EasyFL	EasyFL: A Low-code Federated Learning Platform For Dummies	NTU
FedLab	FedLab: A Flexible Federated Learning Framework	SMILELab-FL
Galaxy Federated Learning	GFL: A Decentralized Federated Learning Framework Based On Blockchain	Zhejiang University
FedTree		National University of Singapore

⬆ Return to top