Skip to content

Latest commit

 

History

History
304 lines (265 loc) · 23.6 KB

README.md

File metadata and controls

304 lines (265 loc) · 23.6 KB
Raw

Federated Learning with New Knowledge

This is all you need for a brand new but quite important topic -- Federated Learning with New Knowledge, including research papers, datasets, tools, and you name it. Federated Learning (FL) is a privacy-preserving distributed learning approach that is rapidly developing in an era where privacy protection is increasingly valued. It is this rapid development trend, along with the continuous emergence of new demands for FL in the real world, that prompts us to focus on a very important problem: Federated Learning with New Knowledge. The primary challenge here is to effectively incorporate various new knowledge into existing FL systems and evolve these systems to reduce costs, extend their lifespan, and facilitate sustainable development. In this work, we systematically define the main sources of new knowledge in FL, including new features, tasks, models, and algorithms. For each source, we thoroughly analyze and discuss how to incorporate new knowledge into existing FL systems and examine the impact of the form and timing of new knowledge arrival on the incorporation process. Furthermore, we comprehensively discuss the potential future directions for FL with new knowledge, considering a variety of factors such as scenario setups, efficiency, and security. For more detailed information, please refer to our survey paper Federated Learning with New Knowledge: Fundamentals, Advances, and Futures.

overview

Overview of an FL system with new knowledge from different sources. Different types of clients encounter new features and tasks over time, which reflect new demands for FL systems, e.g., client $C_{k_2}$ needs to deal with the night scenes and conduct segmentation when snowing, and client $C_{k_3}$ joins FL with the need to handle night scenes and deraining when raining. From a global view, new more advanced models with better architecture (Transformers) and larger sizes (GPT 4) are also needed to incorporate. Besides, new algorithms with better performance (Scaffold) and security guarantees (SecAgg) should be continuously employed in FL as well.

Taxonomy

New Features

Federated Domain Generalization

Federated Out-of-Distribution Detection

Federated Domain Adaptation

New Tasks

Task-personalized Federated Learning

Self-supervised Federated Learning

New Tasks with New Features

New Models

New Algorithms

Computer Vision

  • Non-IID data and Continual Learning processes in Federated Learning: A long road ahead
    • (Survey, Information Fusion 2022) [paper]

Pure Classification

  • Partitioned Variational Inference: A unified framework encompassing federated and continual learning
  • Federated and continual learning for classification tasks in a society of devices
  • A New Analysis Framework for Federated Learning on Time-Evolving Heterogeneous Data
  • Federated Continual Learning with Weighted Inter-client Transfer
  • Federated Class-Incremental Learning
  • Learn From Others and Be Yourself in Heterogeneous Federated Learning
  • Towards Federated Learning on Time-Evolving Heterogeneous Data
  • Concept drift detection and adaptation for federated and continual learning
    • (Multimedia Tools and Applications 2021) [paper]
  • Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning
  • ODE: A Data Sampling Method for Practical Federated Learning with Streaming Data and Limited Buffer
  • Diurnal or Nocturnal? Federated Learning from Periodically Shifting Distributions
  • Knowledge Lock: Overcoming Catastrophic Forgetting in Federated Learning
  • Continual Federated Learning Based on Knowledge Distillation
  • A Federated Incremental Learning Algorithm Based on Dual Attention Mechanism
  • Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal
  • Federated Continuous Learning With Broad Network Architecture
    • (IEEE Transactions on Cybernetics 2022) [paper]
  • Addressing Client Drift in Federated Continual Learning with Adaptive Optimization
  • Continual Horizontal Federated Learning for Heterogeneous Data
  • Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Drift
  • Better Generative Replay for Continual Federated Learning
  • Federated Learning for Data Streams
  • FedKNOW: Federated Continual Learning with Signature Task Knowledge Integration at Edge
  • No One Left Behind: Real-World Federated Class-Incremental Learning
  • Federated probability memory recall for federated continual learning
  • GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgetting
  • Addressing Catastrophic Forgetting in Federated Class-Continual Learning
  • Federated Learning under Distributed Concept Drift
  • Asynchronous Federated Continual Learning
    • (CVPR FedVision Workshop 2023) [paper]
  • Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learning
  • Distributed Offline Policy Optimization Over Batch Data
  • CoDeC: Communication-Efficient Decentralized Continual Learning
  • Ensemble and continual federated learning for classification tasks
  • To Store or Not? Online Data Selection for Federated Learning with Limited Storage
  • Masked Autoencoders are Efficient Continual Federated Learners
  • Semi-supervised federated learning on evolving data streams
    • (Information Sciences 2023) [paper]
  • A federated learning-based approach to recognize subjects at a high risk of hypertension in a non-stationary scenario
    • (Information Sciences 2023) [paper]
  • Fed-CPrompt: Contrastive Prompt for Rehearsal-Free Federated Continual Learning
  • Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory
  • FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformer
  • Federated Multi-Phase Curriculum Learning to Synchronously Correlate User Heterogeneity
    • (IEEE Transactions on Artificial Intelligence 2023) [paper]
  • Federated Class-Incremental Learning with Prompting
  • FedRCIL: Federated Knowledge Distillation for Representation based Contrastive Incremental Learning
  • Distributed Continual Learning with CoCoA in High-dimensional Linear Regression
  • Concept Matching: Clustering-based Federated Continual Learning
  • Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges
  • HePCo: Data-Free Heterogeneous Prompt Consolidation for Continual Federated Learning
  • Decentralized Deep Learning under Distributed Concept Drift: A Novel Approach to Dealing with Changes in Data Distributions Over Clients and Over Time
  • A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks
  • Accurate Forgetting for Heterogeneous Federated Continual Learning
  • Variational Federated Continual Learning
  • Towards Out-of-federation Generalization in Federated Learning
  • FedGP: Buffer-based Gradient Projection for Continual Federated Learning
  • Traceable Federated Continual Learning
  • Prototypes-Injected Prompt for Federated Class Incremental Learning

Advanced CV Tasks (object detection, semantic segmentation)

  • FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space
  • Federated Incremental Semantic Segmentation

Out-of-Distribution Learning (domain adaptation, domain generalization, out-of-distribution detection)

  • Uncertainty-Aware Aggregation for Federated Open Set Domain Adaptation
  • FedSR: A Simple and Effective Domain Generalization Method for Federated Learning
  • Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer
  • Federated Domain Generalization with Generalization Adjustment
  • Rethinking Federated Learning with Domain Shift: A Prototype View
  • Turning the Curse of Heterogeneity in Federated Learning into a Blessing for Out-of-Distribution Detection
  • Test-Time Robust Personalization for Federated Learning
  • PerAda: Parameter-Efficient and Generalizable Federated Learning Personalization with Guarantees
  • FedConceptEM: Robust Federated Learning Under Diverse Distribution Shifts
  • MEC-DA: Memory-Efficient Collaborative Domain Adaptation for Mobile Edge Devices
    • (IEEE Transactions on Mobile Computing 2023) [paper]
  • FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering
  • A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging
  • Benchmarking Algorithms for Federated Domain Generalization
  • Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting
  • FedNovel: Federated Novel Class Learning
  • Federated Generalization via Information-Theoretic Distribution Diversification
  • FedOD: Federated Outlier Detection via Neural Approximation

Natural Language Processing

  • Federated Learning Of Out-Of-Vocabulary Words
  • Federated Continual Learning for Text Classification via Selective Inter-client Transfer
  • Quantifying Catastrophic Forgetting in Continual Federated Learning
  • FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformer
  • Coordinated Replay Sample Selection for Continual Federated Learning

Audio and IoT

  • A distillation-based approach integrating continual learning and federated learning for pervasive services
  • FedSpeech: Federated Text-to-Speech with Continual Learning
  • Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges
  • Learnings from Federated Learning in The Real World
  • New Generation Federated Learning
  • Attention-based federated incremental learning for traffic classification in the Internet of Things
    • (Computer Communications 2022) [paper]
  • Federated Learning and catastrophic forgetting in pervasive computing: demonstration in HAR domain
  • Federated Continual Learning through distillation in pervasive computing
  • DILoCC: An approach for Distributed Incremental Learning across the Computing Continuum
  • Cross-FCL: Toward a Cross-edge Federated Continual Learning Framework in Mobile Edge Computing Systems
  • Urban Traffic Forecasting using Federated and Continual Learning
  • ICMFed: An Incremental and Cost-Efficient Mechanism of Federated Meta-Learning for Driver Distraction Detection
  • Personalized Federated Continual Learning for Task-incremental Biometrics
    • (IEEE Internet of Things Journal 2023) [paper]
  • Continual adaptation of federated reservoirs in pervasive environments
  • Continual Federated Learning For Network Anomaly Detection in 5G Open-RAN
  • Age-Aware Data Selection and Aggregator Placement for Timely Federated Continual Learning in Mobile Edge Computing
    • (IEEE Transactions on Computers 2023) [paper]

Security Relevant

  • GFCL: A GRU-based Federated Continual Learning Framework against Data Poisoning Attacks in IoV
  • Towards a Defense against Backdoor Attacks in Continual Federated Learning
  • Federated Continual Learning with Differentially Private Data Sharing
  • FL-IIDS: A novel federated learning-based incremental intrusion detection system
    • (Future Generation Computer Systems 2023) [paper]
  • POSTER: Advancing Federated Edge Computing with Continual Learning for Secure and Efficient Performance

Other Topics

  • Federated Continual Learning to Detect Accounting Anomalies in Financial Auditing
  • Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing
  • Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer
  • Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shifts
  • Concept Drift Detection and Adaptation for Robotics and Mobile Devices in Federated and Continual Settings
  • Incremental learning and federated learning for heterogeneous medical image analysis
  • Continual adaptation of federated reservoirs in pervasive environments
  • Peer-to-Peer Federated Continual Learning for Naturalistic Driving Action Recognition

Citation

If you find our survey helpful for your research and study, please consider citing our paper.

@inproceedings{Wang2024FederatedLW,
  title={Federated Learning with New Knowledge: Fundamentals, Advances, and Futures},
  author={Lixu Wang and Yang Zhao and Jiahua Dong and Ating Yin and Qinbin Li and Xiao Wang and Dusit Tao Niyato and Qi Zhu},
  year={2024},
  url={https://api.semanticscholar.org/CorpusID:267412120}
}