examples

NVIDIA FLARE Examples

NVIDIA FLARE provides several examples to help you get started using federated learning for your own applications.

The provided examples cover different aspects of NVIDIA FLARE, such as using the provided Controllers for "scatter and gather" or "cyclic weight transfer" workflows and different Executors to implement your own training and validation pipelines. Some examples use the provided "task data" and "task result" Filters for homomorphic encryption and decryption or differential privacy. Furthermore, we show how to use different components for FL algorithms such as FedAvg, FedProx, and FedOpt. We also provide domain-specific examples for deep learning and medical image analysis.

Getting started

To get started with NVIDIA FLARE, please follow the Getting Started Guide in the documentation. This walks you through installation, creating a POC workspace, and deploying your first NVIDIA FLARE Application. The following examples will detail any additional requirements in their requirements.txt.

Set up a virtual environment

We recommend setting up a virtual environment before installing the dependencies of the examples. You need to set up the virtual environment and install nvflare and set additional PYTHONPATH before launch the jupyter lab.

Install dependencies for a virtual environment with:

python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv

(If needed) make all shell scripts executable using:

find . -name ".sh" -exec chmod +x {} \;

Create and activate your virtual environment with the set_env.sh script:

source ./set_env.sh

Install nvflare

(nvflare_example)$ pip install nvflare

In each example folder, install required packages for training:

(nvflare_example)$ pip install --upgrade pip
(nvflare_example)$ pip install -r requirements.txt

(optional) some examples contains script for plotting the TensorBoard event files, if needed, please also install:

(nvflare_example)$ pip install -r plot-requirements.txt

Set up JupyterLab for notebooks

To run examples including notebooks, we recommend using JupyterLab. You need to set up the virtual environment and install nvflare and set additional PYTHONPATH before launch the jupyter lab.

After activating your virtual environment, install JupyterLab:

(nvflare_example)$ pip install jupyterlab

You can register the virtual environment you created, so it is usable in JupyterLab:

(nvflare_example)$ python3 -m ipykernel install --user --name="nvflare_example"

Start a Jupyter Lab:

(nvflare_example)$ jupyter lab .

When you open a notebook, select the kernel nvflare_example using the dropdown menu at the top right.

1. Hello World Examples

Example	Framework	Summary
Notebook for Hello Examples	-	Notebook for examples below.
Hello FedAvg NumPy	Numpy	Example using FedAvg controller workflow.
Hello Cross-Site Validation	Numpy	Example using CrossSiteEval controller workflow, and example using previous results without training workflow.
Hello Cyclic Weight Transfer	PyTorch	Example using CyclicController controller workflow to implement Cyclic Weight Transfer.
Hello PyTorch	PyTorch	Example using an image classifier using FedAvg and PyTorch as the deep learning training framework.
Hello TensorFlow	TensorFlow	Example of using an image classifier using FedAvg and TensorFlow as the deep learning training framework.

2. Step-by-Step Examples

Example	Dataset	Controller-Type	Execution API Type	Framework	Summary
image_stats	CIFAR10	server	Executor	Pandas	Example for federated stats image histogram calculation.
sag	CIFAR10	server	Client API	PyTorch	Example for FedAvg with ScatterAndGather controller workflow using the Client API.
sag_deploy_map	CIFAR10	server	Client API	PyTorch	Example showcasing site-specific configurations and deploy_map.
sag_model_learner	CIFAR10	server	ModelLearner	PyTorch	Example with ScatterAndGather using a ModelLearner.
sag_executor	CIFAR10	server	Executor	PyTorch	Example with ScatterAndGather using an Executor.
sag_mlflow	CIFAR10	server	Client API	PyTorch	MLflow experiment tracking logs with ScatterAndGather using the Client API.
sag_he	CIFAR10	server	Client API	PyTorch	Example with homomorphic encyption using Client API and POC -he mode.
cse	CIFAR10	server	Client API	PyTorch	Example using CrossSiteModelEval controller workflow.
cyclic	CIFAR10	server	Client API	PyTorch	Example for cyclic weight transfer using CyclicController controller workflow.
cyclic_ccwf	CIFAR10	client	Client API	PyTorch	Example for client-controlled cyclic weight transfer using CyclicClientController controller workflow.
swarm	CIFAR10	client	Client API	PyTorch	Example for swarm learning and client-controlled cross-site evaluation using SwarmClientController and CrossSiteEvalClientController controller workflows.
tabular_stats	HIGGS	server	Executor	Pandas	Example for federated stats tabular histogram calculation.
sklearn_linear	HIGGS	server	Client API	sklearn	Example for federated linear model (logistic regression on binary classification) learning on tabular data.
sklearn_svm	HIGGS	server	Client API	sklearn	Example for federated SVM model learning on tabular data.
sklearn_kmeans	HIGGS	server	Client API	sklearn	Example for federated k-Means clustering on tabular data.
xgboost	HIGGS	server	Client API	XGBoost	Example for federated horizontal xgboost learning on tabular data with bagging collaboration.

3. Tutorial notebooks

Example	Summary
Intro to the FL Simulator	Shows how to use the FLARE Simulator to run a local simulation.
Hello FLARE API	Goes through the different commands of the FLARE API.
NVFLARE in POC Mode	Shows how to use POC mode.
Job CLI	Walks through the different commands of the Job CLI.

4. FL algorithms

Example	Summary
Simulated Federated Learning with CIFAR-10	This example includes instructions on running FedAvg, FedProx, FedOpt, and SCAFFOLD algorithms using NVFlare's FL simulator.
Real-world Federated Learning with CIFAR-10	Includes instructions on running FedAvg with streaming of TensorBoard metrics to the server during training and homomorphic encryption.

5. Traditional ML examples

Example	Framework	Notebooks	Summary
Federated Linear Model with Scikit-learn	scikit-learn	FL Model with Scikit-learn on HIGGS Dataset	Shows how to use the NVIDIA FLARE with scikit-learn, a widely used open-source machine learning library.
Federated K-Means Clustering with Scikit-learn	scikit-learn	Federated K-Means Clustering with Scikit-learn on Iris Dataset	NVIDIA FLARE with scikit-learn and k-Means.
Federated SVM with Scikit-learn	scikit-learn	Federated SVM with Scikit-learn on Breast Cancer Dataset	NVIDIA FLARE with scikit-learn and SVM.
Histogram-based FL for XGBoost	XGBoost	Histogram-based FL for XGBoost on HIGGS Dataset	Histogram-based algorithm for XGBoost
Tree-based Federated Learning for XGBoost	XGBoost	Tree-based FL for XGBoost on HIGGS Dataset	Tree-based algorithms includes bagging and cyclic approaches.
Federated Learning for Random Forest based on XGBoost	XGBoost	Federated Random Forest on HIGGS Dataset	Example of using NVIDIA FLARE with scikit-learn and Random Forest.
Federated Vertical XGBoost	XGBoost	Federated Vertical XGBoost	Example using Private Set Intersection and XGBoost on vertically split HIGGS data.

6. Medical Image Analysis

Example	Framework	Summary
NVFlare + MONAI integration	MONAI	For an example of using NVIDIA FLARE to train a 3D medical image analysis model using federated averaging (FedAvg) and MONAI Bundle, see here.
Federated Learning with Differential Privacy for BraTS18 segmentation	MONAI	Illustrates the use of differential privacy for training brain tumor segmentation models using federated learning.
Federated Learning for Prostate Segmentation from Multi-source Data	MONAI	Example of training a multi-institutional prostate segmentation model using FedAvg, FedProx, and Ditto.

7. Federated Statistics

Federated Statistics Overview discusses the overall federated statistics features.

Example	Notebooks	Summary
Federated Statistics for Images	Image Histograms	Example of gathering local image histogram to compute the global dataset histograms.
Federated Statistics for DataFrame	Data Frame Federated Statistics, Visualization	Example of gathering local statistics summary from Pandas DataFrame to compute the global dataset statistics.
Federated Hierarchical Statistics for DataFrame	Federated Hierarchical Statistics, Visualization	Example of generating federated hierarchical statistics for data that can be represented as Pandas DataFrame.

8. Federated Policies

Example	Summary
Federated Policies	Discuss the federated site policies for authorization, resource and data privacy management.
Custom Authentication	Demonstrate the custom authentication policy and secure mode.
Job-Level Authorization	Demonstrate the job-level authorization policy and secure mode.
KeyCloak Site Authentication Integration	Demonstrate KeyCloak integration for supporting site-specific authentication.

9. Experiment tracking

Example	Framework	Summary
Hello PyTorch with TensorBoard Streaming	PyTorch	Example building upon Hello PyTorch showcasing the TensorBoard streaming capability from the clients to the server.
FL Experiment Tracking with MLflow	PyTorch	Example integrating Hello PyTorch with MLflow streaming capability from clients to the server.
FL Experiment Tracking with Weights and Biases	PyTorch	Example integrating Hello PyTorch with Weights and Biases streaming capability from clients to the server.
MONAI FLARE Integration Experiment Tracking	MONAI	Example using FLARE and MONAI integration with experiment tracking streaming from clients to server.

10. NLP

Example	Summary
NLP-NER	Illustrates both BERT and GPT-2 models from Hugging Face (BERT-base-uncased, GPT-2) on a Named Entity Recognition (NER) task using the NCBI disease dataset.

11. Federated Learning Hub

Example	Framework	Summary
FL Hub	PyTorch/MONAI	Allow hierarchical interaction between several levels of nvflare FL systems, e.g. Tier-1 (hub) and Tier-2 (sub-systems).

12. Federated Large Language Model (LLM)

Example	Framework	Summary
Parameter Efficient Fine Turning	NeMo	Example utilizing NeMo's PEFT methods to adapt a LLM to a downstream task.
Prompt-Tuning Example	NeMo	Example for using FLARE with NeMo for prompt learning.
Supervised Fine Tuning (SFT)	NeMo	Example to fine-tune all parameters of a LLM on supervised data.
LLM Tuning via HuggingFace SFT Trainer	NeMo	Example for using FLARE with a HuggingFace trainer for LLM tuning tasks.

13. Graph Neural Network (GNN)

Example	Framework	Summary
Protein Classification	PyTorch Geometric	Example using GNNs for Protein Classification using PPI dataset using GraphSAGE.
Financial Transaction Classification	PyTorch Geometric	Example using GNNs for Financial Transaction Classification with Elliptic++ dataset using GraphSAGE.

14. Financial Applications

Example	Framework	Summary
Financial Application with Federated XGBoost Methods	XGBoost	Example using XGBoost in various ways to train a federated model to perform fraud detection with a finance dataset.