diff --git a/docs/flare_overview.rst b/docs/flare_overview.rst index c2f6ecfb91..15eaafa8d3 100644 --- a/docs/flare_overview.rst +++ b/docs/flare_overview.rst @@ -26,7 +26,7 @@ Built for productivity FLARE is designed for maximum productivity, providing a range of tools to enhance user experience and research efficiency at different stages of the development process: - **FLARE Client API:** Enables users to transition seamlessly from ML/DL to FL with just a few lines of code changes. -- **Simulator CLI:** Allows users to simulate federated learning or computing jobs in multi-thread settings within a single computer, offering quick response and debugging. The same job can be deployed directly to production. +- **Simulator CLI:** Allows users to simulate federated learning or computing jobs in multi-process settings within a single computer, offering quick response and debugging. The same job can be deployed directly to production. - **POC CLI:** Facilitates the simulation of federated learning or computing jobs in multi-process settings within one computer. Different processes represent server, clients, and an admin console, providing users with a realistic sense of the federated network. It also allows users to simulate project deployment on a single host. - **Job CLI:** Permits users to create and submit jobs directly in POC or production environments. - **FLARE API:** Enables users to run jobs directly from Python code or notebooks. diff --git a/docs/getting_started.rst b/docs/getting_started.rst index 6f896b17b4..cee26e8f9e 100644 --- a/docs/getting_started.rst +++ b/docs/getting_started.rst @@ -66,6 +66,11 @@ Installation .. note:: The server and client versions of nvflare must match, we do not support cross-version compatibility. +Supported Operating Systems +--------------------------- +- Linux +- OSX (Note: some optional dependencies are not compatible, such as tenseal and openmined.psi) + Python Version -------------- @@ -120,7 +125,6 @@ You may find that the pip and setuptools versions in the venv need updating: (nvflare-env) $ python3 -m pip install -U pip (nvflare-env) $ python3 -m pip install -U setuptools - Install Stable Release ---------------------- @@ -130,6 +134,11 @@ Stable releases are available on `NVIDIA FLARE PyPI ` for modules and components with optional dependencies. .. _containerized_deployment: @@ -213,7 +222,7 @@ Production mode is secure with TLS certificates - depending the choice the deplo - HA or non-HA - Local or remote - - On-premise or on cloud + - On-premise or on cloud (See :ref:`cloud_deployment`) Using non-HA, secure, local mode (all clients and server running on the same host), production mode is very similar to POC mode except it is secure. diff --git a/docs/index.rst b/docs/index.rst index 3e73c525fd..1c8527e62c 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -48,18 +48,21 @@ and simulation to real-world production deployment. Some of the key components - **Management tools** for secure provisioning and deployment, orchestration, and management - **Specification-based API** for extensibility -Learn more in the :ref:`FLARE Overview `, :ref:`What's New `, and the -:ref:`User Guide ` and :ref:`Programming Guide `. +Learn more about FLARE features in the :ref:`FLARE Overview ` and :ref:`What's New `. Getting Started =============== For first-time users and FL researchers, FLARE provides the :ref:`FL Simulator ` that allows you to build, test, and deploy applications locally. The :ref:`Getting Started ` guide covers installation and walks through an example application using the FL Simulator. +Additional examples can be found at the :ref:`Examples Applications `, which showcase different federated learning workflows and algorithms on various machine learning and deep learning tasks. +FLARE for Users +=============== +If you want to learn how to interact with the FLARE system, please refer to the :ref:`User Guide `. When you are ready to for a secure, distributed deployment, the :ref:`Real World Federated Learning ` section covers the tools and process required to deploy and operate a secure, real-world FLARE project. FLARE for Developers ==================== -When you're ready to build your own application, the :ref:`Programming Best Practices `, :ref:`FAQ`, and -:ref:`Programming Guide ` give an in depth look at the FLARE platform and APIs. +When you're ready to build your own application, the :ref:`Programming Guide `, :ref:`Programming Best Practices `, :ref:`FAQ`, and :ref:`API Reference ` +give an in depth look at the FLARE platform and APIs. diff --git a/docs/programming_guide/experiment_tracking.rst b/docs/programming_guide/experiment_tracking.rst index 06a274ee81..c0ccbbe939 100644 --- a/docs/programming_guide/experiment_tracking.rst +++ b/docs/programming_guide/experiment_tracking.rst @@ -37,6 +37,13 @@ provided examples, the Receiver is on the FL server, but it could also be on the - Server-side experiment tracking also can organize different clients' results into different experiment runs so they can be easily compared side-by-side. +.. note:: + + This page covers experiment tracking using :class:`LogWriters `, + which are configured and used with :ref:`executor` or :ref:`model_learner` on the FLARE-side code. + However if using the Client API, please refer to :ref:`client_api` and :ref:`nvflare.client.tracking` for adding experiment tracking to your custom training code. + + ************************************** Tools, Sender, LogWriter and Receivers ************************************** @@ -60,9 +67,9 @@ where the actual experiment logs are recorded. The components that receive these logs are called Receivers based on :class:`AnalyticsReceiver `. The receiver component leverages the experiment tracking tool and records the logs during the experiment run. -In a normal setting, we would have pairs of sender and receivers, such as: +In a normal setting, we would have pairs of sender and receivers, with some provided implementations in :mod:`nvflare.app_opt.tracking`: - - TBWriter <-> TBReceiver + - TBWriter <-> TBAnalyticsReceiver - MLflowWriter <-> MLflowReceiver - WandBWriter <-> WandBReceiver @@ -94,13 +101,11 @@ There are three things to consider for developing a custom experiment tracking t Data Type ========= -Currently, the supported data types are metrics, params, and text. If you require other data types, may sure you add -the type to :class:`AnalyticsDataType `. +Currently, the supported data types are listed in :class:`AnalyticsDataType `, and other data types can be added as needed. Writer ====== - -Implement LogWriter interface with the API syntax. For each tool, we mimic the API syntax of the underlying tool, +Implement :class:`LogWriter ` interface with the API syntax. For each tool, we mimic the API syntax of the underlying tool, so users can use what they are familiar with without learning a new API. For example, for Tensorboard, TBWriter uses add_scalar() and add_scalars(); for MLflow, the syntax is log_metric(), log_metrics(), log_parameter(), and log_parameters(); for W&B, the writer just has log(). @@ -109,7 +114,7 @@ The data collected with these calls will all send to the AnalyticsSender to deli Receiver ======== -Implement AnalyticsReceiver interface and determine how to represent different sites' logs. In all three implementations +Implement :class:`AnalyticsReceiver ` interface and determine how to represent different sites' logs. In all three implementations (Tensorboard, MLflow, WandB), each site's log is represented as one run. Depending on the individual tool, the implementation can be different. For example, for both Tensorboard and MLflow, we create different runs for each client and map to the site name. In the WandB implementation, we have to leverage multiprocess and let each run in a different process. @@ -121,13 +126,19 @@ Examples Overview The :github_nvflare_link:`experiment tracking examples ` illustrate how to leverage different writers and receivers. All examples are based upon the hello-pt example. +TensorBoard +=========== The example in the "tensorboard" directory shows how to use the Tensorboard Tracking Tool (for both the sender and receiver). See :ref:`tensorboard_streaming` for details. +MLflow +====== Under the "mlflow" directory, the "hello-pt-mlflow" job shows how to use MLflow for tracking with both the MLflow sender and receiver. The "hello-pt-tb-mlflow" job shows how to use the Tensorboard Sender, while the receiver is MLflow. See :ref:`experiment_tracking_mlflow` for details. +Weights & Biases +================ Under the :github_nvflare_link:`wandb ` directory, the "hello-pt-wandb" job shows how to use Weights and Biases for experiment tracking with the WandBWriter and WandBReceiver to log metrics. diff --git a/docs/user_guide/nvflare_cli/fl_simulator.rst b/docs/user_guide/nvflare_cli/fl_simulator.rst index b0ffe8fa10..f52fd47078 100644 --- a/docs/user_guide/nvflare_cli/fl_simulator.rst +++ b/docs/user_guide/nvflare_cli/fl_simulator.rst @@ -49,7 +49,7 @@ Command examples Run a single NVFlare app ======================== -This command will run the same ``hello-numpy-sag`` app on the server and 8 clients using 1 thread. The client names will be site-1, site-2, ... , site-8: +This command will run the same ``hello-numpy-sag`` app on the server and 8 clients using 1 process. The client names will be site-1, site-2, ... , site-8: .. code-block:: python @@ -829,22 +829,29 @@ application run. status = run_simulator(args) sys.exit(status) -**************************** -Threads, Clients, and Events -**************************** +****************************** +Processes, Clients, and Events +****************************** -Specifying threads -================== -The simulator ``-t`` option provides the ability to specify how many threads to run the simulator with. +Specifying number of processes +============================== +The simulator ``-t`` option provides the ability to specify how many processes to run the simulator with. -When you run the simulator with ``-t 1``, there is only one client active and running at a time, and the clients will be running in -turn. This is to enable the simulation of large number of clients using a single machine with limited resources. +.. note:: + + The ``-t`` and ``--threads`` option for simulator was originally due to clients running in separate threads. + However each client now actually runs in a separate process. This distinction will not affect the user experience. + +- N = number of clients (``-n``) +- T = number of processes (``-t``) -Note that if you have fewer threads than the number of clients, ClientRunner/learner object will go thorugh setup and -teardown in every round. +When running the simulator with fewer processes than clients (T < N) +the simulator will need to swap-in/out the clients for the processes, resulting in some of the clients running sequentially as processes are available. +This also will cause the ClientRunner/learner objects to go through setup and teardown in every round. +Using T < N is only needed when trying to simulate of large number of clients using a single machine with limited resources. -With ``-t=num_client``, the simulator will run the number of clients in separate threads at the same time. Each -client will always be running in memory with no swap_in / swap_out, but it will require more resources available. +In most cases, run the simulator with the same number of processes as clients (T = N). The simulator will run the number of clients in separate processes at the same time. Each +client will always be running in memory with no swap-in/out, but it will require more resources available. For the dataset / tensorboard initialization, you could make use of EventType.SWAP_IN and EventType.SWAP_OUT in the application. diff --git a/examples/advanced/experiment-tracking/wandb/README.md b/examples/advanced/experiment-tracking/wandb/README.md index 27d06fa243..aaadc13b55 100644 --- a/examples/advanced/experiment-tracking/wandb/README.md +++ b/examples/advanced/experiment-tracking/wandb/README.md @@ -26,7 +26,9 @@ export PYTHONPATH=${PWD}/.. Import the W&B Python SDK and log in: ``` -wandb.login() +python3 +>>> import wandb +>>> wandb.login() ``` Provide your API key when prompted. diff --git a/examples/hello-world/step-by-step/README.md b/examples/hello-world/step-by-step/README.md index aba2957283..59a9c487b0 100644 --- a/examples/hello-world/step-by-step/README.md +++ b/examples/hello-world/step-by-step/README.md @@ -7,7 +7,7 @@ To run the notebooks in each example, please make sure you first set up a virtua These step-by-step example series are aimed to help users quickly get started and learn about FLARE. For consistency, each example in the series uses the same dataset- CIFAR10 for image data and the HIGGS dataset for tabular data. -The examples will build upon previous ones to showcase different features, workflows, or APIs, allowing users to gain a comprehensive understanding of FLARE functionalities. See the README in each directory for more details about each series. +The examples will build upon previous ones to showcase different features, workflows, or APIs, allowing users to gain a comprehensive understanding of FLARE functionalities (Note: each example is self-contained, so going through them in order is not required, but recommended). See the README in each directory for more details about each series. ## Common Questions diff --git a/examples/hello-world/step-by-step/cifar10/cse/cse.ipynb b/examples/hello-world/step-by-step/cifar10/cse/cse.ipynb index 176fc875f7..a25c7ffd32 100644 --- a/examples/hello-world/step-by-step/cifar10/cse/cse.ipynb +++ b/examples/hello-world/step-by-step/cifar10/cse/cse.ipynb @@ -180,9 +180,9 @@ "id": "48271064", "metadata": {}, "source": [ - "For additional resources, see other examples for SAG with CSE using the [ModelLearner](../sag_model_learner/sag_model_learner.ipynb), [Executor](../sag_executor/sag_executor.ipynb), and [Hello-Numpy](https://github.com/NVIDIA/NVFlare/tree/main/examples/hello-world/hello-numpy-cross-val).\n", + "For additional resources, see other examples for SAG with CSE using the [ModelLearner](../sag_model_learner/sag_model_learner.ipynb) and [Executor](../sag_executor/sag_executor.ipynb). [Hello-Numpy](https://github.com/NVIDIA/NVFlare/tree/main/examples/hello-world/hello-numpy-cross-val) also demonstrates how to run cross-site evaluation using the previous training results.\n", "\n", - "Also the ability to run Cross-site Evaluation without having to re-run training will be added in the near future." + "Next we will look at the [cyclic](../cyclic/cyclic.ipynb) example, which shows the cyclic workflow for the Cyclic Weight Transfer algorithm." ] }, { diff --git a/examples/hello-world/step-by-step/cifar10/cyclic/cyclic.ipynb b/examples/hello-world/step-by-step/cifar10/cyclic/cyclic.ipynb index 153a4b6468..5dbf03dd2c 100644 --- a/examples/hello-world/step-by-step/cifar10/cyclic/cyclic.ipynb +++ b/examples/hello-world/step-by-step/cifar10/cyclic/cyclic.ipynb @@ -140,7 +140,10 @@ "id": "48271064", "metadata": {}, "source": [ - "As an additional resource, also see the [hello-cyclic](../../../../hello-world/hello-cyclic/README.md) for a Tensorflow Executor implementation using the MNIST dataset." + "As an additional resource, also see the [hello-cyclic](../../../../hello-world/hello-cyclic/README.md) for a Tensorflow Executor implementation using the MNIST dataset.\n", + "\n", + "While this example focused on the server-controlled cyclic workflow, now we will introduce the idea of client-controlled workflows.\n", + "The next [cyclic_ccwf](../cyclic_ccwf/cyclic_ccwf.ipynb) example is a client-controlled version of the cyclic workflow." ] }, { diff --git a/examples/hello-world/step-by-step/cifar10/cyclic_ccwf/cyclic_ccwf.ipynb b/examples/hello-world/step-by-step/cifar10/cyclic_ccwf/cyclic_ccwf.ipynb index f90ce77d13..778943998e 100644 --- a/examples/hello-world/step-by-step/cifar10/cyclic_ccwf/cyclic_ccwf.ipynb +++ b/examples/hello-world/step-by-step/cifar10/cyclic_ccwf/cyclic_ccwf.ipynb @@ -145,7 +145,9 @@ "cell_type": "markdown", "id": "9bef3134", "metadata": {}, - "source": [] + "source": [ + "Lastly, we have the [swarm](../swarm/swarm.ipynb) example, which covers swarm learning and client-controlled cross-site evaluation workflows." + ] } ], "metadata": { diff --git a/examples/hello-world/step-by-step/cifar10/sag/sag.ipynb b/examples/hello-world/step-by-step/cifar10/sag/sag.ipynb index 3ef045687f..f29f41249e 100644 --- a/examples/hello-world/step-by-step/cifar10/sag/sag.ipynb +++ b/examples/hello-world/step-by-step/cifar10/sag/sag.ipynb @@ -262,7 +262,10 @@ "id": "b055bde7-432d-4e6b-9163-b5ab7ede7b73", "metadata": {}, "source": [ - "The job should be running in the simulator mode. We are done with the training. " + "The job should be running in the simulator mode. We are done with the training. \n", + "\n", + "The next 5 examples will use the same ScatterAndGather workflow, but will demonstrate different execution APIs and feature.\n", + "In the next example [sag_deploy_map](../sag_deploy_map/sag_deploy_map.ipynb), we will learn about the deploy_map configuration for deployment of apps to different sites." ] } ], diff --git a/examples/hello-world/step-by-step/cifar10/sag_deploy_map/sag_deploy_map.ipynb b/examples/hello-world/step-by-step/cifar10/sag_deploy_map/sag_deploy_map.ipynb index 91d4beef0d..b59dbad293 100644 --- a/examples/hello-world/step-by-step/cifar10/sag_deploy_map/sag_deploy_map.ipynb +++ b/examples/hello-world/step-by-step/cifar10/sag_deploy_map/sag_deploy_map.ipynb @@ -261,7 +261,10 @@ "id": "0af8036f-1f94-426d-8eb7-6e8b9be70a7e", "metadata": {}, "source": [ - "The job should be running in the simulator mode. We are done with the training. " + "The job should be running in the simulator mode. We are done with the training. \n", + "\n", + "In the next example [sag_model_learner](../sag_model_learner/sag_model_learner.ipynb), we will illustrate how to use the Model Learner API instead of the Client API,\n", + "and highlight why and when to use it." ] } ], diff --git a/examples/hello-world/step-by-step/cifar10/sag_executor/sag_executor.ipynb b/examples/hello-world/step-by-step/cifar10/sag_executor/sag_executor.ipynb index aed5f6ba74..e71dd87cbf 100644 --- a/examples/hello-world/step-by-step/cifar10/sag_executor/sag_executor.ipynb +++ b/examples/hello-world/step-by-step/cifar10/sag_executor/sag_executor.ipynb @@ -222,7 +222,12 @@ "id": "48271064", "metadata": {}, "source": [ - "For additional resources, take a look at the various other executors with different use cases in the app_common, app_opt, and examples folder." + "For additional resources, take a look at the various other executors with different use cases in the app_common, app_opt, and examples folder.\n", + "\n", + "In the previous examples we have finished covering each of Execution API types: the Client API, Model Learner, and Executor.\n", + "Now we will be using the Client API in future examples to highlight other features and workflows.\n", + "\n", + "Next we have the [sag_mlflow](../sag_mlflow/sag_mlflow.ipynb) example, which shows how to enable MLflow experiment tracking logs." ] }, { diff --git a/examples/hello-world/step-by-step/cifar10/sag_he/sag_he.ipynb b/examples/hello-world/step-by-step/cifar10/sag_he/sag_he.ipynb index 12936dd208..c80f7d37af 100644 --- a/examples/hello-world/step-by-step/cifar10/sag_he/sag_he.ipynb +++ b/examples/hello-world/step-by-step/cifar10/sag_he/sag_he.ipynb @@ -197,7 +197,10 @@ "id": "b19da336", "metadata": {}, "source": [ - "As an additional resource, see the [CIFAR10 Real World Example](https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/cifar10/cifar10-real-world) for creating a secure workspace for HE using provisioning instead of POC mode." + "As an additional resource, see the [CIFAR10 Real World Example](https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/cifar10/cifar10-real-world) for creating a secure workspace for HE using provisioning instead of POC mode.\n", + "\n", + "Now we will begin to take a look at other workflows besides ScatterAndGather.\n", + "First we have the [cse](../cse/cse.ipynb) example, which shows the server-controlled cross-site evaluation workflow." ] } ], diff --git a/examples/hello-world/step-by-step/cifar10/sag_mlflow/sag_mlflow.ipynb b/examples/hello-world/step-by-step/cifar10/sag_mlflow/sag_mlflow.ipynb index cb39afaf61..fa295c0e3b 100644 --- a/examples/hello-world/step-by-step/cifar10/sag_mlflow/sag_mlflow.ipynb +++ b/examples/hello-world/step-by-step/cifar10/sag_mlflow/sag_mlflow.ipynb @@ -183,12 +183,12 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "id": "e69c9ed2-359a-4f97-820f-25e9323a4e92", + "cell_type": "markdown", + "id": "58037d1e", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "Next we will look at the [sag_he](../sag_he/sag_he.ipynb) example, which demonstrates how to enable homomorphic encryption using the POC -he mode." + ] } ], "metadata": { diff --git a/examples/hello-world/step-by-step/cifar10/sag_model_learner/sag_model_learner.ipynb b/examples/hello-world/step-by-step/cifar10/sag_model_learner/sag_model_learner.ipynb index 261275427c..ae8aec8a6c 100644 --- a/examples/hello-world/step-by-step/cifar10/sag_model_learner/sag_model_learner.ipynb +++ b/examples/hello-world/step-by-step/cifar10/sag_model_learner/sag_model_learner.ipynb @@ -204,7 +204,9 @@ "id": "48271064", "metadata": {}, "source": [ - "As an additional resource, also see the [CIFAR10 examples](../../../../advanced/cifar10/README.md) for a comprehensive implementation of a PyTorch ModelLearner." + "As an additional resource, also see the [CIFAR10 examples](../../../../advanced/cifar10/README.md) for a comprehensive implementation of a PyTorch ModelLearner.\n", + "\n", + "In the next example [sag_executor](../sag_executor/sag_executor.ipynb), we will illustrate how to use the Executor API for more specific use cases." ] }, { diff --git a/examples/hello-world/step-by-step/cifar10/stats/image_stats.ipynb b/examples/hello-world/step-by-step/cifar10/stats/image_stats.ipynb index 0ba0d4e5fd..ede5f7b1e1 100644 --- a/examples/hello-world/step-by-step/cifar10/stats/image_stats.ipynb +++ b/examples/hello-world/step-by-step/cifar10/stats/image_stats.ipynb @@ -664,9 +664,8 @@ "\n", "If you would like to see another example of federated statistics calculations and configurations, please checkout [federated_statistics](https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/federated-statistics) and [fed_stats with spleen_ct_segmentation](https://github.com/NVIDIA/NVFlare/tree/main/integration/monai/examples/spleen_ct_segmentation_sim)\n", "\n", - "Let's move on to the next example and see how can we train the image classifier using pytorch with CIFAR10 data.\n", - "\n", - "\n" + "Let's move on to the next examples and see how can we train the image classifier using pytorch with CIFAR10 data.\n", + "First we will look at the [sag](../sag/sag.ipynb) example, which illustrates how to use the ScatterAndGather workflow for FedAvg with the Client API.\n" ] } ], diff --git a/examples/hello-world/step-by-step/cifar10/swarm/swarm.ipynb b/examples/hello-world/step-by-step/cifar10/swarm/swarm.ipynb index af5ba361b3..bc66e200f2 100644 --- a/examples/hello-world/step-by-step/cifar10/swarm/swarm.ipynb +++ b/examples/hello-world/step-by-step/cifar10/swarm/swarm.ipynb @@ -154,7 +154,10 @@ "id": "48271064", "metadata": {}, "source": [ - "As an additional resource, also see the [Swarm Learning Example](../../../../advanced/swarm_learning/README.md) which utilizes the CIFAR10 ModelLearner instead of the Client API." + "As an additional resource, also see the [Swarm Learning Example](../../../../advanced/swarm_learning/README.md) which utilizes the CIFAR10 ModelLearner instead of the Client API.\n", + "\n", + "Congratulations! You have completed the CIFAR10 step-by-step example series.\n", + "Next take a look at the [higgs](../../higgs/README.md) example series for how to use machine learning methods for federated learning on tabular data." ] }, { diff --git a/examples/hello-world/step-by-step/higgs/README.md b/examples/hello-world/step-by-step/higgs/README.md index 877a0fb118..9f344b43d3 100644 --- a/examples/hello-world/step-by-step/higgs/README.md +++ b/examples/hello-world/step-by-step/higgs/README.md @@ -1,7 +1,8 @@ # Training traditional ML classifiers with HIGGS dataset -The [HIGGS dataset](https://archive.ics.uci.edu/dataset/280/higgs) contains 11 million instances, each with 28 attributes, for binary classification to predict whether an event corresponds to the decayment of a Higgs boson or not. (Please note that the [UCI's website](https://archive.ics.uci.edu/dataset/280/higgs) may experience occasional downtime) +The [HIGGS dataset](https://archive.ics.uci.edu/dataset/280/higgs) contains 11 million instances, each with 28 attributes, for binary classification to predict whether an event corresponds to the decayment of a Higgs boson or not. Follow the [prepare_data.ipynb](prepare_data.ipynb) notebook to download the HIGGS dataset and prepare the data splits. +(Please note that the [UCI's website](https://archive.ics.uci.edu/dataset/280/higgs) may experience occasional downtime) The first 21 features (columns 2-22) are kinematic properties measured by the particle detectors in the accelerator. The data has been produced using Monte Carlo simulations. The first 21 features are kinematic properties measured by the particle detectors in the accelerator. The last 7 features are functions of the first 21 features; these are high-level features derived by physicists to help discriminate between the two classes. diff --git a/examples/hello-world/step-by-step/higgs/sklearn-kmeans/sklearn_kmeans.ipynb b/examples/hello-world/step-by-step/higgs/sklearn-kmeans/sklearn_kmeans.ipynb index 961ab69c66..1989af4eaa 100644 --- a/examples/hello-world/step-by-step/higgs/sklearn-kmeans/sklearn_kmeans.ipynb +++ b/examples/hello-world/step-by-step/higgs/sklearn-kmeans/sklearn_kmeans.ipynb @@ -452,7 +452,10 @@ "HIGGS dataset is challenging for unsupervised clustering, as we can observe from the result. As shown by the local training with same number of iterations, the score is `model homogeneity_score: 0.0049`. As compared with the FL score of `0.0068`, FL in this case still provides some benefit from the collaborative learning.\n", "\n", "## We are done !\n", - "Congratulations! you have just completed the federated k-Means clustering for tabular data. " + "Congratulations! you have just completed the federated k-Means clustering for tabular data. \n", + "\n", + "Now we will move on from scikit-learn and take a look at how to use federated XGBoost.\n", + "In the next example [xgboost](../xgboost/xgboost_horizontal.ipynb), we will show a federated horizontal xgboost learning with bagging collaboration." ] }, { diff --git a/examples/hello-world/step-by-step/higgs/sklearn-linear/sklearn_linear.ipynb b/examples/hello-world/step-by-step/higgs/sklearn-linear/sklearn_linear.ipynb index 6e653a86bc..462a7448bc 100644 --- a/examples/hello-world/step-by-step/higgs/sklearn-linear/sklearn_linear.ipynb +++ b/examples/hello-world/step-by-step/higgs/sklearn-linear/sklearn_linear.ipynb @@ -454,12 +454,14 @@ "id": "ea7bbacc-b059-4f82-9785-2b22bf840ef9", "metadata": {}, "source": [ - "In this experiment, all three clients have relatively large amount data wiht homogeneous distribution, we would expect the three numbers align within reasonable variation range. \n", + "In this experiment, all three clients have relatively large amount data with homogeneous distribution, we would expect the three numbers align within reasonable variation range. \n", "\n", "The final result for iterative learning is `ending model AUC: 0.6352`, and one-shot learning is `local model AUC: 0.6355`, as compared with FL's `local model AUC: 0.6351`, the numbers do align.\n", "\n", "## We are done !\n", - "Congratulations! you have just completed the federated linear model for tabular data. " + "Congratulations! you have just completed the federated linear model for tabular data. \n", + "\n", + "In the next example [sklearn-svm](../sklearn-svm/sklearn_svm.ipynb), we will demonstrate training a federated SVM model." ] }, { diff --git a/examples/hello-world/step-by-step/higgs/sklearn-svm/sklearn_svm.ipynb b/examples/hello-world/step-by-step/higgs/sklearn-svm/sklearn_svm.ipynb index 29a85b8c44..850cc955c4 100644 --- a/examples/hello-world/step-by-step/higgs/sklearn-svm/sklearn_svm.ipynb +++ b/examples/hello-world/step-by-step/higgs/sklearn-svm/sklearn_svm.ipynb @@ -431,7 +431,9 @@ "The final result for local SVM learning is `model AUC: 0.6217`, as compared with FL's `model AUC: 0.6403`, this confirms our expectation.\n", "\n", "## We are done !\n", - "Congratulations! you have just completed the federated SVM model for tabular data. " + "Congratulations! you have just completed the federated SVM model for tabular data. \n", + "\n", + "In the next example [sklearn-kmeans](../sklearn-kmeans/sklearn_kmeans.ipynb), we will illustrate a federated k-Means clustering." ] }, { diff --git a/examples/hello-world/step-by-step/higgs/stats/tabular_stats.ipynb b/examples/hello-world/step-by-step/higgs/stats/tabular_stats.ipynb index 8946a902ae..455309941c 100644 --- a/examples/hello-world/step-by-step/higgs/stats/tabular_stats.ipynb +++ b/examples/hello-world/step-by-step/higgs/stats/tabular_stats.ipynb @@ -293,7 +293,7 @@ "source": [ "## Create Federated Statistics Job\n", "\n", - "We are going to use NVFLARE job cli to create job. For detailed instructions on Job CLI, please follow the [job cli tutorial](https://github.com/NVIDIA/NVFlare/blob/main/examples/tutorials/job_cli.ipynb)\n", + "We are going to use NVFLARE job cli to create a job. For detailed instructions on Job CLI, please follow the [job cli tutorial](https://github.com/NVIDIA/NVFlare/blob/main/examples/tutorials/job_cli.ipynb)\n", "\n", "Let's check the available job templates, we are going to use one of the existing job templates and modify it to fit our needs. The job template is nothing but server and client-side job configurations." ] @@ -607,7 +607,10 @@ "## We are done !\n", "Congratulations! you have just completed the federated stats calulation for tabular data. \n", "\n", - "If you would like to see a detailed discussion regarding privacy filtering, please checkout the example in [federated statistics](https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/federated-statistics) examples." + "If you would like to see a detailed discussion regarding privacy filtering, please checkout the example in [federated statistics](https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/federated-statistics) examples.\n", + "\n", + "Let's move on to the next examples and see how can we use scikit-learn to train federated models on tabular data.\n", + "First we will look at the [sklearn-linear](../sklearn-linear/sklearn_linear.ipynb) example, which illustrates how to train a federated linear model (logistic regression on binary classification)." ] }, { diff --git a/examples/hello-world/step-by-step/higgs/xgboost/xgboost_horizontal.ipynb b/examples/hello-world/step-by-step/higgs/xgboost/xgboost_horizontal.ipynb index 6b1702d5ca..54b35705ab 100644 --- a/examples/hello-world/step-by-step/higgs/xgboost/xgboost_horizontal.ipynb +++ b/examples/hello-world/step-by-step/higgs/xgboost/xgboost_horizontal.ipynb @@ -481,7 +481,11 @@ "Both oneshot and iterative training schemes yield idential results of `local model AUC: 0.81928`. As compared with FL's `global model AUC: 0.82085`, we can notice FL gives some benefits, even under homogeneous data distribution across clients.\n", "\n", "## We are done !\n", - "Congratulations! you have just completed the federated xgboost model for tabular data. " + "Congratulations! you have just completed the federated xgboost model for tabular data. \n", + "\n", + "You have now completed the HIGGS step-by-step example series.\n", + "Next either take a look at the [cifar10](../../cifar10/README.md) example series for how to train an image classifier with PyTorch, or explore the\n", + "[examples/advanced](../../../../advanced/README.md) directory for more in-depth examples." ] }, { diff --git a/examples/tutorials/job_cli.ipynb b/examples/tutorials/job_cli.ipynb index d57d855f90..858c199069 100644 --- a/examples/tutorials/job_cli.ipynb +++ b/examples/tutorials/job_cli.ipynb @@ -15,7 +15,9 @@ "tags": [] }, "source": [ - "In this notebook, we will go through the different commands of the Job CLI to show the syntax and usage of each.\n" + "In this notebook, we will go through the different commands of the Job CLI to show the syntax and usage of each.\n", + "Refer to the [Job CLI Documentation](https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/job_cli.html) for more details.\n", + "\n" ] }, { diff --git a/integration/monai/README.md b/integration/monai/README.md index e8ef9a2be2..f1803fba72 100644 --- a/integration/monai/README.md +++ b/integration/monai/README.md @@ -9,10 +9,6 @@ Add `ClientAlgoExecutor` class to allow using MONAI's `ClientAlgo` class in fede Allow the use of bundles from the MONAI [model zoo](https://github.com/Project-MONAI/model-zoo) or custom configurations with NVFlare. -### Non-goals: - -n/a - ## Background MONAI allows the definition of AI models using the "[bundle](https://docs.monai.io/en/latest/bundle.html)" concept. It allows for easy experimentation and sharing of models that have been developed using MONAI. diff --git a/integration/nemo/examples/peft/README.md b/integration/nemo/examples/peft/README.md index bab36e0487..762173e230 100644 --- a/integration/nemo/examples/peft/README.md +++ b/integration/nemo/examples/peft/README.md @@ -25,6 +25,7 @@ docker run --runtime=nvidia -it --rm --shm-size=16g -p 8888:8888 -p 6006:6006 -- For easy experimentation with NeMo, install NVFlare and mount the code inside the [nemo_nvflare](./nemo_nvflare) folder. ``` +cd nemo_nvflare pip install nvflare~=2.4.0rc7 export PYTHONPATH=${PYTHONPATH}:/workspace ``` @@ -34,6 +35,7 @@ export PYTHONPATH=${PYTHONPATH}:/workspace We use [JupyterLab](https://jupyterlab.readthedocs.io) for this example. To start JupyterLab, run ``` +cd /workspace jupyter lab . ``` and open [peft.ipynb](./peft.ipynb). diff --git a/integration/nemo/examples/prompt_learning/README.md b/integration/nemo/examples/prompt_learning/README.md index c25457496e..b7e1049045 100644 --- a/integration/nemo/examples/prompt_learning/README.md +++ b/integration/nemo/examples/prompt_learning/README.md @@ -25,6 +25,7 @@ docker run --runtime=nvidia -it --rm --shm-size=16g -p 8888:8888 -p 6006:6006 -- For easy experimentation with NeMo, install NVFlare and mount the code inside the [nemo_nvflare](./nemo_nvflare) folder. ``` +cd nemo_nvflare pip install nvflare~=2.4.0rc7 export PYTHONPATH=${PYTHONPATH}:/workspace ``` @@ -34,6 +35,7 @@ export PYTHONPATH=${PYTHONPATH}:/workspace We use [JupyterLab](https://jupyterlab.readthedocs.io) for this example. To start JupyterLab, run ``` +cd /workspace jupyter lab . ``` and open [prompt_learning.ipynb](./prompt_learning.ipynb). diff --git a/integration/nemo/examples/prompt_learning/prompt_learning.ipynb b/integration/nemo/examples/prompt_learning/prompt_learning.ipynb index 4d76fcd860..66f80a2dc9 100644 --- a/integration/nemo/examples/prompt_learning/prompt_learning.ipynb +++ b/integration/nemo/examples/prompt_learning/prompt_learning.ipynb @@ -14,7 +14,13 @@ "The prompt learning technique shown in the example is [p-tuning](https://arxiv.org/abs/2103.10385), which adds a small prompt encoder network to the LLM\n", "to produce virtual token embeddings that guide the model toward the desired output of the downstream task.\n", "\n", - "For more details on how to change hyperparameters for prompt learning in NeMo, see this [tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Multitask_Prompt_and_PTuning.ipynb) which is also the basis for this NVFlare tutorial." + "For more details on how to change hyperparameters for prompt learning in NeMo, see this [tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Multitask_Prompt_and_PTuning.ipynb) which is also the basis for this NVFlare tutorial.\n", + "\n", + "\n", + "\n", + "In our federated implementation, the LLM parameters stay fixed. Prompt encoder parameters are trained/updated and averaged on the server.\n", + "\n", + "" ] }, { diff --git a/integration/nemo/examples/supervised_fine_tuning/README.md b/integration/nemo/examples/supervised_fine_tuning/README.md index 610814d3d8..f34a4a8539 100644 --- a/integration/nemo/examples/supervised_fine_tuning/README.md +++ b/integration/nemo/examples/supervised_fine_tuning/README.md @@ -24,6 +24,7 @@ docker run --runtime=nvidia -it --rm --shm-size=16g -p 8888:8888 -p 6006:6006 -- For easy experimentation with NeMo, install NVFlare and mount the code inside the [nemo_nvflare](./nemo_nvflare) folder. ``` +cd nemo_nvflare pip install nvflare~=2.4.0rc7 export PYTHONPATH=${PYTHONPATH}:/workspace ``` diff --git a/job_templates/readme.md b/job_templates/readme.md index 6963c6165f..245ce3c198 100644 --- a/job_templates/readme.md +++ b/job_templates/readme.md @@ -13,11 +13,13 @@ Each job template contains the following informations * information card: info.md for display purpose * information config: used by program +Refer to the [Job CLI Documentation](https://nvflare.readthedocs.io/en/main/user_guide/nvflare_cli/job_cli.html) for details on how to use the Job Templates with the Job CLI. + ## Configuration format Configurations are written in HOCON (human optimized object Notation). As a variant of JSON, .conf can also use json format. The pyhocon format allows for comments, and you can remove many of the double quotes as well as replace ":" with "=" to make the configurations look cleaner. -You can find details in [pyhoconb: HOCON Parser for python](https://github.com/chimpler/pyhocon). +You can find details in [pyhocon: HOCON Parser for python](https://github.com/chimpler/pyhocon). ## List of Job Templates