Skip to content

Commit

Permalink
docs: document the federated learning use case
Browse files Browse the repository at this point in the history
  • Loading branch information
ferrarimarco committed Jan 14, 2025
1 parent 9588c0b commit c85c950
Show file tree
Hide file tree
Showing 6 changed files with 629 additions and 43 deletions.
40 changes: 36 additions & 4 deletions docs/use-cases/federated-learning/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,38 @@
# Federated learning
# Federated learning on Google Cloud

For more information about the federated learning use case, see:
This repository contains a blueprint and a reference architecture to implement
and realize federated learning use cases on Google Cloud, such as:

- [Cross-silo and cross-device federated learning on Google Cloud](https://cloud.google.com/architecture/cross-silo-cross-device-federated-learning-google-cloud)
- [Federated learning on Google Cloud](/platforms/gke/base/use-cases/federated-learning/README.md)
- Cross-silo federated learning: where the participating members are
organizations or companies, and the number of members is usually small such as
within a hundred.
- Cross-device federated computation: is a type of federated computation where
the participating members are end user devices such as mobile phones and
vehicles. The number of members can reach up to a scale of millions or even
tens of millions.

This reference architecture is based on Google Kubernetes Engine (GKE) and
follows security best practices that help you secure your Google Cloud
environment.

This reference architecture configures separated, isolated runtime environments.
Each runtime environment gets:

- A dedicated Kubernetes namespace
- A dedicated GKE node pool

You can run potentially untrusted federated learning workloads in these runtime
environments, granting these workloads only the minimum permissions that they
need.

This reference architecture assumes that you are familiar with GKE and
Kubernetes.

For more information about federated learning, see
[Cross-silo and cross-device federated learning on Google Cloud](https://cloud.google.com/architecture/cross-silo-cross-device-federated-learning-google-cloud).

## Get started

For more information about deploying this reference architecture in your Google
Cloud environment, see
[Deploy the Federated learning reference architecture on Google Cloud](/platforms/gke/base/use-cases/federated-learning/README.md)
2 changes: 1 addition & 1 deletion platforms/gke/base/_shared_config/cluster_variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ variable "cluster_binary_authorization_evaluation_mode" {

variable "cluster_confidential_nodes_enabled" {
default = false
description = "Enable Confidential GKE Nodes for this node pool, to enforce encryption of data in-use"
description = "Enable Confidential GKE Nodes for this node pool, to enforce encryption of data in-use. When setting this to true, ensure that the machine types you configured for your node pools support Confidential GKE Nodes. Ref: https://cloud.google.com/kubernetes-engine/docs/how-to/confidential-gke-nodes"
type = bool
}

Expand Down
66 changes: 37 additions & 29 deletions platforms/gke/base/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,80 +2,88 @@

## Pull the source code

- Clone the repository and change directory to the guide directory
1. Open [Cloud Shell](https://cloud.google.com/shell).

```
git clone https://github.com/GoogleCloudPlatform/accelerated-platforms && \
cd accelerated-platforms
```
1. Clone the repository and change directory to the guide directory

- Set environment variables
```shell
git clone https://github.com/GoogleCloudPlatform/accelerated-platforms && \
cd accelerated-platforms
```

```
export ACP_REPO_DIR=$(pwd) && \
echo "export ACP_REPO_DIR=${ACP_REPO_DIR}" >> ${HOME}/.bashrc
```
1. Set environment variables

```
cd ${ACP_REPO_DIR}/platforms/gke/base && \
export ACP_PLATFORM_BASE_DIR=$(pwd) && \
echo "export ACP_PLATFORM_BASE_DIR=${ACP_PLATFORM_BASE_DIR}" >> ${HOME}/.bashrc
```
```shell
ACP_REPO_DIR="$(pwd)" && \
export ACP_REPO_DIR && \
echo "export ACP_REPO_DIR=${ACP_REPO_DIR}" >> ${HOME}/.bashrc
```

```
cd ${ACP_REPO_DIR}/platforms/gke/base/core && \
export ACP_PLATFORM_CORE_DIR=$(pwd) && \
echo "export ACP_PLATFORM_CORE_DIR=${ACP_PLATFORM_CORE_DIR}" >> ${HOME}/.bashrc
```
```shell
cd "${ACP_REPO_DIR}/platforms/gke/base" && \
ACP_PLATFORM_BASE_DIR="$(pwd)" && \
export ACP_PLATFORM_BASE_DIR && \
echo "export ACP_PLATFORM_BASE_DIR=${ACP_PLATFORM_BASE_DIR}" >> ${HOME}/.bashrc
```

## Configure
```shell
cd "${ACP_REPO_DIR}/platforms/gke/base/core" && \
ACP_PLATFORM_CORE_DIR="$(pwd)" && \
export ACP_PLATFORM_CORE_DIR && \
echo "export ACP_PLATFORM_CORE_DIR=${ACP_PLATFORM_CORE_DIR}" >> ${HOME}/.bashrc
```

## Configure the Core GKE Accelerated Platform

Terraform loads variables in the following order, with later sources taking precedence over earlier ones:

- Environment variables (`TF_VAR_<variable_name>`)
- Any `*.auto.tfvars` or files, processed in lexical order of their filenames.
- Any `-var` and `-var-file` options on the command line, in the order they are provided.

For more information about providing values for Terraform input variables, see
[Terraform input variables](https://developer.hashicorp.com/terraform/language/values/variables).

- Set the cluster project ID

```
```shell
export TF_VAR_cluster_project_id="<PROJECT_ID>"
```

**-- OR --**

```
```shell
vi ${ACP_PLATFORM_BASE_DIR}/_shared_config/cluster.auto.tfvars
```

```
```hcl
cluster_project_id = "<PROJECT_ID>"
```

- Set the Terraform project ID

```
```shell
export TF_VAR_terraform_project_id="<PROJECT_ID>"
```

**-- OR --**

```
```shell
vi ${ACP_PLATFORM_BASE_DIR}/_shared_config/terraform.auto.tfvars
```

```
```hcl
terraform_project_id = "<PROJECT_ID>"
```

## Deploy

```
```shell
${ACP_PLATFORM_CORE_DIR}/deploy.sh
```

## Teardown

```
```shell
${ACP_PLATFORM_CORE_DIR}/teardown.sh
```
Loading

0 comments on commit c85c950

Please sign in to comment.