Autosync kubeflow.org groups using GitOps (kubeflow#367)

* This is a replacement for shelling out to GAM.

* Import all the existing groups to YAML files.
  * Get rid of the old text files and GAM code

* Only import members for groups that already defined in GitHub
  * We don't want to be the ones putting people's emails into GitHub
  * As part of the migration we will ask members of groups not
    currently being auto-sync'd to open up PRs to add themselves
  * We will then enable AutoSync for all groups.
  * We import stubs for all Groups but disable auto-sync; this way
    we can start asking people to submit PRs to join the group before
    we turn on the sync.

* Support reading the OAuth2 credential from GCS
* Add a Makefile to make it easy to run the commands.

* Use a content hash to detect when files have changed and determine when to resync.

* Also periodically force a resync so if there is any drift due to changes
  made outside the system they are reverted.

Related to kubeflow/internal-acls#365 auto sync the Google groups

Co-authored-by: Jeremy Lewi <jlewi@google.com>

.gitignore

@@ -0,0 +1,2 @@
+**/.idea
+**/.build

google_groups/Dockerfile

@@ -0,0 +1,26 @@
+# Build the manager binary
+FROM golang:1.13 as builder
+
+WORKDIR /workspace
+# Copy the Go Modules manifests
+COPY go.mod go.mod
+COPY go.sum go.sum
+# cache deps before building and copying source so that we don't need to re-download as much
+# and so that source changes don't invalidate our downloaded layer
+RUN go mod download
+
+# Copy the go source
+COPY cmd/ cmd/
+COPY pkg/ pkg/
+
+# Build
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o groups ./cmd/
+
+# Use distroless as minimal base image to package the manager binary
+# Refer to https://github.com/GoogleContainerTools/distroless for more details
+FROM gcr.io/distroless/static:nonroot
+WORKDIR /
+COPY --from=builder /workspace/groups .
+USER nonroot:nonroot
+
+ENTRYPOINT ["/groups"]

google_groups/Makefile

@@ -0,0 +1,43 @@
+GIT_VERSION:=$(shell git describe --dirty --always)
+
+PROJECT:=kf-infra-gitops
+
+IMAGE:=gcr.io/$(PROJECT)/groups-sync:$(GIT_VERSION)
+
+# Echo commands
+SHELL = sh -xv
+
+LATESTIMAGES=latest_image.yaml
+
+CONTEXT:=kf-autobot
+
+# Build it locally
+build:
+	mkdir -p .build
+	go build -o .build/groups ./cmd
+
+# Synchronize the groups using personal credentials.
+sync: build
+	./.build/groups run --input=./groups/*.yaml \
+		--continuous=false \
+		--credentials-file=gs://kf-infra-gitops_secrets/autobot-at-kubeflow_client_secret.json
+
+build-submit:
+	gcloud --project=$(PROJECT) builds submit --machine-type=n1-highcpu-32 --tag=$(IMAGE) ./
+
+build-output:
+	yq w -i latest_image.yaml $(PROJECT).groups.image \
+	  $(IMAGE)@$(shell gcloud container images describe $(IMAGE) | yq r - image_summary.digest)
+
+set-image:
+	cd  manifests && kustomize edit set image groups=$(shell yq r $(LATESTIMAGES) $(PROJECT).groups.image)
+
+update-image: build-submit build-output set-image
+
+
+deploy:
+	kustomize build manifests | kubectl --context=$(CONTEXT) apply -f -
+
+# Build the image using GCB
+# We can't use skaffold due to: https://github.com/GoogleContainerTools/skaffold/issues/3468
+build: build-submit build-output

google_groups/README.md

@@ -1,29 +1,99 @@
-# internal-acls
+# Google Groups Sync For kubeflow.org
 
-Folder used to maintain google group ACLs used by the Kubeflow community.
+This directory contains
 
-The text files contain lists of folks that should be added
-to various Google Groups in kubeflow.org that control
-access to various shared resources.
+* A Go Binary to auto sync Google group membership based on YAML configurations checked into GitHub
+* The YAML configurations in groups/
 
-The script `sync_groups.sh` can be used to sync groups
-using the GAM CLI. Only administrators with appropriate
-permissions will be able to sync groups.
+## How this Works
 
-To create a new group
+* We use the [Directory API](https://developers.google.com/admin-sdk/directory/v1/quickstart/go) 
+  to programmatically update Google groups based on YAML files
+
+* The script runs using the gsuite account `autobot@kubeflow.org`
+
+  * Password and recovery codes are stored in secret manager
+
+    * **secret**: [projects/kf-infra-gitops/secrets/autobot-at-kubeflow-oauth-admin-api](https://console.cloud.google.com/security/secret-manager/secret/autobot-at-kubeflow-oauth-admin-api?project=kf-infra-gitops)
+
+      ```
+      gcloud --project=kf-infra-gitops secrets versions access latest --secret="autobot-at-kubeflow-oauth-admin-api"
+      ```
+
+* An OAuthClient ID is stored in gcs: `gs://kf-infra-gitops_secrets/autobot-at-kubeflow_client_secret.json`
+
+* An OAuth2 refresh token is stored in secret manager to allow the script to run without human intervention
+
+* When `groups` runs it uses a GSA to read the OAuth2 refresh token from secret manager and then uses it
+  to authenticate as `autobot@kubeflow.org` to the calendar API
+
+* A side car runs git-sync](https://github.com/kubernetes/git-sync) in a side car to synchronize the repo to a volume mount
+
+* The `groups` program polls the location of the YAML file and when it detects a change (based on a hash of file contents) it runs a synchronization
+
+  * The groups program will also periodically force a sync even if no changes are detected to deal with any
+    drift for this to occur
+
+
+* The account `autobot@kubeflow.org` is a groups admin for kubeflow.org
+
+## To Manually Synchronize the Groups
+
+In order to run the sync you need the following
+
+  * You must be a Groups admin for kubeflow.org
+
+    * You must use an @kubeflow.org email an @google.com account won't work.
+
+  * You must have access to an OAuth2 ClientSecret
+
+    * The Makefile command below reads the OAuth2 credentials from GCS
+    * You must have access to that GCS file or else change it to use a different file
 
 ```
-GROUP_NAME=google-kubecon-europe
-GROUP_EMAIL=${GROUP_NAME}@kubeflow.org 
-DESCRIPTION="Some group description"
+make sync
+```
+
+## Refreshing the OAuth2 Refresh Token For `autobot@kubeflow.org`
+
+The OAuth2 refresh token is stored inside a secret in secret manager
+
+  * **secret** [projects/kf-infra-gitops/secrets/autobot-at-kubeflow-oauth-admin-api](https://console.cloud.google.com/security/secret-manager/secret/autobot-at-kubeflow-oauth-admin-api?project=kf-infra-gitops)
+
+To regenerate the refresh token
+
+1. Destroy the latest version of the secret stored in secret manager
+
+1. Run a sync manually
+
+   ```
+   run --input=./groups/*.yaml \
+      --credentials-file=gs://kf-infra-gitops_secrets/autobot-at-kubeflow_client_secret.json \
+      --secret=kf-infra-gitops/autobot-at-kubeflow-oauth-admin-api
+   ```
 
-/home/jlewi/bin/gam/gam create group ${GROUP_EMAIL} who_can_join invited_can_join \
-  name ${GROUP_NAME} description "${DESCRIPTION}" \
-  allow_external_members true
+   * You will be directed through the OAuth2 Web Flow
+   * Be sure to login using the account **autobot@kubeflow.org**
+
+     * The password and recovery codes for **autobot@kubeflow.org** are stored in secret manager
+
+       * **secret** [projects/kf-infra-gitops/secrets/autobot-kubeflow-org-password](https://console.cloud.google.com/security/secret-manager/secret/autobot-kubeflow-org-password?project=kf-infra-gitops)
+
+
+## Importing Settings
+
+The groups binary has an `import` command which can be used to update the YAML files with the latest configuration
+in Google Groups
+
+* This is useful for intializing the configs for any Groups not currently controlled via GitOps
+
+```
+groups import \
+  --domain=kubeflow.org \
+  --output=./google_groups/groups \
+  --credentials-file=gs://kf-infra-gitops_secrets/autobot-at-kubeflow_client_secret.json
 ```
 
-## Setting Up GAM to run `sync_groups.sh`
+## References
 
-1. You need a `xxxx@kubeflow.org` email address and you must be an admin in kubeflow.org gsuite. Ask an existing admin to invite you and give you permissions.
-1. Follow instructions in https://github.com/jay0lee/GAM/wiki to set up the GAM CLI tool.
-1. You can now run `sync_groups.sh`.
+* https://developers.google.com/admin-sdk/directory/v1/quickstart/go