Allow users to request shared namespaces for projects

[blairdrummond]

Terraform with CI to create namespace. Also a webportal frontend for user submissions for namespaces

[brendangadd]

• @sylus to implement PR-driven pipeline to manage Kubeflow profiles with an associated list of users who have access
• @sylus to communicate required PR format to @jptissot
• @jptissot to implement web portal form (with authentication) to generate these PRs

[sylus]

Hey an example of what a profile looks like is here:

https://github.com/kubeflow/kubeflow/blob/960587ac9025b6a4ba7facaabc8aa1769fb01dc8/components/profile-controller/config/samples/profile_v1_profile.yaml

Which validates against the following CRD:

https://github.com/kubeflow/kubeflow/blob/master/components/profile-controller/config/crd/bases/kubeflow.org_profiles.yaml

[jptissot]

Which repo do I need to make the PR to ?

[zachomedia]

@jptissot You should make it against the private version of the terraform-kubernetes-aks-platform-daaas

[bmanbzan]

Have a few questions about this:

1. When a user requests a namespace, how will they know the status of the request since the GitHub repo is private? Will the repo mangers be communicating with the requestor using the email address included with the request? (FYI @cloud.statcan.ca is the default email for our Azure accounts...)

2. Confirming that the user only needs to enter the name of the namespace they would like to create. All other details should be automated by the form.

3. Are these namespaces specific to Kubeflow?

[brendangadd]

@bbronzan

1. Two options, or a transition from (a) to (b):
   a. They know it's completed when they see the workspace in Kubeflow, maybe with human comms as you suggest
   b. The portal stores a reference and draws a status from the PR (open/closed/merged)
2. I believe the email addresses of the collaborators will be required for the content of the PR
3. Probably to start, but could extend into features like storage (e.g. see [Epic] Self-serve Minio object storage #9)

[bmanbzan]

1. Sounds good. It's very important for a human to reach out to the user and let them know that the namespace has been created in the meantime. (and to provide them with a link if possible)
2. We chatted about this briefly at standup. Since the namespace owner can manage access in Kubeflow, we won't include this on the request form.
3. Cool.

[bmanbzan]

A few questions about your workflow for handling these requests when it comes to denying/rejecting a requested namespace.

1. Is there an explicit action that will be taken in GitHub to reject the PR?
2. What are some reasons for rejecting a requested namespace?
3. Is this task limited to a certain person, group, or roles?

[justbert]

In relation to #23, will these shared namespaces have shared KV stores for secrets?

[brendangadd]

From @M-A-Boucher:

• I need the bot's GitHub token so that the GitHub API calls are done by the bot.
• Is there a particular branch name format? Should the namespace inputed by the user be part of the branch name?
• Commit title and message format? Include the namespace in the title and/or message?
• PR title and message format? Include the namespace in the title and/or message?

I also need the YAML template... the workflow will create a branch, inject the namespace into the template (wherever it needs to be) and commit it to that branch, and then create a PR.

• Branch: profile-req-<name>
• Commit: chore(profile): Add profile <name>
• PR: Same as commit

@sylus, @zachomedia: GitHub token done via k8s secret? Vault? Injected into pod filesystem?

/cc @justbert

[jptissot]

We will need a service account token that has access to the private github repo to use when creating the commit / pr. Maybe limit the account's access to only be able to push branches.

[M-A-Boucher]

Do we also put the requesting user's email in the PR so they can be contacted if the request is approved or rejected?

[brendangadd]

A few questions about your workflow for handling these requests when it comes to denying/rejecting a requested namespace.

1. Is there an explicit action that will be taken in GitHub to reject the PR?
2. What are some reasons for rejecting a requested namespace?
3. Is this task limited to a certain person, group, or roles?

@bbronzan:

1. PR is closed, not merged. These are to be distinct states in the API.
2. .

• User is making frivolous requests
• Profile name is offensive
• Profile name contains confidential information
• User likes pineapple on their pizza

3. We're doing identity verification and controlled intake, so it is currently appropriate for any authenticated user to be able to submit a request.

[brendangadd]

@M-A-Boucher No need to add contact information to the PR entity; it will be resolvable from the diff.

Right, @sylus?

[bmanbzan]

A few questions about your workflow for handling these requests when it comes to denying/rejecting a requested namespace.

1. Is there an explicit action that will be taken in GitHub to reject the PR?
2. What are some reasons for rejecting a requested namespace?
3. Is this task limited to a certain person, group, or roles?

@bbronzan:

1. PR is closed, not merged. These are to be distinct states in the API.
2. .

• User is making frivolous requests
• Profile name is offensive
• Profile name contains confidential information
• User likes pineapple on their pizza

1. We're doing identity verification and controlled intake, so it is currently appropriate for any authenticated user to be able to submit a request.

1. Cool, JP and MA are set.
2. I recommend adding anyone who likes mushrooms period - they're vile creatures.
3. Perfect.

So we'll still need to capture this scenario (PR is closed) in the user journey so they aren't sitting around waiting for their namespace to be created. heh

When a PR is closed, would the "closer" enter a reason why it was not closed? Would this reason be viewable by the original requester? (i.e. link to PR in GitHub)

[brendangadd]

@bbronzan The repository is (necessarily) private, so we can't provide a link to the actual PR.

If there is no portal-tracking of state, then we would have to rely on administrative staff contacting the user to share any relevant info.

Unfortunately, the GitHub API looks like comments and closures are independent. If there is a means of reliably linking a "Close and comment"-style comment to the closure event (e.g. an identical timestamp), then we can use that comment as the reason and pass it on to the end-user.

If no such mechanism exists, then... well... there's more to talk about.

/cc @jptissot @M-A-Boucher

[bmanbzan]

Cool thanks, we're going to look into catching this. We'll update you as we go.

[nelucas72]

@zachomedia @M-A-Boucher is still waiting on the YAML template to use.

[brendangadd]

@M-A-Boucher Is the Profile sample @sylus provided above not sufficient?

[jptissot]

What needs to be dynamic in that file?

[sylus]

@zachomedia mind giving explanation here?

[zachomedia]

There are basically two dynamic fields in the profile YAML:

apiVersion: kubeflow.org/v1
kind: Profile
metadata:
  name: $PROFILE_NAME
spec:
  owner:
    kind: User
    name: $OWNER_EMAIL

[zachomedia]

The GitHub token has been provided, please ensure to token is stored securely. Ideally, it should go into Hashicorp Vault and be injected in, but I can accept it in a Kubernetes secret for the time being.

[jptissot]

@zachomedia we were going to store the token in the database as encrypted. Is that acceptable? If not I can do a workaround so it's loaded via enciorronment variables

[M-A-Boucher]

@zachomedia, the bot token we were given, I hope it's not an account with admin privileges, it should only have rights to create branches, commit files, and open PRs. Is this correct?

[zachomedia]

@M-A-Boucher Yes, that is correct

[bmanbzan]

Feature is complete!