Create a method to use secrets in QHub config

[aktech]

After discussion with @costrouc we realise we don't have a way to use secrets in the Qhub ops, one example is secrets are expected to be in the config.yaml (which is not ideal):

https://github.com/Quansight/qhub-ops/blob/e5dd52df558c2c4eb805594b6ae574f12e0986a1/tests/assets/config_aws.yaml#L11

The solution we came upon was to make qhub read from environment variables and refer those environment variables in the cookiecutter's generation of the project.

Some thing like:

QHUB_GITHUB_CLIENT_SECRET
QHUB_GITHUB_CLIENT_ID
...

[costrouc]

To add why this seems like a reasonable path forward is that terraform does the exact same thing https://www.terraform.io/docs/configuration/variables.html#environment-variables

[costrouc]

• override capability should be applied to the validate https://github.com/Quansight/qhub/blob/master/qhub/cli/validate.py and render https://github.com/Quansight/qhub/blob/master/qhub/cli/render.py commands
• Add the ability to ignore environment variables as well via a command line option e.g. --ignore-environment

The override should be done similarly to Terraform. E.g. suppose we have a config file

a:
  b_e:
    - c: 1
      d: [1, 2, 3]

We would override via QHUB__a__b_e__0__c=2. I realize this adds several requirements. Since we already have underscores in our keys and terminals typically only allow [A-Za-z0-9_] we have to parse the attribute path and handle the underscores properly (we could get arround this by requiring __ two underscores between all keys. Also we should handle list accessing by checking if the key is an integer and treat it as a list. Additionally we should attempt to cast the value as an integer since there are several integers in the config file if that fails just supply the value as a string.

We would apply all environment variables that start with QHUB__

[tylerpotts]

In the course of working on this, we realized that this will require a more nuanced approach. Editing the input config doesn't actually hide the secrets, as they will get written out in plain text in the infrastructure files

[costrouc]

Closing this issue since we don't think that adding this feature is a good idea due to the secrets being in the repo regardless

[tylerpotts]

@aktech @costrouc Let's continue this discussion

[danlester]

I think a possible solution for Auth0 at least could be to use the Auth0 terraform provider: https://registry.terraform.io/providers/alexkappa/auth0/latest

The management API key/secret can be in GitHub secrets / env vars going into qhub deploy, then the provider should be able to sync with Auth0 to get details for a specific named client.

The final step would be to take the client key/secret and pass it to jupyterhub config, probably through env vars.

This way, the client key/secret isn't needed in the qhub-config.yaml at all. It would only be in the remote state.

I haven't found a similar provider for GitHub but haven't looked hard yet...

[github-actions]

This issue has been automatically marked as stale because there was no recent activity in 60 days. Remove the stale label or add a comment, otherwise, this issue will automatically be closed in 7 days if no further activity occurs.

[github-actions]

This issue was closed because it has been stalled for 7 days with no activity.

[leej3]

I will try to work through this in the next week or two.

[costrouc]

Reopening since this feature was removed in #1003.

[viniciusdc]

This needs to be addressed, this issue was resolved with #720. As per @costrouc
comment, this was removed from 0.4.0, so I will link this as a new issue for re-enabling this feature.