TenantSpec.credsSecret field is not optional

[dnskr]

Tenant spec documents the credsSecret field as Optional but actually it is not.
The MinIO Operator does not autogenerate the secret when credsSecret field is omitted.

Expected Behavior

Following the documentation
The MinIO Operator automatically generates the secret along with appropriate values for the access key and secret key if this field is omitted

Current Behavior

Tenant stuck with message

kubectl get tenant --all-namespaces
NAMESPACE   NAME    STATE                                                       AGE
data24      minio   please set credsSecret secret with credentials for Tenant   40m

Steps to Reproduce (for bugs)

Create Tenant without credsSecret field

[ravindk89]

@dvaldivia did the functionality change at some point, or was this always true and we just missed it in review?

[dnskr]

@dvaldivia @ravindk89 I found one more problem with credsSecret field and it might be related to this issue.

If I delete Tenant and create it again but change accesskey and/or secretkey in the Secret than in the minio pod logs I see

Verifying if 3 buckets are consistent across drives...
API: SYSTEM()
Time: 11:41:36 UTC 04/30/2021
DeploymentID: 3556e1ae-fcd6-47c9-a680-7e93e34a4a06
Error: Unable to initialize config, some features may be missing Invalid credentials (*fmt.wrapError)
       3: cmd/server-main.go:385:cmd.initAllSubsystems()
       2: cmd/server-main.go:317:cmd.initServer()
       1: cmd/server-main.go:524:cmd.serverMain()
Status:         4 Online, 0 Offline. 
Endpoint: http://172.17.0.15:9000  http://127.0.0.1:9000 
Browser Access:
   http://172.17.0.15:9000  http://127.0.0.1:9000
Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
API: SYSTEM()
Time: 11:41:36 UTC 04/30/2021
DeploymentID: 3556e1ae-fcd6-47c9-a680-7e93e34a4a06
Error: Unable to migrate IAM users and policies to new format: sio: data is not authentic (*fmt.wrapError)
       1: cmd/iam.go:617:cmd.(*IAMSys).Init()
API: SYSTEM()
Time: 11:41:36 UTC 04/30/2021
DeploymentID: 3556e1ae-fcd6-47c9-a680-7e93e34a4a06
Error: IAM sub-system is partially initialized, some users may not be available (*errors.errorString)
       1: cmd/iam.go:618:cmd.(*IAMSys).Init()

At the same time I don't have this issue if I change CONSOLE_SECRET_KEY for users secrets or consoleSecret

[dnskr]

Some info regarding my last comment.
For example, we have the following Secret and Tenant definitions:
Secret

apiVersion: v1
kind: Secret
metadata:
  name: minio-creds-secret
type: Opaque
stringData:
  accesskey: minio
  secretkey: minio123

Tenant

apiVersion: minio.min.io/v2
kind: Tenant
metadata:
  name: minio
spec:
  ...
  credsSecret:
    name: minio-creds-secret
  ...

In this case Opeator creates minio pods with env variables

MINIO_ROOT_USER=minio
MINIO_ROOT_PASSWORD=minio123

These variables are used to create root user and encrypt the config stored at the backend. So if I change minio-creds-secret values and recreate pod than the pod will use new accesskey/secretkey (MINIO_ROOT_USER/MINIO_ROOT_PASSWORD) and will try to decrypt config which was encrypted with old values by new accesskey/secretkey. I think this is the root cause of the problem.

The questions I have for this case:
How should root user accesskey and secretkey be updated/rotated?

[dvaldivia]

@dnskr at the moment the root credentials cannot be rotated, I think we are making changes to the server to actually deprecate the root credentials, however at the moment they are needed, credsSecret was never optional

[ravindk89]

I'll push a fix to update the docs before end of day.

[harshavardhana]

@dnskr at the moment the root credentials cannot be rotated, I think we are making changes to the server to actually deprecate the root credentials, however at the moment they are needed, credsSecret was never optional

Root credentials are not used with encryption anymore with v4.0.10 release of operator defaults to latest release of MinIO - which brings this change. This would allow for root credentials to change anytime you feel like it.

Backend IAM/Config encryption now fully depends on KMS configuration i.e KES deployment. We can consider this issue closed.

@ravindk89 you may still send the PR to fix the docs.

[mehemken]

The format of that secret is not clear in the documentation. It took me about an hour to figure this out.

Ideally this page should make it clear that the secret should have this format:

apiVersion: v1
kind: Secret
metadata:
  name: storage-configuration
type: Opaque
stringData:
  config.env: |-
    export MINIO_ROOT_USER=admin
    export MINIO_ROOT_PASSWORD=password123

[ramondeklein]

The documentation refers to the example hat has the following example secret:

apiVersion: v1
kind: Secret
metadata:
  name: storage-configuration
  namespace: minio-tenant
type: Opaque
stringData:
  config.env: |-
    export MINIO_ROOT_USER="minio"
    export MINIO_ROOT_PASSWORD="minio123"
    export MINIO_STORAGE_CLASS_STANDARD="EC:2"
    export MINIO_BROWSER="on"