Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Reading TLS certificates when deployed in K8S #10601

Merged
merged 1 commit into from
Sep 30, 2020

Conversation

Alevsk
Copy link
Contributor

@Alevsk Alevsk commented Sep 30, 2020

Description

Ignore all regular files, CAs directory and any directory that starts
with .. inside the .minio/certs folder

Motivation and Context

When minio is deployed in Kubernetes, k8s will mount TLS certificates as symlinks, ie:

/tmp/certs # ls -la
total 4
drwxrwxrwt    3 root     root           200 Sep 30 02:14 .
drwxrwxrwt    1 root     root          4096 Sep 30 02:14 ..
drwxr-xr-x    6 root     root           160 Sep 30 02:14 ..2020_09_30_02_14_31.905068249
lrwxrwxrwx    1 root     root            31 Sep 30 02:14 ..data -> ..2020_09_30_02_14_31.905068249
lrwxrwxrwx    1 root     root            10 Sep 30 02:14 CAs -> ..data/CAs
lrwxrwxrwx    1 root     root            17 Sep 30 02:14 hostname-0 -> ..data/hostname-0
lrwxrwxrwx    1 root     root            17 Sep 30 02:14 hostname-1 -> ..data/hostname-1
lrwxrwxrwx    1 root     root            17 Sep 30 02:14 hostname-2 -> ..data/hostname-2
lrwxrwxrwx    1 root     root            18 Sep 30 02:14 private.key -> ..data/private.key
lrwxrwxrwx    1 root     root            17 Sep 30 02:14 public.crt -> ..data/public.crt

By default folders to symlinks are not considered Folders by file.isDir (return false) so MinIO is ignoring those certificates

How to test this PR?

Upload certificates to your certs folder with the same folder structure as above

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation needed
  • Unit tests needed

@Alevsk Alevsk self-assigned this Sep 30, 2020
@Alevsk Alevsk force-pushed the certificates-symlinks branch from 33d079f to d322fae Compare September 30, 2020 02:42
Ignore all regular files, CAs directory and any directory that starts
with `..` inside the `.minio/certs` folder
@Alevsk Alevsk force-pushed the certificates-symlinks branch from d322fae to 907a727 Compare September 30, 2020 02:44
@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-zoned.sh ✔️
mint-gateway-nas.sh ✔️
mint-gateway-azure.sh more...

10601-33d079f/mint-gateway-azure.sh.log:

Running with
SERVER_ENDPOINT:      minio-dev6.minio.io:31946
ACCESS_KEY:           minioazure
SECRET_KEY:           ***REDACTED***
ENABLE_HTTPS:         0
SERVER_REGION:        us-east-1
MINT_DATA_DIR:        /mint/data
MINT_MODE:            full
ENABLE_VIRTUAL_STYLE: 0

To get logs, run 'docker cp 00680321fcc1:/mint/log /tmp/mint-logs'

(1/15) Running aws-sdk-go tests ... done in 8 seconds
(2/15) Running aws-sdk-java tests ... done in 2 seconds
(3/15) Running aws-sdk-php tests ... done in 1 minutes and 10 seconds
(4/15) Running aws-sdk-ruby tests ... done in 20 seconds
(5/15) Running awscli tests ... done in 2 minutes and 9 seconds
(6/15) Running healthcheck tests ... done in 0 seconds
(7/15) Running mc tests ... done in 3 minutes and 56 seconds
(8/15) Running minio-dotnet tests ... done in 1 minutes and 42 seconds
(9/15) Running minio-go tests ... done in 6 minutes and 36 seconds
(10/15) Running minio-java tests ... done in 10 minutes and 30 seconds
(11/15) Running minio-js tests ... done in 2 minutes and 37 seconds
(12/15) Running minio-py tests ... done in 19 minutes and 11 seconds
(13/15) Running s3cmd tests ... done in 2 minutes and 18 seconds
(14/15) Running s3select tests ... FAILED in 2 seconds
{
  "name": "s3select:test_csv_input_quote_char",
  "function": "select_object_content(bucket_name, object_name, opts)",
  "args": {
    "bucket_name": "s3select-test-e8016270-191f-4341-87dd-13ec799b0d31"
  },
  "duration": 2446,
  "message": "Test test_2 unexpectedly failed with: 'event-type'",
  "error": "Traceback (most recent call last):\n  File \"/mint/run/core/s3select/csv.py\", line 39, in test_sql_api\n    for d in data.stream(10*1024):\n  File \"/usr/local/lib/python3.6/dist-packages/minio/select/reader.py\", line 211, in stream\n    message = self.__extract_message()\n  File \"/usr/local/lib/python3.6/dist-packages/minio/select/reader.py\", line 161, in __extract_message\n    event_type = header_map[\"event-type\"]\nKeyError: 'event-type'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"./tests.py\", line 52, in main\n    test_csv_input_custom_quote_char(client, log_output)\n  File \"/mint/run/core/s3select/csv.py\", line 119, in test_csv_input_custom_quote_char\n    data, sql_opts, expected_output)\n  File \"/mint/run/core/s3select/csv.py\", line 45, in test_sql_api\n    'Test {} unexpectedly failed with: {}'.format(test_name, select_err))\nValueError: Test test_2 unexpectedly failed with: 'event-type'\n",
  "status": "FAIL"
}
(14/15) Running security tests ... done in 0 seconds

Executed 14 out of 15 tests successfully.

Deleting image on docker hub
Deleting image locally
Error: No such image: minio/minio:10601-33d079f

Alevsk added a commit to Alevsk/minio-operator that referenced this pull request Sep 30, 2020
Users can define multiple certificates to be used by MinIO via the TenantSpec.ExternalCertSecret field:

```
  externalCertSecret:
    - name: tls-test-1
      type: kubernetes.io/tls
    - name: tls-test-2
      type: kubernetes.io/tls
    - name: tls-test-3
      type: kubernetes.io/tls
```

Will create the following folder structure inside the MinIO container:

```
certs/
 │
 ├─ public.crt
 ├─ private.key
 ├─ CAs/          // CAs directory is ignored
 │   │
 │    ...
 │
 ├─ example.com/
 │   │
 │   ├─ public.crt
 │   └─ private.key
 └─ foobar.org/
     │
     ├─ public.crt
     └─ private.key
   ...
```

This PR depends on minio/minio#10601
@harshavardhana harshavardhana merged commit bea87a5 into minio:master Sep 30, 2020
@Alevsk Alevsk deleted the certificates-symlinks branch September 30, 2020 17:42
harshavardhana pushed a commit to minio/operator that referenced this pull request Sep 30, 2020
Users can define multiple certificates to be used by MinIO via the TenantSpec.ExternalCertSecret field:

```
  externalCertSecret:
    - name: tls-test-1
      type: kubernetes.io/tls
    - name: tls-test-2
      type: kubernetes.io/tls
    - name: tls-test-3
      type: kubernetes.io/tls
```

Will create the following folder structure inside the MinIO container:

```
certs/
 │
 ├─ public.crt
 ├─ private.key
 ├─ CAs/          // CAs directory is ignored
 │   │
 │    ...
 │
 ├─ example.com/
 │   │
 │   ├─ public.crt
 │   └─ private.key
 └─ foobar.org/
     │
     ├─ public.crt
     └─ private.key
   ...
```

This PR depends on minio/minio#10601
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants