Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding an s3 default provisioner based on minio #89

Merged

Conversation

astromechza
Copy link
Member

@astromechza astromechza commented Nov 22, 2024

Fixes: #65

This PR brings some much needed compatibility with score-compose by adding an s3 bucket provisioner by default using an in-cluster object storage based on minio.

This follows the current best practises of k8s provisioner development:

  1. A single minio instance is created with stateful set and PVC based data storage. The pods run as non-root.
  2. A k8s Service is created routing both api and console ports to the minio instance.
  3. An s3 bucket is created per s3 resource. We intentionally use a shared access key across all buckets for now to match score-compose behavior.
  4. All secret outputs are encoded as references to in-cluster secrets.

testing

This was tested with the following score file:

apiVersion: score.dev/v1b1
metadata:
    name: demo-app
containers:
    main:
        image: quay.io/minio/minio
        command: ["/bin/bash", "-c", "sleep 3600"]
        variables:
            S3_ENDPOINT: ${resources.bucket-a.endpoint}
            S3_BUCKET: ${resources.bucket-a.bucket}
            S3_ACCESS_KEY_ID: ${resources.bucket-a.access_key_id}
            S3_SECRET_KEY: ${resources.bucket-a.secret_key}
resources:
    bucket-a:
        type: s3
    bucket-b:
        type: s3

This started up a pod with the minio object storage client cli.

I could then run the following:

$ mc alias set myminio $S3_ENDPOINT $S3_ACCESS_KEY_ID $S3_SECRET_KEY
...
Added `myminio` successfully.
$ mc ls myminio
[2024-11-22 09:23:22 UTC]     0B bucket-087c173f-4d3f-4a6b-e637-68d3ac846c34/
[2024-11-22 09:23:22 UTC]     0B bucket-927541d4-cb81-43b6-9722-3c5cceb8e9c0/
$ mc ls myminio/$S3_BUCKET
$ mc put ./etc/hostname myminio/$S3_BUCKET
/etc/hostname:               26 B / 26 B ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 695 B/s 0s
$ mc ls myminio/$S3_BUCKET
[2024-11-22 09:40:30 UTC]    26B STANDARD hostname

I've also tested it with the following score file that uses a duckdb s3 client: https://gist.github.com/astromechza/b5cedbce68dd0baa6252728ff1de9505

Signed-off-by: Ben Meier <ben.meier@humanitec.com>
Signed-off-by: Ben Meier <ben.meier@humanitec.com>
Signed-off-by: Ben Meier <ben.meier@humanitec.com>
@astromechza astromechza linked an issue Nov 22, 2024 that may be closed by this pull request
Signed-off-by: Ben Meier <ben.meier@humanitec.com>
Signed-off-by: Ben Meier <ben.meier@humanitec.com>
Copy link
Contributor

@mathieu-benoit mathieu-benoit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, wonderful!

@astromechza astromechza merged commit 20cc595 into main Nov 22, 2024
4 checks passed
@astromechza astromechza deleted the 65-feature-request-add-a-default-in-cluster-s3-provisioner branch November 22, 2024 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature request] Add a default in-cluster s3 provisioner
2 participants