feat: support bucket region for all routes #35

Juiced66 · 2024-11-25T10:20:43Z

Modularized Controllers, Bucket Controller, and Testing Migration

Summary

This PR introduces significant structural changes and enhancements to the kuzzle-plugin-s3 project. It modularizes controllers, adds a dedicated bucket controller with full CRUD capabilities, migrates tests to jest, and enforces stricter configuration requirements.

Key Changes

Controller Refactoring:
- Split controllers into separate modules for improved maintainability and scalability.
- Modularized design ensures cleaner separation of concerns and easier future enhancements.
Bucket Controller:
- Implements CRUD operations for S3 buckets:
  - Create: Create buckets with region-specific configurations.
  - Read: Check bucket existence.
  - Update: Update policies or public access settings.
  - Delete: Safely remove empty buckets.
  - Empty: Empties a bucket.
- Allows bucket configuration management (e.g., CORS, policies).
Testing Framework Migration:
- Migrated all tests to jest, replacing the existing framework.
- Added comprehensive unit and integration tests to ensure reliability of the new features.
Breaking Changes:
- Mandatory Parameters:
  - bucketName and bucketRegion are now required for all applicable API calls.
- Configuration Format:
  - The configuration file format has changed to support multi-region setups and enhanced flexibility.

Migration Notes

API Clients:
- Ensure all requests include bucketName and bucketRegion where required.
Configuration Update:
- Update your kuzzlerc and other configuration files to match the new format. Example configurations for multiple regions are provided in the updated documentation.

Benefits

Improved modularity and maintainability.
Comprehensive bucket management capabilities.
Enhanced test coverage for robust validation.
Compatibility with multi-region setups.

Testing and Validation

Verified functionality through extensive tests using jest.

Additional Notes

This is a breaking change. Ensure that configurations and client integrations are updated accordingly.

For a detailed view of changes, refer to the diff view.

…PublicAccess) - also enable default CORS rules on bucket creation - support API param dynamic change of bucket name - fix fileGetUrl support expire time and bucket region

upgrade to uuid v9 was not taken into account in code use

stack trace is insufficient in plugin calling point to identify which step is causing a crash when creating a bucket

…ucket request params - this is not handled at the moment because we need to disable block public request before setting a bucket policy that makes access public. - it is then currently advided to first create the bucket, then disable blockPublicAccess, then put bucket policies in order to make sure bucket permissions are set in the right order. - it could be a feature to support proper/automatic disable of block public access based on bucket policies ? see roadmap for this plugin

…node LTS

… enable buckets use by default

sebtiz13

Globally, why not use typescript in this plugin ?

tests/setup/jest.setup.js

Juiced66 · 2024-12-18T12:29:14Z

Globally, why not use typescript in this plugin ?

I 100% agree it should, unfortunately, this PR is required ASAP and I cannot take time to do so

kuzzle · 2025-01-02T13:15:21Z

🎉 This PR is included in version 2.2.0-beta.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Mathis Gardon and others added 17 commits November 21, 2024 13:43

feat (S3Plugin) add bucket controller (create, delete, exists, enable…

cdfd690

…PublicAccess) - also enable default CORS rules on bucket creation - support API param dynamic change of bucket name - fix fileGetUrl support expire time and bucket region

feat (tests) add support for bucketName and bucket controller

496a68c

chore (docs) add docs

0183ff9

chore (S3 config) add default region eu-west-3

216a117

fix (uuid) update uuid package usage

f8e5bcd

upgrade to uuid v9 was not taken into account in code use

fix (S3) rename call to putBucketPolicy

1ceffd9

chore (S3plugin) formatting

76cf7b3

fix (bucketRegion) always use the same config key to set bucket region

32572b0

fix (bucket delete) properly await for delete call

d90a109

chore (bucket create) properly log errors at each create step

0cc3955

stack trace is insufficient in plugin calling point to identify which step is causing a crash when creating a bucket

fix (package-lock) upgrade package-lock version to conform to latest …

5eb1c4f

…node LTS

fix (S3Plugin) fix error logging when checking bucket exists

56fdb06

fix (S3Plugin.bucketExists) ignore minio not found error

6bc2bd0

fix (S3Plugin) minio should be called when setting bucket policies to…

01f1c72

… enable buckets use by default

chore (S3Plugin) removes invalid filter on error.code

fd5ab37

feat: support bucket region for all routes

1158a80

Juiced66 force-pushed the bucket-controller branch from a30b154 to 1158a80 Compare November 25, 2024 10:24

Juiced66 self-assigned this Nov 25, 2024

Juiced66 added 2 commits November 25, 2024 11:48

chore: lint

f5266bc

fix: http routes

1959063

Juiced66 marked this pull request as draft November 26, 2024 15:24

Juiced66 added 8 commits December 10, 2024 12:29

feat!: full support of multi region and bucket controller

9a2ce71

fix: vault is single credential source of truth

6953105

fix: credentials usage, error handling, request body parsing

90c8949

tests: add BaseController tests

945f460

fix: fileGetUrl http path

8879186

tests: fix test for fileGetUrl

95807d8

fix: remove public url usage

95cc3ee

fix: fileGetUrl and tests

c035775

Juiced66 added 11 commits December 13, 2024 10:21

feat: permit to use any S3 option for clients (endpoint level)

0e15ca1

feat: add publicUrl support as a optionnal parameter

99b59ff

docs: update doc

11464e2

chore: auto review fixes

7b86815

fixup! chore: auto review fixes

263da29

fix: api path

ec69546

fixup! fix: api path

e22bd11

fix: default config

3761585

fix: controller names

1080542

fix: bucket create for minio

c391b51

tests: fix tests to reflect api changes

36fcf2c

sebtiz13 reviewed Dec 18, 2024

View reviewed changes

tests/setup/jest.setup.js Outdated Show resolved Hide resolved

rolljee and others added 9 commits December 18, 2024 14:13

chore: update doco file

5066202

chore: Update workflow file

888e8d1

fix: remove unused env in docker compose, add needed ones

3d14452

chore: remove html test file as it does not work anymore

c7dbb19

fix: parse signedUrlTTL if needed

c2eee74

fix(tests): fix ttl test

03fdec7

docs: fix http api calls

507533e

feat: add empty method to bucket controller

20afb01

chore: move testTimeout to config

26553fb

alexandrebouthinon force-pushed the master branch from 39690ad to 18d4326 Compare December 20, 2024 14:40

Juiced66 mentioned this pull request Jan 2, 2025

feat: bucket controller #33

Closed

rolljee force-pushed the master branch from e9974bf to 31c7d11 Compare January 2, 2025 10:09

Juiced66 changed the base branch from master to beta January 2, 2025 13:10

Juiced66 merged commit d2301b8 into beta Jan 2, 2025
8 checks passed

kuzzle added the released on @beta label Jan 2, 2025

sebtiz13 deleted the bucket-controller branch January 9, 2025 14:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support bucket region for all routes #35

feat: support bucket region for all routes #35

Juiced66 commented Nov 25, 2024 •

edited

Loading

sebtiz13 left a comment

Juiced66 commented Dec 18, 2024

kuzzle commented Jan 2, 2025

feat: support bucket region for all routes #35

feat: support bucket region for all routes #35

Conversation

Juiced66 commented Nov 25, 2024 • edited Loading

Modularized Controllers, Bucket Controller, and Testing Migration

Summary

Key Changes

Migration Notes

Benefits

Testing and Validation

Additional Notes

sebtiz13 left a comment

Choose a reason for hiding this comment

Juiced66 commented Dec 18, 2024

kuzzle commented Jan 2, 2025

Juiced66 commented Nov 25, 2024 •

edited

Loading