Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check Minio (S3 Storage) as a replacement for current Cassandra/File System Storage #607

Open
4 tasks
jpowie01 opened this issue Mar 20, 2019 · 0 comments
Open
4 tasks
Labels
Backend 🏭 Stuff related to Backend part. Enhancement 🌪 Things that improve our project. P1 Things with #1 Priority

Comments

@jpowie01
Copy link
Member

Current Behavior

Currently we use Cassandra as a key-value storage for images. It seems to work quite well and performant. However, it uses too much resources and its Python driver seems too be quiet for some time. It's high time to consider a change and support S3 Storage.

Two years ago, there were not so many good storages that could be set up locally in Docker. The only S3 Storage that was reasonable to use was Amazon S3 in the cloud... Now, there are many more!

Expected Behavior

Let's prepare a POC for Minio (https://github.com/minio/minio). Remove Cassandra support from MedTagger and replace it with S3 Storage backed by e.g. Minio or any other S3-compatible storage.

This will allow us to scale even better in the cloud!

Tasks to do

  • Prepare Docker environment with Minio
  • Replace existing Cassandra code and migrate it to Minio
  • Test it using existing functional and e2e tests
  • Benchmark it and compare with existing architecture
@jpowie01 jpowie01 changed the title Check Minio (S3 Storage) as a replacement for current Cassandra or File System Check Minio (S3 Storage) as a replacement for current Cassandra/File System Storage Mar 20, 2019
@jpowie01 jpowie01 added Backend 🏭 Stuff related to Backend part. Enhancement 🌪 Things that improve our project. P1 Things with #1 Priority labels Mar 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backend 🏭 Stuff related to Backend part. Enhancement 🌪 Things that improve our project. P1 Things with #1 Priority
Projects
None yet
Development

No branches or pull requests

1 participant