DX: Easily support different workflows by sharing compose configurations

[aaslamin]

Describe the solution you'd like

The current docker compose setup in Hydra makes it cumbersome to switch between various workflows (e.g. switching between mysql/postgres, setting up tracing, etc). Currently, one has to uncomment the services/configurations they are after.

Instead, we can use multiple compose files to customize the application to support a completely different workflow. This is supported by compose - https://docs.docker.com/compose/extends/#multiple-compose-files

I propose we have a default configuration with postgres, with extensions/overrides for mysql and setting up tracing via jaeger:

• docker-compose.with-mysql.yml
• docker-compose.with-jaeger.yml

For example, if I want to setup Hydra to use mysql instead AND configure tracing via Jaeger:

docker-compose -f docker-compose.yml -f docker-compose.with-mysql.yml -f docker-compose.with-jaeger.yml up

This makes it easy for people to add new workflows in the future (e.g. a setup for Prometheus).

[aeneasr]

Oh yeah, super nice!