Provide a slimed down version of Sentry self-hosted

[cleptric]

Problem Statement

Over the years, the product we offer at Sentry has grown rapidly in features but also complexity. I do remember when I first started to self-host Sentry, this was around the 9.x days, and how little of a VM you could use to get everything running.

Today, you need a much beefier machine, given the countless features and resulting docker containers the setup requires.

Solution Brainstorm

It might be worth thinking about whether we can offer a more light, stripped-down version of self-hosted Sentry that only supports a few core features.
I'm aware that this won't be an easy task, as so many features are now intertwined with each other.

[aldy505]

This might go hand in hand with #2533, as one solution that I could think of is make the features be enabled interactively (using some kind of user prompt.... or something). For current implementation, that would mean that the docker-compose.yml file will be created programmatically instead of having it already set up on current repo. This would create another new overhead for maintenance.

Reducing the numbers of Snuba containers (see getsentry/snuba#1670) might help, but I often wonder about how much resource you would actually save. This would also save configuration overhead if some users only want some features.

One thing that we would do for now, is to add some inline documentation on this specific code to have explanation on what kind of features you can disable if you don't want X feature on your instance, and what other containers that you can strip out from the docker-compose.yml file.

self-hosted/sentry/sentry.conf.example.py

Lines 264 to 292 in 16bea1e

	for feature in (
	"organizations:discover",
	"organizations:events",
	"organizations:global-views",
	"organizations:incidents",
	"organizations:integrations-issue-basic",
	"organizations:integrations-issue-sync",
	"organizations:invite-members",
	"organizations:metric-alert-builder-aggregate",
	"organizations:sso-basic",
	"organizations:sso-rippling",
	"organizations:sso-saml2",
	"organizations:performance-view",
	"organizations:advanced-search",
	"organizations:session-replay",
	"organizations:issue-platform",
	"organizations:profiling",
	"organizations:dashboards-mep",
	"organizations:mep-rollout-flag",
	"organizations:dashboards-rh-widget",
	"organizations:metrics-extraction",
	"organizations:transaction-metrics-extraction",
	"projects:custom-inbound-filters",
	"projects:data-forwarding",
	"projects:discard-groups",
	"projects:plugins",
	"projects:rate-limits",
	"projects:servicehooks",
	)

Today, you need a much beefier machine, given the countless features and resulting docker containers the setup requires.

Don't get me wrong, I love having all the features Sentry could over today, and yes I do have a beefy machine dedicated for running Sentry (currently 6 vCPU, 12 GB RAM, 3 GB swapfile, 47 GB disk).

[hubertdeng123]

This effort looks like it goes hand in hand with #1763

[brunis]

And now your "beefy" machine is already outdated.. I just added a 20GB swap file just get Sentry to start and I have 16GB of memory. My sentry machine requires more hardware than the application we log from :'(

[dusatvoj]

@brunis , @aldy505 , I had issue with infinite resource consumption increase and I found that. Take a look at Kafka and Redis, maybe it will help you https://yuretspro.medium.com/on-premise-sentry-under-the-high-load-how-to-stay-up-running-be6a0a33791f

But yes, It's insane that sentry, even without such load, is consuming 13+GB of RAM and it's smth like 55+ containers 🔥 🤯

Would be so happy to have some fitness version of sentry 🙏