Need advice on Solid Queue's memory usage

[yjchieng]

Ruby: 3.3.4
Rails: 7.2.1
Solid Queue: 0.7.0, 0.8.2

---

I run a Rails App on AWS EC2 instance with 1G of memory.
I notice the solid queue process takes up 15-20% of the instance's memory, which becomes the single largest process by memory usage.

---

What I checked:

1. Check memory usage by start/stop supervisorctl
   (I use it to manage my solid queue process)

stop supervisorctl - free memory 276MB
start supervisorctl - free memory 117MB

It increases 159MB

2. Stop supervisorctl service, and run "solid_queue:start" directly
   Trying to see if this is something related to supervisor.

before solid_queue:start - free memory 252MB
after solid_queue:start - free memory 109MB

It increases 143MB

3. Then I notice there is a latest version.
   I upgraded to 0.8.2 (was 0.7.0).

stop supervisorctl - free memory 220MB
start supervisorctl - free memory 38MB

It increases 182MB

---

I need some advise:

1. Is 150-200MB the minimum requirement to run "solid_queue:start"?
2. Is there any setting/feature that I can switch off to reduce memory usage?
3. Is there any setting that I can limit the maximum memory usage?

And, thanks a lot for making this wonderful gem. :)

[rosa]

Hey @yjchieng, thanks for opening this issue! 🙏 I think it depends a lot on your app. A brand new Rails app seems to use around 74.6MB memory for me after booting (without Solid Queue, just running Puma). I think the consumption you're seeing is from all the processes together and not just the supervisor, measuring free memory before starting the supervisor and after, as the supervisor will fork more processes. Are you running multiple workers or just one? I think reducing the number of workers there would help. Another thing that might help is using bin/jobs, which preloads the whole app before forking, but the gains there are usually quite modest.

[rosa]

There might also be something else going on because the only changes from version 0.7.0 to 0.8.2 were for the installing part of Solid Queue; nothing was changed besides the initial installation, so the memory footprint shouldn't have changed. I imagine there is other stuff going on in your AWS instance at the same time that might consuming memory as well.