Best way to play around with Solid Queue in development?

[jeromedalbert]

Solid Queue used to use migrations on a single database, which was nice if you wanted to have everything in the same database in development.

Now Solid Queue uses schemas, which is nice since it does not clutter a brand new Rails 8 app with migrations. The schemas are for separate databases, which makes sense for production (it would be nice to document why by the way, e.g. better isolation or performance). I understand that the library is more intended for production as explained by Rosa:

Solid Queue is more intended for production, Rails ships with another adapter for development, the async adapter.

But it would be nice to be able to easily play around with it in on your local environment, to kick the tires, reproduce production issues in your local, get a feel for how it works with mission_control-jobs, and see if any configuration changes you're making to your Solid Queue setup works correctly on your local before deploying them to minimize issues.

If Solid Queue in development is not a default, then I think it should be easy to set it up on one's local. So far I can think of 3 ways:

1. Keep Solid Queue in a separate schema in production, but one schema in development. I don't know if that's possible. But having everything in one schema keeps the development setup simple, since having multiple databases looks more like a production concern. That would be the simplest setup short of using the async adapter. However it would be harder to remove solid queue (short of using --skip-solid) since I guess you would have to manually edit the schema.

2. Provide an example database.yml setup in some way, like in docs or as comments in that file, so someone can get up and running quickly even if they are not familiar with multi-db setups. This is how I set it up on my local for a new app using postgres:

   default: &default
     adapter: postgresql
     encoding: unicode
     pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
   
   cache: &cache
     <<: *default
     migrations_paths: db/cache_migrate
   queue: &queue
     <<: *default
     migrations_paths: db/queue_migrate
   
   development:
     primary:
       <<: *default
       database: myapp_development
     cache:
       <<: *cache
       database: myapp_development_cache
     queue:
       <<: *queue
       database: myapp_development_queue
   
   test:
     <<: *default
     database: myapp_test
   
   production:
     primary:
       <<: *default
       url: <%= ENV['DATABASE_URL'] %>
     cache:
       <<: *cache
       url: <%= ENV['DATABASE_URL']&.+('_cache') %>
     queue:
       <<: *queue
       url: <%= ENV['DATABASE_URL']&.+('_queue') %>

   That works fine although I feel like it is a bit cluttered for a new app. Maybe I could simplify this but I am unsure how. I guess I could dynamically set the database names like database: "myapp_#{Rails.env}_queue") and hoist those definitions in &queue etc. Edit: that doesn't actually work when creating or dropping multiple DBs at the same time

3. Provide a command like bin/rails solid_queue:install:dev that sets up everything for your development environment (with a database.yml similar to point 2).

What do people think is the best way to play around with Solid Queue in development?

(I realize this question applies to Solid Cache too since it also uses a separate schema, so it may be more of a general Rails question. I can repost on the Rails forum or the Rails github discussions if needed)

[rosa]

Hey @jeromedalbert, good question! I think the best way to play around with Solid Queue (and Solid Cache) in development is to keep the databases separate as well.

I think your idea of an install command for dev or a commented out example is good, that'll help people for sure.