Code

Introduce concurrency controls#38

Code

Merged

rosa merged 32 commits intomainrails/solid_queue:mainfrom concurrency-controlsrails/solid_queue:concurrency-controlsCopy head branch name to clipboardNov 27, 2023

+700-122

Commits

Commits on Nov 22, 2023

• Implement a very basic concurrency control API for ActiveJob

  Show description for e1c7884
  rosa

  committed
  e1c7884

  Copy full SHA for e1c7884

• Implement basic concurrency control on enqueuing using semaphores

  Show description for f405340
  rosa

  committed
  f405340

  Copy full SHA for f405340

• Move concurrency control unit tests to job model test

  Show description for 4ce223a
  rosa

  committed
  4ce223a

  Copy full SHA for 4ce223a

• Refactor slightly how executions assume attributes from job

  Show description for 6050088
  rosa

  committed
  6050088

  Copy full SHA for 6050088

• Dispatch blocked executions when a concurrency-limited job finishes

  Show description for 5c14277
  rosa

  committed
  5c14277

  Copy full SHA for 5c14277

• Test concurrency limited to more than 1 job of a kind

  rosa

  committed
  e217ae0

  Copy full SHA for e217ae0

• Dispatch blocked jobs after job perform finishes outside that transaction

  Show description for 873760d
  rosa

  committed
  873760d

  Copy full SHA for 873760d

• Avoid race-conditions when releasing blocked executions

  Show description for d0eeed2
  rosa

  committed
  d0eeed2

  Copy full SHA for d0eeed2

• Correct transactions to release blocked executions

  Show description for 9fa0aa2
  rosa

  committed
  9fa0aa2

  Copy full SHA for 9fa0aa2

• Unblock releasable blocked executions before polling

  Show description for 7ab243d
  rosa

  committed
  7ab243d

  Copy full SHA for 7ab243d

• Consider blocked executions without a semaphore as releasable

  Show description for 04dd571
  rosa

  committed
  04dd571

  Copy full SHA for 04dd571

• Tweak concurrency related tables

  Show description for 7814338
  rosa

  committed
  7814338

  Copy full SHA for 7814338

• Delegate concurrency limit and duration to job class and store expires_at

  Show description for 016674b
  rosa

  committed
  016674b

  Copy full SHA for 016674b

• Include concurrency controls extension directly in ActiveJob

  rosa

  committed
  f22a46c

  Copy full SHA for f22a46c

• Use a concurrent timer task to expire semaphores and unblock executions

  Show description for ede2bfb
  rosa

  committed
  ede2bfb

  Copy full SHA for ede2bfb

• Don't instantiate job class to check concurrency limit if the key is not set

  rosa

  committed
  8484ca8

  Copy full SHA for 8484ca8

• Rename Semaphore.concurrency_key to Semaphore.key and simplify method sigantures

  Show description for 91256ee
  rosa

  committed
  91256ee

  Copy full SHA for 91256ee

• Improve a bit some execution methods (DRY and order)

  rosa

  committed
  9f80e5f

  Copy full SHA for 9f80e5f

• Rename concurrency_limit_duration to concurrency_duration

  rosa

  committed
  8993dd9

  Copy full SHA for 8993dd9

• Remove unused scope

  rosa

  committed
  cfaf02f

  Copy full SHA for cfaf02f

• Rename concurrency limit spec to limits_concurrency to: limit ...

  Show description for 6e14ea0
  rosa

  committed
  6e14ea0

  Copy full SHA for 6e14ea0

• Remove unused method, replaced by a scope

  rosa

  committed
  7c3007a

  Copy full SHA for 7c3007a

Commits on Nov 23, 2023

• Fix index on blocked_executions for releasing and add index on semaphores

  Show description for 363ae88
  rosa

  committed
  363ae88

  Copy full SHA for 363ae88

• Use intermediate Proxy object for the Semaphore low-level actions

  Show description for fbef301
  rosa

  committed
  fbef301

  Copy full SHA for fbef301

• Add support for concurrency controls across jobs (aka. concurrency "groups")

  Show description for 427482e
  rosa

  committed
  427482e

  Copy full SHA for 427482e

• Don't have a default key for concurrency controls, force always a choice there

  Show description for 93eb7f8
  rosa

  committed
  93eb7f8

  Copy full SHA for 93eb7f8

• Use a separate config option for concurrency maintenance task interval

  rosa

  committed
  ee5fcc6

  Copy full SHA for ee5fcc6

• Remove unused option in test helper and unused test helper

  rosa

  committed
  feef30b

  Copy full SHA for feef30b

• Store expires_at time in blocked_executions

  Show description for 9d0da03
  rosa

  committed
  9d0da03

  Copy full SHA for 9d0da03

• Add a few style related fixes across the board

  rosa

  committed
  49f7d13

  Copy full SHA for 49f7d13

Commits on Nov 26, 2023

• Rely on blocked_executions.expires_at to select candidates to be released

  Show description for 77b67b0
  rosa

  committed
  77b67b0

  Copy full SHA for 77b67b0

Commits on Nov 27, 2023

• Fix bug when selecting releasable executions and make sure these are reported

  Show description for 8a5de86
  rosa

  committed
  8a5de86

  Copy full SHA for 8a5de86