[FAQ] Kestra Concurrency Control

[kkh1902]

Course

data-engineering-zoomcamp

Question

What does concurrency mean in Kestra, and how does it work?

Answer

In Kestra, concurrency means controlling how many executions of the same flow can run at the same time.

It is used to prevent problems such as duplicate processing, data corruption, or excessive resource usage when a flow is triggered multiple times.

---

Concurrency Limit

concurrency:
  limit: 1

This means only one execution of the flow can run at a time.

If the flow is already running and another execution is triggered, Kestra applies the behavior defined below.

---

Concurrency Behavior Options

FAIL

concurrency:
  behavior: FAIL
  limit: 1

• If the limit is exceeded, the new execution fails immediately.
• Useful when overlapping executions must never happen.

---

QUEUE

concurrency:
  behavior: QUEUE
  limit: 1

• Extra executions are queued and wait until the running one finishes.
• Useful for sequential batch jobs.

---

CANCEL

concurrency:
  behavior: CANCEL
  limit: 1

• The currently running execution is cancelled, and the new one starts.
• Useful when only the latest execution matters.

Checklist

• I have searched existing FAQs and this question is not already answered
• The answer provides accurate, helpful information
• I have included any relevant code examples or links

[github-actions]

✅ FAQ NEW proposal created in PR #69

Please review and approve the changes.