diff --git a/docs/images/run-lifecycle.png b/docs/images/run-lifecycle.png new file mode 100644 index 0000000000..0d5fb5503d Binary files /dev/null and b/docs/images/run-lifecycle.png differ diff --git a/docs/images/run-with-batchTriggerAndWait().png b/docs/images/run-with-batchTriggerAndWait().png new file mode 100644 index 0000000000..e71ae7813e Binary files /dev/null and b/docs/images/run-with-batchTriggerAndWait().png differ diff --git a/docs/images/run-with-delay.png b/docs/images/run-with-delay.png new file mode 100644 index 0000000000..ec290edf54 Binary files /dev/null and b/docs/images/run-with-delay.png differ diff --git a/docs/images/run-with-retries.png b/docs/images/run-with-retries.png new file mode 100644 index 0000000000..be614c6716 Binary files /dev/null and b/docs/images/run-with-retries.png differ diff --git a/docs/images/run-with-triggerAndWait().png b/docs/images/run-with-triggerAndWait().png new file mode 100644 index 0000000000..f4a8cc8bb6 Binary files /dev/null and b/docs/images/run-with-triggerAndWait().png differ diff --git a/docs/images/run-with-ttl.png b/docs/images/run-with-ttl.png new file mode 100644 index 0000000000..53291fb71e Binary files /dev/null and b/docs/images/run-with-ttl.png differ diff --git a/docs/mint.json b/docs/mint.json index 41b49b2a25..41f936223c 100644 --- a/docs/mint.json +++ b/docs/mint.json @@ -121,6 +121,7 @@ "pages": ["tasks/overview", "tasks/scheduled"] }, "triggering", + "runs-and-attempts", "apikeys", { "group": "Configuration", diff --git a/docs/runs-and-attempts.mdx b/docs/runs-and-attempts.mdx new file mode 100644 index 0000000000..c00f4131e6 --- /dev/null +++ b/docs/runs-and-attempts.mdx @@ -0,0 +1,187 @@ +--- +title: "Runs & attempts" +description: "Understanding the lifecycle of task execution in Trigger.dev" +--- + +In Trigger.dev, the concepts of runs and attempts are fundamental to understanding how tasks are executed and managed. This article explains these concepts in detail and provides insights into the various states a run can go through during its lifecycle. + +## What are runs? + +A run is created when you trigger a task (e.g. calling `yourTask.trigger({ foo: "bar" })`). It represents a single instance of a task being executed and contains the following key information: + +- A unique run ID +- The current status of the run +- The payload (input data) for the task +- Lots of other metadata + +## The run lifecycle + +A run can go through **various** states during its lifecycle. The following diagram illustrates a typical state transition where a single run is triggered and completes successfully: + +![Run Lifecycle](/images/run-lifecycle.png) + +Runs can also find themselves in lots of other states depending on what's happening at any given time. The following sections describe all the possible states in more detail. + +### Initial States + + **Waiting for deploy**: If a task is triggered before it has been deployed, the run enters this state and waits for the task to be deployed. + + **Delayed**: When a run is triggered with a delay, it enters this state until the specified delay period has passed. + + **Queued**: The run is ready to be executed and is waiting in the queue. + +### Execution States + + **Executing**: The task is currently running. + + **Reattempting**: The task has failed and is being retried. + + **Frozen**: Task has been frozen and is waiting to be resumed. + +### Final States + + **Completed**: The task has successfully finished execution. + + **Canceled**: The run was manually canceled by the user. + + **Failed**: The task has failed to complete successfully. + + **Timed out**: Task has failed because it exceeded its `maxDuration`. + + **Crashed**: The worker process crashed during execution (likely due to an Out of Memory error). + + **Interrupted**: In development mode, when the CLI is disconnected. + + **System failure**: An unrecoverable system error has occurred. + + **Expired**: The run's Time-to-Live (TTL) has passed before it could start executing. + +## Attempts + +An attempt represents a single execution of a task within a run. A run can have one or more attempts, depending on the task's retry settings and whether it fails. Each attempt has: + +- A unique attempt ID +- A status +- An output (if successful) or an error (if failed) + +When a task fails, it will be retried according to its retry settings, creating new attempts until it either succeeds or reaches the retry limit. + +![Run with retries](/images/run-with-retries.png) + +## Run completion + +A run is considered finished when: + +1. The last attempt succeeds, or +2. The task has reached its retry limit and all attempts have failed + +At this point, the run will have either an output (if successful) or an error (if failed). + +## Advanced run features + +### Idempotency Keys + +When triggering a task, you can provide an idempotency key to ensure the task is executed only once, even if triggered multiple times. This is useful for preventing duplicate executions in distributed systems. + +```javascript +yourTask.trigger({ foo: "bar" }, { idempotencyKey: "unique-key" }); +``` + +- If a run with the same idempotency key is already in progress, the new trigger will be ignored. +- If the run has already finished, the previous output or error will be returned. + +### Canceling runs + +You can cancel an in-progress run using the API or the dashboard: + +```ts +runs.cancel(runId); +``` + +When a run is canceled: + +– The task execution is stopped + +– The run is marked as canceled + +– The task will not be retried + +– Any in-progress child runs are also canceled + +### Time-to-live (TTL) + +You can set a TTL when triggering a run: + +```ts +yourTask.trigger({ foo: "bar" }, { ttl: "10m" }); +``` + +If the run hasn't started within the specified TTL, it will automatically expire. This is useful for time-sensitive tasks. Note that dev runs automatically have a 10-minute TTL. + +![Run with TTL](/images/run-with-ttl.png) + +### Delayed runs + +You can schedule a run to start after a specified delay: + +```ts +yourTask.trigger({ foo: "bar" }, { delay: "1h" }); +``` + +This is useful for tasks that need to be executed at a specific time in the future. + +![Run with delay](/images/run-with-delay.png) + +### Replaying runs + +You can create a new run with the same payload as a previous run: + +```ts +runs.replay(runId); +``` + +This is useful for re-running a task with the same input, especially for debugging or recovering from failures. The new run will use the latest version of the task. + +You can also replay runs from the dashboard using the same or different payload. Learn how to do this [here](/replaying). + +### Waiting for runs + +#### triggerAndWait() + +The `triggerAndWait()` function triggers a task and then lets you wait for the result before continuing. [Learn more about triggerAndWait()](/triggering#yourtask-triggerandwait). + +![Run with triggerAndWait](/images/run-with-triggerAndWait().png) + +#### batchTriggerAndWait() + +Similar to `triggerAndWait()`, the `batchTriggerAndWait()` function lets you batch trigger a task and wait for all the results [Learn more about batchTriggerAndWait()](/triggering#yourtask-batchtriggerandwait). + +![Run with batchTriggerAndWait](/images/run-with-batchTriggerAndWait().png) + +### Runs API + +The runs API provides methods to interact with and manage runs: + +```ts +// List all runs +runs.list(); + +// Get a specific run by ID +runs.retrieve(runId); + +// Replay a run +runs.replay(runId); + +// Reschedule a run +runs.reschedule(runId, delay); + +// Cancel a run +runs.cancel(runId); +``` + +These methods allow you to access detailed information about runs and their attempts, including payloads, outputs, parent runs, and child runs. + +### Triggering runs for undeployed tasks + +It's possible to trigger a run for a task that hasn't been deployed yet. The run will enter the "Waiting for deploy" state until the task is deployed. Once deployed, the run will be queued and executed normally. +This feature is particularly useful in CI/CD pipelines where you want to trigger tasks before the deployment is complete. diff --git a/docs/triggering.mdx b/docs/triggering.mdx index c8d524fb6e..e6ef4b61ce 100644 --- a/docs/triggering.mdx +++ b/docs/triggering.mdx @@ -6,7 +6,7 @@ description: "Tasks need to be triggered in order to run." Trigger tasks **from your backend**: | Function | This works | What it does | -| ------------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------------- | +| :----------------------- | :--------- | :-------------------------------------------------------------------------------------------------------------------------- | | `tasks.trigger()` | Anywhere | Triggers a task and gets a handle you can use to fetch and manage the run. [Read more](#tasks-trigger) | | `tasks.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to fetch and manage the runs. [Read more](#tasks-batchtrigger) | | `tasks.triggerAndPoll()` | Anywhere | Triggers a task and then polls the run until it’s complete. [Read more](#tasks-triggerandpoll) | @@ -14,11 +14,11 @@ Trigger tasks **from your backend**: Trigger tasks **from inside a run**: | Function | This works | What it does | -| -------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `yourTask.trigger()` | Anywhere | Triggers a task and gets a handle you can use to monitor and manage the run. It does not wait for the result. [Read more](#task-trigger) | -| `yourTask.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to monitor and manage the runs. It does not wait for the results. [Read more](#task-batchtrigger) | -| `yourTask.triggerAndWait()` | Inside task | Triggers a task and then waits until it's complete. You get the result data to continue with. [Read more](#task-triggerandwait) | -| `yourTask.batchTriggerAndWait()` | Inside task | Triggers a task multiple times in parallel and then waits until they're all complete. You get the resulting data to continue with. [Read more](#task-batchtriggerandwait) | +| :------------------------------- | :---------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `yourTask.trigger()` | Anywhere | Triggers a task and gets a handle you can use to monitor and manage the run. It does not wait for the result. [Read more](#yourtask-trigger) | +| `yourTask.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to monitor and manage the runs. It does not wait for the results. [Read more](#yourtask-batchtrigger) | +| `yourTask.triggerAndWait()` | Inside task | Triggers a task and then waits until it's complete. You get the result data to continue with. [Read more](#yourtask-triggerandwait) | +| `yourTask.batchTriggerAndWait()` | Inside task | Triggers a task multiple times in parallel and then waits until they're all complete. You get the resulting data to continue with. [Read more](#yourtask-batchtriggerandwait) | Additionally, [scheduled tasks](/tasks/scheduled) get **automatically** triggered on their schedule and webhooks when receiving a webhook.