Skip to content

Commit

Permalink
docs: added batch jobs concept documentation (#2160)
Browse files Browse the repository at this point in the history
* added batch jobs concept doc

* added to sidebar
  • Loading branch information
shahednasser authored Sep 7, 2022
1 parent a71cf60 commit 7b98e6e
Show file tree
Hide file tree
Showing 3 changed files with 74 additions and 7 deletions.
61 changes: 61 additions & 0 deletions docs/content/advanced/backend/batch-jobs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Batch Jobs

In this document, you’ll learn what Batch Jobs are and how they work in Medusa.

## What are Batch Jobs?

Batch Jobs are tasks that can be performed asynchronously and iteratively. They can be [created using the Admin API](https://docs.medusajs.com/api/admin/#tag/Batch-Job/operation/PostBatchJobs), then, once confirmed, they are processed asynchronously.

Batch jobs require Redis, which Medusa uses as a queuing system to register and handle events. Every status change of a batch job triggers an event that can be handled using [subscribers](../subscribers/overview.md).

Medusa uses batch jobs in its core to perform some asynchronous tasks. For example, the Export Products functionality uses batch jobs.

You can also create a custom batch job or overwrite Medusa’s batch jobs.

### BatchJob Entity Overview

A batch job is stored in the database as a [BatchJob](https://docs.medusajs.com/references/entities/classes/BatchJob) entity. Some of its important attributes are:

- `status`: A string that determines the current status of the Batch Job.
- `context`: An object that can be used to store configurations related to the batch job. For example, you can use it to store listing configurations such as the limit or offset of the list to be retrieved during the processing of the batch job.
- `dry_run`: A boolean value that indicates whether this batch job should actually produce an output. By default it’s false, meaning that by default it produces an output. It can be used to simulate a batch job.
- `type`: A string that is used to later resolve the batch job strategy that should be used to handle the batch job.
- `result`: An object that includes important properties related to the result of the batch job. Some of these properties are:
- `errors`: An error object that contains any errors that occur during the batch job’s execution.
- `progress`: A numeric value indicating the progress of the batch job.
- `count`: A number that includes the total count of records related to the operation. For example, in the case of product exports, it is used to indicate the total number of products exported.
- `advancement_count`: A number that indicates the number of records processed so far. Can be helpful when retrying a batch job.

## What are Batch Job Strategies

Batch jobs are handled by batch job strategies. A batch job strategy is a class that extends the `AbstractBatchJobStrategy` abstract class and implements the methods defined in that class to handle the different states of a batch job.

A batch job strategy must implement the necessary methods to handle the preparation of a batch job before it is created, the preparation of the processing of the batch job after it is created, and the processing of the batch job once it is confirmed.

When you create a batch job strategy, the `batchType` class property indicates the batch job types this strategy handles. Then, when you create a new batch job, you set the batch job’s type to the value of `batchType` in your strategy.

## How Batch Jobs Work

A batch job’s flow from creation to completion is:

1. A batch job is created using the [Create Batch Job API endpoint](https://docs.medusajs.com/api/admin/#tag/Batch-Job/operation/PostBatchJobs).
2. Once the batch job is created, the batch job’s status is changed to `created` and the `batch.created` event is triggered by the `BatchJobService`.
3. The `BatchJobSubscriber` handles the `created` event. It resolves the batch job strategy based on the `type` of the batch job, then uses it to pre-process the batch job. After this, the batch job’s status is changed to `pre_processed`. Only when the batch job has the status `pre_processed` can be confirmed.
4. The batch job can be confirmed using the [Confirm Batch Job API](https://docs.medusajs.com/api/admin/#tag/Batch-Job/operation/PostBatchJobsBatchJobConfirmProcessing) endpoint.
5. Once the batch job is confirmed, the batch job’s status is changed to `confirmed` and the `batch.confirmed` event is triggered by the `BatchJobService`.
6. The `BatchJobSubscriber` handles the `confirmed` event. It resolves the batch job strategy, then uses it to process the batch job.
7. Once the batch job is processed succesfully, the batch job has the status `completed`.

You can track the progress of the batch job at any point using the [Retrieve Batch Job](https://docs.medusajs.com/api/admin/#tag/Batch-Job/operation/GetBatchJobsBatchJob) endpoint.

:::info

If the batch job fails at any point in this flow, its status is changed to `failed`.

:::

![Flowchart summarizing the batch job's flow from creation to completion](https://i.imgur.com/Qja0kAz.png)

## What’s Next?

- Learn about the [Batch Job’s events](../subscribers/events-list.md#batch-jobs-events).
4 changes: 3 additions & 1 deletion docs/content/advanced/backend/subscribers/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ In Medusa, there are events that are emitted when a certain action occurs. For e

The purpose of these events is to allow other parts of the platform, or third-party integrations, to listen to those events and perform a certain action. That is done by creating subscribers.

Medusa's queuing and events system is handled by Redis. So, you must have [Redis configured](../../../tutorial/0-set-up-your-development-environment.mdx#redis) on your server to use subscribers.

## What are Subscribers?

Subscribers register handlers for an events and allows you to perform an action when that event occurs. For example, if you want to send your customer an email when they place an order, then you can listen to the `order.placed` event and send the email when the event is emitted.
Expand All @@ -21,4 +23,4 @@ Whenever an event is emitted, the subscriber’s registered handler method is ex
## What's Next :rocket:

- Learn [how to create a Subscriber](create-subscriber.md).
- [View the list of all events](events-list.md).
- [View the list of all events](events-list.md).
16 changes: 10 additions & 6 deletions www/docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ module.exports = {
},
]
},

{
type: "category",
label: "Setup & Deployment",
Expand Down Expand Up @@ -250,6 +249,11 @@ module.exports = {
type: "category",
label: "Conceptual Guides",
items: [
{
type: "doc",
id: "advanced/backend/entities/overview",
label: "Entities"
},
{
type: "doc",
id: "advanced/backend/services/overview",
Expand All @@ -260,11 +264,6 @@ module.exports = {
id: "advanced/backend/subscribers/overview",
label: "Subscribers"
},
{
type: "doc",
id: "advanced/backend/entities/overview",
label: "Entities"
},
{
type: "doc",
id: "advanced/backend/shipping/overview",
Expand All @@ -290,6 +289,11 @@ module.exports = {
id: "advanced/backend/migrations/overview",
label: "Migrations"
},
{
type: "doc",
id: "advanced/backend/batch-jobs/index",
label: "Batch Jobs"
},
]
},
{
Expand Down

0 comments on commit 7b98e6e

Please sign in to comment.