feat: product import strategy

[fPolic]

What

• implemented product import strategy
• fix a bug where a product variant couldn't be created if an MA record with region_id was sent in the payload
• return a fileKey from MinIO file service (needs to be stored in the BatchJob context)
• useRedis testing util that spins up a Redis container for integration testing
• handling a batch job failure case in the subscriber
• add ioredis types

How

• In the preprocessing faze of a batch job a CSV file is parsed, validated and processed. From row data, 4 types of instructions for creating or updating products and variants are generadet and stored in helper files in a bucket.
• In the processing phase one set of instructions is pulled at a time, and each instruction is handled by the corresponding service method.

Testing

• unit testing of data structures that are stored in Redis after preprocessing
• integration snapshot test of a product endpoint response which contains products, variants, options and prices after a batch job has been completed

---

TODO

• expend on integration testing
• documentation/usage guide

Testing locally with Postman

1. POST http://localhost:9000/admin/auth

• set body content {"email": "admin@medusa-test.com", "password": "supersecret"} and type to JSON

2. POST http://localhost:9000/admin/uploads

• note: minio file service needs to be added to the store instance
• body should be a form-data where the CSV file should be uploaded under key files
• response will contain a key which is used by the file service to locate file e.g. "key": "product-import.csv"
• the key is needed for the next step

3. POST http://localhost:9000/admin/batch-jobs/

• send a request with following body to create a batch job and trigger a run:
  {"type": "product_import", "dry_run": false, "context": {"fileKey": "product-import.csv"}}

Fixes CORE-413

[changeset-bot]

⚠️ No Changeset found

Latest commit: f23b4e7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[srindom]

Nice work - will do one more pass on this soon; but have added a few comments here :)

[adrien2p]

Very nice work 👍 - I ve done a first review and will do another one once seb and my comments are processed. It is a super clean work man 🚀

[adrien2p]

Few comments, but very nice work 💪 I've also seen that there is still some of seb comments and that there is still some redis stuff, is that intended?

[fPolic]

there is still some of seb comments and that there is still some redis stuff, is that intended?

I made the change as Seb suggested. The import strategy doesn't depend on Redis anymore. Other Redis changes aren't related to the import strategy (I added type support for ioredis and useRedis testing utility).

The other unresolved comments still need to be discussed a bit more :)

[srindom]

Really nice work @frane - just a few small tweaks left; I assume that we will include the Sales Channel related stuff in a separate ticket?

[srindom]

todo: this is insufficient as we would have to get the correct product option for the product being worked on. For example, this could exist in the DB:

• Product Option 1: { id: "option_1", title: "Size", product_id: "prod_a" }
• Product Option 2: { id: "option_2", title: "Size", product_id: "prod_b" }

Now let's say we are adding a variant to Product B; we could end up having a situation where the variant has a product option value like this:

• Product Option Value: { id: "optval_1", value: "Medium", option_id: "option_1", variant_id: "new_variant" }

In this case, the option value refers to a ProductVariant belonging to Product B but a ProductOption belonging to Product A.

In the case where we are working with an existing product option we should be able to structure the query to be something along the lines of:

{ title: o._title, product_id: variantOp.product_id },

[srindom]

LGTM! Can't wait to start rolling this out 😍