Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core, medusa, cli): Enable migration scripts #10960

Merged
merged 18 commits into from
Jan 15, 2025

Conversation

adrien2p
Copy link
Member

@adrien2p adrien2p commented Jan 14, 2025

RESOLVES SUP-602
RESOLVES SUP-603

What
Enable the ability to create migration data scripts and execute them as part of the db:migrate, also allow to skip running the migration scripts using --skip-scripts.

How
We are looking for any file (except barrel files) inside the migration-scripts directory in medusa, the user project and the plugins.
We expect the files to export a default function as follow export default function migration1({ container }: { container: MedusaContainer }) => Promise<void>

The migration flow is the following

  • Ensure the script_migrations table exists otherwise create it
  • Look for pending migration scripts (All scripts that have not been ran previously)
  • take a lock
  • loop
    • insert script migration that will be executed
    • execute script
    • update finished_at
  • release lock

Copy link

changeset-bot bot commented Jan 14, 2025

🦋 Changeset detected

Latest commit: daac3a5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 65 packages
Name Type
@medusajs/medusa Patch
@medusajs/framework Patch
@medusajs/cli Patch
integration-tests-http Patch
@medusajs/test-utils Patch
@medusajs/api-key Patch
@medusajs/auth Patch
@medusajs/cache-inmemory Patch
@medusajs/cache-redis Patch
@medusajs/cart Patch
@medusajs/currency Patch
@medusajs/customer Patch
@medusajs/event-bus-local Patch
@medusajs/event-bus-redis Patch
@medusajs/file Patch
@medusajs/fulfillment Patch
@medusajs/index Patch
@medusajs/inventory Patch
@medusajs/link-modules Patch
@medusajs/locking Patch
@medusajs/notification Patch
@medusajs/order Patch
@medusajs/payment Patch
@medusajs/pricing Patch
@medusajs/product Patch
@medusajs/promotion Patch
@medusajs/region Patch
@medusajs/sales-channel Patch
@medusajs/stock-location Patch
@medusajs/store Patch
@medusajs/tax Patch
@medusajs/user Patch
@medusajs/workflow-engine-inmemory Patch
@medusajs/workflow-engine-redis Patch
@medusajs/auth-emailpass Patch
@medusajs/auth-github Patch
@medusajs/auth-google Patch
@medusajs/file-local Patch
@medusajs/file-s3 Patch
@medusajs/fulfillment-manual Patch
@medusajs/locking-postgres Patch
@medusajs/locking-redis Patch
@medusajs/notification-local Patch
@medusajs/notification-sendgrid Patch
@medusajs/payment-stripe Patch
@medusajs/core-flows Patch
@medusajs/js-sdk Patch
@medusajs/modules-sdk Patch
@medusajs/orchestration Patch
@medusajs/types Patch
@medusajs/utils Patch
@medusajs/workflows-sdk Patch
@medusajs/medusa-oas-cli Patch
@medusajs/oas-github-ci Patch
@medusajs/telemetry Patch
@medusajs/admin-bundler Patch
@medusajs/admin-sdk Patch
@medusajs/admin-shared Patch
@medusajs/admin-vite-plugin Patch
@medusajs/dashboard Patch
@medusajs/icons Patch
@medusajs/toolbox Patch
@medusajs/ui-preset Patch
create-medusa-app Patch
medusa-dev-cli Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

vercel bot commented Jan 14, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
medusa-dashboard ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 15, 2025 5:39pm
6 Skipped Deployments
Name Status Preview Comments Updated (UTC)
api-reference ⬜️ Ignored (Inspect) Jan 15, 2025 5:39pm
api-reference-v2 ⬜️ Ignored (Inspect) Visit Preview Jan 15, 2025 5:39pm
docs-ui ⬜️ Ignored (Inspect) Visit Preview Jan 15, 2025 5:39pm
docs-v2 ⬜️ Ignored (Inspect) Visit Preview Jan 15, 2025 5:39pm
medusa-docs ⬜️ Ignored (Inspect) Visit Preview Jan 15, 2025 5:39pm
resources-docs ⬜️ Ignored (Inspect) Visit Preview Jan 15, 2025 5:39pm

@adrien2p
Copy link
Member Author

/snapshot-this

@adrien2p
Copy link
Member Author

/snapshot-this

Copy link
Contributor

🚀 A snapshot release has been made for this PR

Test the snapshots by updating your package.json with the newly published versions:

yarn add @medusajs/admin-bundler@2.2.1-snapshot-20250114114841
yarn add @medusajs/admin-sdk@2.2.1-snapshot-20250114114841
yarn add @medusajs/admin-shared@2.2.1-snapshot-20250114114841
yarn add @medusajs/admin-vite-plugin@2.2.1-snapshot-20250114114841
yarn add @medusajs/dashboard@2.2.1-snapshot-20250114114841
yarn add create-medusa-app@2.2.1-snapshot-20250114114841
yarn add @medusajs/cli@2.2.1-snapshot-20250114114841
yarn add medusa-dev-cli@2.2.1-snapshot-20250114114841
yarn add @medusajs/medusa-oas-cli@2.2.1-snapshot-20250114114841
yarn add @medusajs/core-flows@2.2.1-snapshot-20250114114841
yarn add @medusajs/framework@2.2.1-snapshot-20250114114841
yarn add @medusajs/js-sdk@2.2.1-snapshot-20250114114841
yarn add @medusajs/modules-sdk@2.2.1-snapshot-20250114114841
yarn add @medusajs/orchestration@2.2.1-snapshot-20250114114841
yarn add @medusajs/types@2.2.1-snapshot-20250114114841
yarn add @medusajs/utils@2.2.1-snapshot-20250114114841
yarn add @medusajs/workflows-sdk@2.2.1-snapshot-20250114114841
yarn add @medusajs/icons@2.2.1-snapshot-20250114114841
yarn add @medusajs/ui-preset@2.2.1-snapshot-20250114114841
yarn add @medusajs/medusa@2.2.1-snapshot-20250114114841
yarn add @medusajs/telemetry@2.2.1-snapshot-20250114114841
yarn add @medusajs/test-utils@2.2.1-snapshot-20250114114841
yarn add @medusajs/api-key@2.2.1-snapshot-20250114114841
yarn add @medusajs/auth@2.2.1-snapshot-20250114114841
yarn add @medusajs/cache-inmemory@2.2.1-snapshot-20250114114841
yarn add @medusajs/cache-redis@2.2.1-snapshot-20250114114841
yarn add @medusajs/cart@2.2.1-snapshot-20250114114841
yarn add @medusajs/currency@2.2.1-snapshot-20250114114841
yarn add @medusajs/customer@2.2.1-snapshot-20250114114841
yarn add @medusajs/event-bus-local@2.2.1-snapshot-20250114114841
yarn add @medusajs/event-bus-redis@2.2.1-snapshot-20250114114841
yarn add @medusajs/file@2.2.1-snapshot-20250114114841
yarn add @medusajs/fulfillment@2.2.1-snapshot-20250114114841
yarn add @medusajs/index@2.2.1-snapshot-20250114114841
yarn add @medusajs/inventory@2.2.1-snapshot-20250114114841
yarn add @medusajs/link-modules@2.2.1-snapshot-20250114114841
yarn add @medusajs/locking@2.2.1-snapshot-20250114114841
yarn add @medusajs/notification@2.2.1-snapshot-20250114114841
yarn add @medusajs/order@2.2.1-snapshot-20250114114841
yarn add @medusajs/payment@2.2.1-snapshot-20250114114841
yarn add @medusajs/pricing@2.2.1-snapshot-20250114114841
yarn add @medusajs/product@2.2.1-snapshot-20250114114841
yarn add @medusajs/promotion@2.2.1-snapshot-20250114114841
yarn add @medusajs/auth-emailpass@2.2.1-snapshot-20250114114841
yarn add @medusajs/auth-github@2.2.1-snapshot-20250114114841
yarn add @medusajs/auth-google@2.2.1-snapshot-20250114114841
yarn add @medusajs/file-local@2.2.1-snapshot-20250114114841
yarn add @medusajs/file-s3@2.2.1-snapshot-20250114114841
yarn add @medusajs/fulfillment-manual@2.2.1-snapshot-20250114114841
yarn add @medusajs/locking-postgres@2.2.1-snapshot-20250114114841
yarn add @medusajs/locking-redis@2.2.1-snapshot-20250114114841
yarn add @medusajs/notification-local@2.2.1-snapshot-20250114114841
yarn add @medusajs/notification-sendgrid@2.2.1-snapshot-20250114114841
yarn add @medusajs/payment-stripe@2.2.1-snapshot-20250114114841
yarn add @medusajs/region@2.2.1-snapshot-20250114114841
yarn add @medusajs/sales-channel@2.2.1-snapshot-20250114114841
yarn add @medusajs/stock-location@2.2.1-snapshot-20250114114841
yarn add @medusajs/store@2.2.1-snapshot-20250114114841
yarn add @medusajs/tax@2.2.1-snapshot-20250114114841
yarn add @medusajs/user@2.2.1-snapshot-20250114114841
yarn add @medusajs/workflow-engine-inmemory@2.2.1-snapshot-20250114114841
yarn add @medusajs/workflow-engine-redis@2.2.1-snapshot-20250114114841

Latest commit: cf62a0a

@adrien2p adrien2p changed the title feat(core, medusa): Enable migration scripts feat(core, medusa, cli): Enable migration scripts Jan 14, 2025
@adrien2p
Copy link
Member Author

/snapshot-this

@adrien2p
Copy link
Member Author

/snapshot-this

@adrien2p
Copy link
Member Author

/snapshot-this

Copy link
Contributor

🚀 A snapshot release has been made for this PR

Test the snapshots by updating your package.json with the newly published versions:

yarn add @medusajs/admin-bundler@2.2.1-snapshot-20250114140512
yarn add @medusajs/admin-sdk@2.2.1-snapshot-20250114140512
yarn add @medusajs/admin-shared@2.2.1-snapshot-20250114140512
yarn add @medusajs/admin-vite-plugin@2.2.1-snapshot-20250114140512
yarn add @medusajs/dashboard@2.2.1-snapshot-20250114140512
yarn add create-medusa-app@2.2.1-snapshot-20250114140512
yarn add @medusajs/cli@2.2.1-snapshot-20250114140512
yarn add medusa-dev-cli@2.2.1-snapshot-20250114140512
yarn add @medusajs/medusa-oas-cli@2.2.1-snapshot-20250114140512
yarn add @medusajs/core-flows@2.2.1-snapshot-20250114140512
yarn add @medusajs/framework@2.2.1-snapshot-20250114140512
yarn add @medusajs/js-sdk@2.2.1-snapshot-20250114140512
yarn add @medusajs/modules-sdk@2.2.1-snapshot-20250114140512
yarn add @medusajs/orchestration@2.2.1-snapshot-20250114140512
yarn add @medusajs/types@2.2.1-snapshot-20250114140512
yarn add @medusajs/utils@2.2.1-snapshot-20250114140512
yarn add @medusajs/workflows-sdk@2.2.1-snapshot-20250114140512
yarn add @medusajs/icons@2.2.1-snapshot-20250114140512
yarn add @medusajs/ui-preset@2.2.1-snapshot-20250114140512
yarn add @medusajs/medusa@2.2.1-snapshot-20250114140512
yarn add @medusajs/telemetry@2.2.1-snapshot-20250114140512
yarn add @medusajs/test-utils@2.2.1-snapshot-20250114140512
yarn add @medusajs/api-key@2.2.1-snapshot-20250114140512
yarn add @medusajs/auth@2.2.1-snapshot-20250114140512
yarn add @medusajs/cache-inmemory@2.2.1-snapshot-20250114140512
yarn add @medusajs/cache-redis@2.2.1-snapshot-20250114140512
yarn add @medusajs/cart@2.2.1-snapshot-20250114140512
yarn add @medusajs/currency@2.2.1-snapshot-20250114140512
yarn add @medusajs/customer@2.2.1-snapshot-20250114140512
yarn add @medusajs/event-bus-local@2.2.1-snapshot-20250114140512
yarn add @medusajs/event-bus-redis@2.2.1-snapshot-20250114140512
yarn add @medusajs/file@2.2.1-snapshot-20250114140512
yarn add @medusajs/fulfillment@2.2.1-snapshot-20250114140512
yarn add @medusajs/index@2.2.1-snapshot-20250114140512
yarn add @medusajs/inventory@2.2.1-snapshot-20250114140512
yarn add @medusajs/link-modules@2.2.1-snapshot-20250114140512
yarn add @medusajs/locking@2.2.1-snapshot-20250114140512
yarn add @medusajs/notification@2.2.1-snapshot-20250114140512
yarn add @medusajs/order@2.2.1-snapshot-20250114140512
yarn add @medusajs/payment@2.2.1-snapshot-20250114140512
yarn add @medusajs/pricing@2.2.1-snapshot-20250114140512
yarn add @medusajs/product@2.2.1-snapshot-20250114140512
yarn add @medusajs/promotion@2.2.1-snapshot-20250114140512
yarn add @medusajs/auth-emailpass@2.2.1-snapshot-20250114140512
yarn add @medusajs/auth-github@2.2.1-snapshot-20250114140512
yarn add @medusajs/auth-google@2.2.1-snapshot-20250114140512
yarn add @medusajs/file-local@2.2.1-snapshot-20250114140512
yarn add @medusajs/file-s3@2.2.1-snapshot-20250114140512
yarn add @medusajs/fulfillment-manual@2.2.1-snapshot-20250114140512
yarn add @medusajs/locking-postgres@2.2.1-snapshot-20250114140512
yarn add @medusajs/locking-redis@2.2.1-snapshot-20250114140512
yarn add @medusajs/notification-local@2.2.1-snapshot-20250114140512
yarn add @medusajs/notification-sendgrid@2.2.1-snapshot-20250114140512
yarn add @medusajs/payment-stripe@2.2.1-snapshot-20250114140512
yarn add @medusajs/region@2.2.1-snapshot-20250114140512
yarn add @medusajs/sales-channel@2.2.1-snapshot-20250114140512
yarn add @medusajs/stock-location@2.2.1-snapshot-20250114140512
yarn add @medusajs/store@2.2.1-snapshot-20250114140512
yarn add @medusajs/tax@2.2.1-snapshot-20250114140512
yarn add @medusajs/user@2.2.1-snapshot-20250114140512
yarn add @medusajs/workflow-engine-inmemory@2.2.1-snapshot-20250114140512
yarn add @medusajs/workflow-engine-redis@2.2.1-snapshot-20250114140512

Latest commit: cf62a0a

Copy link
Contributor

🚀 A snapshot release has been made for this PR

Test the snapshots by updating your package.json with the newly published versions:

yarn add @medusajs/admin-bundler@2.2.1-snapshot-20250114144017
yarn add @medusajs/admin-sdk@2.2.1-snapshot-20250114144017
yarn add @medusajs/admin-shared@2.2.1-snapshot-20250114144017
yarn add @medusajs/admin-vite-plugin@2.2.1-snapshot-20250114144017
yarn add @medusajs/dashboard@2.2.1-snapshot-20250114144017
yarn add create-medusa-app@2.2.1-snapshot-20250114144017
yarn add @medusajs/cli@2.2.1-snapshot-20250114144017
yarn add medusa-dev-cli@2.2.1-snapshot-20250114144017
yarn add @medusajs/medusa-oas-cli@2.2.1-snapshot-20250114144017
yarn add @medusajs/core-flows@2.2.1-snapshot-20250114144017
yarn add @medusajs/framework@2.2.1-snapshot-20250114144017
yarn add @medusajs/js-sdk@2.2.1-snapshot-20250114144017
yarn add @medusajs/modules-sdk@2.2.1-snapshot-20250114144017
yarn add @medusajs/orchestration@2.2.1-snapshot-20250114144017
yarn add @medusajs/types@2.2.1-snapshot-20250114144017
yarn add @medusajs/utils@2.2.1-snapshot-20250114144017
yarn add @medusajs/workflows-sdk@2.2.1-snapshot-20250114144017
yarn add @medusajs/icons@2.2.1-snapshot-20250114144017
yarn add @medusajs/ui-preset@2.2.1-snapshot-20250114144017
yarn add @medusajs/medusa@2.2.1-snapshot-20250114144017
yarn add @medusajs/telemetry@2.2.1-snapshot-20250114144017
yarn add @medusajs/test-utils@2.2.1-snapshot-20250114144017
yarn add @medusajs/api-key@2.2.1-snapshot-20250114144017
yarn add @medusajs/auth@2.2.1-snapshot-20250114144017
yarn add @medusajs/cache-inmemory@2.2.1-snapshot-20250114144017
yarn add @medusajs/cache-redis@2.2.1-snapshot-20250114144017
yarn add @medusajs/cart@2.2.1-snapshot-20250114144017
yarn add @medusajs/currency@2.2.1-snapshot-20250114144017
yarn add @medusajs/customer@2.2.1-snapshot-20250114144017
yarn add @medusajs/event-bus-local@2.2.1-snapshot-20250114144017
yarn add @medusajs/event-bus-redis@2.2.1-snapshot-20250114144017
yarn add @medusajs/file@2.2.1-snapshot-20250114144017
yarn add @medusajs/fulfillment@2.2.1-snapshot-20250114144017
yarn add @medusajs/index@2.2.1-snapshot-20250114144017
yarn add @medusajs/inventory@2.2.1-snapshot-20250114144017
yarn add @medusajs/link-modules@2.2.1-snapshot-20250114144017
yarn add @medusajs/locking@2.2.1-snapshot-20250114144017
yarn add @medusajs/notification@2.2.1-snapshot-20250114144017
yarn add @medusajs/order@2.2.1-snapshot-20250114144017
yarn add @medusajs/payment@2.2.1-snapshot-20250114144017
yarn add @medusajs/pricing@2.2.1-snapshot-20250114144017
yarn add @medusajs/product@2.2.1-snapshot-20250114144017
yarn add @medusajs/promotion@2.2.1-snapshot-20250114144017
yarn add @medusajs/auth-emailpass@2.2.1-snapshot-20250114144017
yarn add @medusajs/auth-github@2.2.1-snapshot-20250114144017
yarn add @medusajs/auth-google@2.2.1-snapshot-20250114144017
yarn add @medusajs/file-local@2.2.1-snapshot-20250114144017
yarn add @medusajs/file-s3@2.2.1-snapshot-20250114144017
yarn add @medusajs/fulfillment-manual@2.2.1-snapshot-20250114144017
yarn add @medusajs/locking-postgres@2.2.1-snapshot-20250114144017
yarn add @medusajs/locking-redis@2.2.1-snapshot-20250114144017
yarn add @medusajs/notification-local@2.2.1-snapshot-20250114144017
yarn add @medusajs/notification-sendgrid@2.2.1-snapshot-20250114144017
yarn add @medusajs/payment-stripe@2.2.1-snapshot-20250114144017
yarn add @medusajs/region@2.2.1-snapshot-20250114144017
yarn add @medusajs/sales-channel@2.2.1-snapshot-20250114144017
yarn add @medusajs/stock-location@2.2.1-snapshot-20250114144017
yarn add @medusajs/store@2.2.1-snapshot-20250114144017
yarn add @medusajs/tax@2.2.1-snapshot-20250114144017
yarn add @medusajs/user@2.2.1-snapshot-20250114144017
yarn add @medusajs/workflow-engine-inmemory@2.2.1-snapshot-20250114144017
yarn add @medusajs/workflow-engine-redis@2.2.1-snapshot-20250114144017

Latest commit: cf62a0a

Copy link
Contributor

@carlos-r-l-rodrigues carlos-r-l-rodrigues left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

packages/core/framework/src/migrations/migrator.ts Outdated Show resolved Hide resolved
@adrien2p
Copy link
Member Author

@shahednasser let me know if I should add more info or anything for the doc if we do the doc :)

Copy link
Contributor

@olivermrbl olivermrbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM

Comment on lines +67 to +68
join(dirname(require.resolve("@medusajs/medusa")), "migration-scripts"),
...plugins.map((plugin) => join(plugin.resolve, "migration-scripts")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment: I don't love "two-word folders", so we should find a better name, however, I don't have any suggestions myself right now 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway, let's merge this now to unblock @fPolic's PR. We can revisit this later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it is not perfect I agree that it feels strange :)

Copy link
Contributor

@fPolic fPolic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

@olivermrbl
Copy link
Contributor

@shahednasser let me know if I should add more info or anything for the doc if we do the doc :)

Initially, we should focus on documenting how this is used in the core to run cross-module migration scripts, rather than what the user can do with this feature. Both are relevant, but the former is more important, since the scripts from our core will run without the user knowing.

@olivermrbl
Copy link
Contributor

@adrien2p, feel free to add automerge when tests are passing.

@adrien2p
Copy link
Member Author

@olivermrbl I suppose the CLI pipeline is failing but its ok right?

@olivermrbl olivermrbl merged commit 0924164 into develop Jan 15, 2025
23 checks passed
noubase pushed a commit to noubase/medusa that referenced this pull request Jan 22, 2025
* feat(core, medusa): Enable migration scripts

* spacing

* rm unnecessary import

* Allow to skip script migration

* fix missing options

* add options

* add tests and small changes

* update

* add checks

* add lock mechanism to be extra safe

* Create six-bears-vanish.md

* update queries

* fix tests

---------

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
jimrarras pushed a commit to jimrarras/medusa that referenced this pull request Jan 28, 2025
* feat(core, medusa): Enable migration scripts

* spacing

* rm unnecessary import

* Allow to skip script migration

* fix missing options

* add options

* add tests and small changes

* update

* add checks

* add lock mechanism to be extra safe

* Create six-bears-vanish.md

* update queries

* fix tests

---------

Co-authored-by: Oli Juhl <59018053+olivermrbl@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants