Shopify Multi-Store Sync App

This app enables seamless data synchronization across multiple Shopify stores, built using the Remix framework.

PS: This is a work in progress (WIP).

Purpose

This app was developed to solve a common challenge in multi-store Shopify management: maintaining consistent data across multiple stores belonging to the same client or employer. It eliminates the manual work and potential errors involved in:

• Updating product information across multiple stores (WIP)
• Maintaining consistent customer data (WIP)
• Syncing order histories for consolidated reporting (WIP)
• Keeping navigation menus aligned across stores
• Managing both one-time migrations and ongoing synchronization needs (WIP)

Key Features

Comprehensive Data Synchronization (WIP)

• Products (including variants, images, and metadata)
• Customers and their historical data
• Order histories
• Online store navigation menus
• Custom collections and smart collections

Flexible Sync Architecture

• Primary store to multiple secondary stores synchronization
• Support for both one-time migrations and continuous sync
• Configurable sync rules per store connection
• Real-time updates using Shopify webhooks (WIP)

Connection Management

• Easy setup of new store connections
• Granular control over which data types sync to each connected store (WIP)
• Connection health monitoring (WIP)
• Sync history and audit logs (WIP)

Quick start

Prerequisites

Before you begin, you'll need the following:

1. Node.js: Download and install it if you haven't already.
2. Shopify Partner Account: Create an account if you don't have one.
3. Test Store: Set up either a development store or a Shopify Plus sandbox store for testing your app.

Setup

If you used the CLI to create the template, you can skip this section.

Using yarn:

yarn install

Using npm:

npm install

Using pnpm:

pnpm install

Local Development

Using yarn:

yarn dev

Using npm:

npm run dev

Using pnpm:

pnpm run dev

Press P to open the URL to your app. Once you click install, you can start development.

Local development is powered by the Shopify CLI. It logs into your partners account, connects to an app, provides environment variables, updates remote config, creates a tunnel and provides commands to generate extensions.

Authenticating and querying data

To authenticate and query data you can use the shopify const that is exported from /app/shopify.server.js:

export async function loader({ request }) {
  const { admin } = await shopify.authenticate.admin(request);

  const response = await admin.graphql(`
    {
      products(first: 25) {
        nodes {
          title
          description
        }
      }
    }`);

  const {
    data: {
      products: { nodes },
    },
  } = await response.json();

  return json(nodes);
}

This template comes preconfigured with examples of:

1. Setting up your Shopify app in /app/shopify.server.ts
2. Querying data using Graphql. Please see: /app/routes/app._index.tsx.
3. Responding to mandatory webhooks in /app/routes/webhooks.tsx

Please read the documentation for @shopify/shopify-app-remix to understand what other API's are available.

Deployment

Application Storage

This template uses Prisma to store session data, by default using an SQLite database. The database is defined as a Prisma schema in prisma/schema.prisma.

This use of SQLite works in production if your app runs as a single instance. The database that works best for you depends on the data your app needs and how it is queried. You can run your database of choice on a server yourself or host it with a SaaS company. Here’s a short list of databases providers that provide a free tier to get started:

Database	Type	Hosters
MySQL	SQL	Digital Ocean, Planet Scale, Amazon Aurora, Google Cloud SQL
PostgreSQL	SQL	Digital Ocean, Amazon Aurora, Google Cloud SQL
Redis	Key-value	Digital Ocean, Amazon MemoryDB
MongoDB	NoSQL / Document	Digital Ocean, MongoDB Atlas

To use one of these, you can use a different datasource provider in your schema.prisma file, or a different SessionStorage adapter package.

Build

Remix handles building the app for you, by running the command below with the package manager of your choice:

Using yarn:

yarn build

Using npm:

npm run build

Using pnpm:

pnpm run build

Hosting

When you're ready to set up your app in production, you can follow our deployment documentation to host your app on a cloud provider like Heroku or Fly.io.

When you reach the step for setting up environment variables, you also need to set the variable NODE_ENV=production.

Hosting on Vercel

Using the Vercel Preset is recommended when hosting your Shopify Remix app on Vercel. You'll also want to ensure imports that would normally come from @remix-run/node are imported from @vercel/remix instead. Learn more about hosting Remix apps on Vercel here.

// vite.config.ts
import { vitePlugin as remix } from "@remix-run/dev";
import { defineConfig, type UserConfig } from "vite";
import tsconfigPaths from "vite-tsconfig-paths";
+ import { vercelPreset } from '@vercel/remix/vite';

installGlobals();

export default defineConfig({
  plugins: [
    remix({
      ignoredRouteFiles: ["**/.*"],
+     presets: [vercelPreset()],
    }),
    tsconfigPaths(),
  ],
});

Gotchas / Troubleshooting

Database tables don't exist

If you get this error:

The table `main.Session` does not exist in the current database.

You need to create the database for Prisma. Run the setup script in package.json using your preferred package manager.

Navigating/redirecting breaks an embedded app

Embedded Shopify apps must maintain the user session, which can be tricky inside an iFrame. To avoid issues:

1. Use Link from @remix-run/react or @shopify/polaris. Do not use <a>.
2. Use the redirect helper returned from authenticate.admin. Do not use redirect from @remix-run/node
3. Use useSubmit or <Form/> from @remix-run/react. Do not use a lowercase <form/>.

This only applies if you app is embedded, which it will be by default.

Non Embedded

Shopify apps are best when they are embedded into the Shopify Admin. This template is configured that way. If you have a reason to not embed your please make 2 changes:

1. Change the isEmbeddedApp prop to false for the AppProvider in /app/routes/app.jsx
2. Remove any use of App Bridge APIs (window.shopify) from your code
3. Update the config for shopifyApp in app/shopify.server.js. Pass isEmbeddedApp: false

OAuth goes into a loop when I change my app's scopes

If you change your app's scopes and authentication goes into a loop and fails with a message from Shopify that it tried too many times, you might have forgotten to update your scopes with Shopify. To do that, you can run the deploy CLI command.

Using yarn:

yarn deploy

Using npm:

npm run deploy

Using pnpm:

pnpm run deploy

My shop-specific webhook subscriptions aren't updated

If you are registering webhooks in the afterAuth hook, using shopify.registerWebhooks, you may find that your subscriptions aren't being updated.

Instead of using the afterAuth hook, the recommended approach is to declare app-specific webhooks in the shopify.app.toml file. This approach is easier since Shopify will automatically update changes to webhook subscriptions every time you run deploy (e.g: npm run deploy). Please read these guides to understand more:

1. app-specific vs shop-specific webhooks
2. Create a subscription tutorial

If you do need shop-specific webhooks, please keep in mind that the package calls afterAuth in 2 scenarios:

• After installing the app
• When an access token expires

During normal development, the app won't need to re-authenticate most of the time, so shop-specific subscriptions aren't updated. To force your app to update the subscriptions, you can uninstall and reinstall it in your development store. That will force the OAuth process and call the afterAuth hook.

Admin created webhook failing HMAC validation

Webhooks subscriptions created in the Shopify admin will fail HMAC validation. This is because the webhook payload is not signed with your app's secret key. There are 2 solutions:

1. Use app-specific webhooks defined in your toml file instead (recommended)
2. Create webhook subscriptions using the shopifyApp object.

Test your webhooks with the Shopify CLI or by triggering events manually in the Shopify admin(e.g. Updating the product title to trigger a PRODUCTS_UPDATE).

Incorrect GraphQL Hints

By default the graphql.vscode-graphql extension for VS Code will assume that GraphQL queries or mutations are for the Shopify Admin API. This is a sensible default, but it may not be true if:

1. You use another Shopify API such as the storefront API.
2. You use a third party GraphQL API.

in this situation, please update the .graphqlrc.ts config.

First parameter has member 'readable' that is not a ReadableStream.

See hosting on Vercel.

Admin object undefined on webhook events triggered by the CLI

When you trigger a webhook event using the Shopify CLI, the admin object will be undefined. This is because the CLI triggers an event with a valid, but non-existent, shop. The admin object is only available when the webhook is triggered by a shop that has installed the app.

Webhooks triggered by the CLI are intended for initial experimentation testing of your webhook configuration. For more information on how to test your webhooks, see the Shopify CLI documentation.

Using Defer & await for streaming responses

To test streaming using defer/await during local development you'll need to use the Shopify CLI slightly differently:

1. First setup ngrok: https://ngrok.com/product/secure-tunnels
2. Create an ngrok tunnel on port 8080: ngrok http 8080.
3. Copy the forwarding address. This should be something like: https://f355-2607-fea8-bb5c-8700-7972-d2b5-3f2b-94ab.ngrok-free.app
4. In a separate terminal run yarn shopify app dev --tunnel-url=TUNNEL_URL:8080 replacing TUNNEL_URL for the address you copied in step 3.

By default the CLI uses a cloudflare tunnel. Unfortunately it cloudflare tunnels wait for the Response stream to finish, then sends one chunk.

This will not affect production, since tunnels are only for local development.

Using MongoDB and Prisma

By default this template uses SQLlite as the database. It is recommended to move to a persisted database for production. If you choose to use MongoDB, you will need to make some modifications to the schema and prisma configuration. For more information please see the Prisma MongoDB documentation.

Alternatively you can use a MongDB database directly with the MongoDB session storage adapter.

Mapping the id field

In MongoDB, an ID must be a single field that defines an @id attribute and a @map("_id") attribute. The prisma adapter expects the ID field to be the ID of the session, and not the _id field of the document.

To make this work you can add a new field to the schema that maps the _id field to the id field. For more information see the Prisma documentation

model Session {
  session_id  String    @id @default(auto()) @map("_id") @db.ObjectId
  id          String    @unique
...
}

Error: The "mongodb" provider is not supported with this command

MongoDB does not support the prisma migrate command. Instead, you can use the prisma db push command and update the shopify.web.toml file with the following commands. If you are using MongoDB please see the Prisma documentation for more information.

[commands]
predev = "npx prisma generate && npx prisma db push"
dev = "npm exec remix vite:dev"

Prisma needs to perform transactions, which requires your mongodb server to be run as a replica set

See the Prisma documentation for connecting to a MongoDB database.

I want to use Polaris v13.0.0 or higher

Currently, this template is set up to work on node v18.20 or higher. However, @shopify/polaris is limited to v12 because v13 can only run on node v20+.

You don't have to make any changes to the code in order to be able to upgrade Polaris to v13, but you'll need to do the following:

• Upgrade your node version to v20.10 or higher.
• Update your Dockerfile to pull FROM node:20-alpine instead of node:18-alpine

Benefits

Shopify apps are built on a variety of Shopify tools to create a great merchant experience.

The Remix app template comes with the following out-of-the-box functionality:

• OAuth: Installing the app and granting permissions
• GraphQL Admin API: Querying or mutating Shopify admin data
• REST Admin API: Resource classes to interact with the API
• Webhooks: Callbacks sent by Shopify when certain events occur
• AppBridge: This template uses the next generation of the Shopify App Bridge library which works in unison with previous versions.
• Polaris: Design system that enables apps to create Shopify-like experiences

Tech Stack

This template uses Remix. The following Shopify tools are also included to ease app development:

• Shopify App Remix provides authentication and methods for interacting with Shopify APIs.
• Shopify App Bridge allows your app to seamlessly integrate your app within Shopify's Admin.
• Polaris React is a powerful design system and component library that helps developers build high quality, consistent experiences for Shopify merchants.
• Webhooks: Callbacks sent by Shopify when certain events occur
• Polaris: Design system that enables apps to create Shopify-like experiences

Resources

• Remix Docs
• Shopify App Remix
• Introduction to Shopify apps
• App authentication
• Shopify CLI
• App extensions
• Shopify Functions
• Getting started with internationalizing your app

Performance Considerations

Data Volume Management

• Batch processing for large data sets
• Rate limiting compliance with Shopify API
• Efficient data diffing to minimize API calls
• Background job processing for heavy sync operations

Sync Strategies

• Initial bulk sync for new store connections
• Incremental updates for ongoing synchronization
• Priority queuing for critical updates
• Conflict resolution handling

Database Schema

The app uses Prisma with the following key models (in addition to the session storage):

• StoreConnections: Manages relationships between primary and secondary stores
• SyncConfigs: Stores sync preferences per connection
• SyncHistory: Tracks all sync operations and their results
• DataMappings: Maintains relationships between objects across stores