Why not run Drizzle Migrations inside Dockerfile? Why a separate scripts/run.sh is used to run migrations?

[deadcoder0904]

This is a totally unrelated question to this project but there are very few resources on Docker + Drizzle Migrations so I thought I'd ask here where I actually found a solution.

So I have a project using Dockerfile that looks like:

FROM node:20-alpine AS base

# 1. Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat

WORKDIR /app

# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
RUN \
  if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  elif [ -f package-lock.json ]; then npm ci; \
  elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i; \
  else echo "Lockfile not found." && exit 1; \
  fi


# 2. Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules

COPY . .

# This will do the trick, use the corresponding env file for each environment.
COPY .env.production .env.production
RUN ls
RUN mkdir -p /data
RUN ls
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
# Generate migration files
RUN npm run db:generate
# COPY /src/app/db/migrations ./migrations
# Create /data/users.prod.sqlite using Volume Mount
RUN npm run db:migrate:prod
RUN ls
RUN npm run build

# 3. Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app

ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1

RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001

COPY --from=builder /app/public ./public

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

# Move the drizzle directory to the runtime image
COPY --from=builder --chown=nextjs:nodejs /app/src/app/db/migrations ./migrations

USER nextjs

EXPOSE 3000

# ENV PORT 3000
# ENV HOSTNAME localhost

CMD ["node", "server.js"]

The problem is my database is not migrated & no tables are created in production. In development, it works fine when I do pnpm db:generate && pnpm db:migrate but in production, I have put those statements above in RUN command & they do run but no *.sqlite is generated when npm run db:migrate:prod is run. Why?

I saw you guys use a scripts/run.sh & you run migrations in that file. Why does that work & this one doesn't?

I also try creating mkdir -p data in the container but even that doesn't happen. It looks like some commands don't work.

@anonysoul can you answer this one as i saw you solved this

also @Meierschlumpf as you have more related commits on Dockerfile :)

[Meierschlumpf]

The problem is, that when you run the migration in the dockerfile it will not run in the container and rather in the image

[deadcoder0904]

@Meierschlumpf that makes sense.

so i guess your process is to do next build locally & then build Dockerfile, right? because i don't see a next build command in Dockerfile. because my next question was gonna be my next build doesn't run without migrations command being run pnpm db:migrate:prod?

[Meierschlumpf]

We run the migration as Run statement so it runs when the container starts, other than that yes

[ajnart]

@deadcoder0904 iirc we had that script run because the underlying to run drizzle / prisma needs to match the processor architecture in order for the binary to be ran and including all the binaries inside of the image did not make much sense. the start script will directly get the correct executable and use it in that context.
As for the build we also build outside of docker because we can use the same build output for all our docker images. Originally the build time was proportional to the number of architectures we target

It's also easier to have dependencies caching and build cache in a GitHub action

[deadcoder0904]

@Meierschlumpf i found your solution a bit complicated & couldn't get it to work. i didn't understand how you installed package.json in migrations... I mean I did see yarn but I didn't understand how you used it outside of it but I found a simpler answer after a week of juggling with this. Its probably much simpler than yours I think. Take a look if you want & thanks for the answer. Your solution was a step in the right direction. Otherwise I would still be struggling.

EDIT: I came to the same conclusion as you guys. On my main repo, it was taking to long because node_modules that I was copying in the final stage was too big. So I created package.json just for migrations & came to the same conclusion as ya'll. Although my code does look more simpler & easier to read, its probably the same thing as yours. Updated the answer on StackOverflow as well.

[Meierschlumpf]

Okay great, I'll definitely gonna take a look