🐛 Bug Report: Experiencing "Error: Cannot find module 'packages/backend/dist/index.cjs.js " #24808
Comments
|
Are you basing your build setup on https://backstage.io/docs/deployment/docker? It looks like you might've skipped the build step |
|
Patrick, thanks for your response. The build step is actually associated within my GitLab CI pipeline runner configuration in which the dockerfile that I added here is leveraged to pull down and install all the preqs (yarn, git, typescript, etc...) via Node.js v18.20.2 image. After that completes per the build stage, the following commands are then executed to build and push the new image with the resulting variables:
The "deploy" stage then leverage the new image per the CI_Registry and create the OpenShift service, deployment, and route configuration per yaml config files within the platform. As noted prior, I'm leveraging the "app-config.production.yaml" within my dockerfile "CMD" script to launch Backstage via production deployment which leads to the error that I'm currently experiencing. I will note that I've tested the "dev" deployment locally with the same code by running "yarn dev" and the application deployed successfully. I'm only hitting a snag when attempting to deploy via production. |
|
@deemanning right, but does that all use either of the docker build options outlined in https://backstage.io/docs/deployment/docker? |
|
I got the same issue, in my case, I had a misconfiguration on .dockerignore |
|
Hey Patrick, sorry for the late response. To answer your question, yes, I initially leveraged that exact article and the build options via my dockerfile config. The problem that I ran into with the documented build options was that a number of the directories associated with the build options no longer exist within the current Backstage binary file directories. The following is an example of an error that I run into with leveraging the provided build options: ERROR: failed to solve: failed to compute cache key: "/packages/backend/dist/bundle.tar.gz" not found: not found There is no "dist" directory within /packages per the download of the current Backstage application binary files. The version of Backstage that I'm leveraging is 1.27.1. So it appears that there are new code updates via the Backstage application but the documentation for Backstage deployment is not updated. Is there by chance an updated deployment doc that correlates with the 1.27.1 version? |
|
@deemanning You need to build the app first that's what's going to generate the
Depending on how you've setup the dockerfile, this command is run either in the container, or it's run on the host machine and then the outputs are mounted into the container. Which one are you trying to do? |
|
+1 we are having a similar issue: We're using the multi-step build from the docs. In the build stage: Then in the final stage: This started happening recently, so I'm wondering if it has to do with the Backstage version? |
|
I was able to reproduce it with and updated Backstage version: |
|
Hi @adamdabbracci, did you replicate it with your instance upgraded or a a new instance using create-app? If it was your instance can you share the full output of |
|
All using our existing instance, here's the full output: |
|
Also, potentially interesting to note, this doesn't happen if we run it in Yarn ( |
|
Ok, just created a fresh Backstage instance so So there's something about your instance that's off but I'm not sure what. Generally when I see multiple versions of the same package - like |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
I'm still troubleshooting this issue, unfortunately, I'm getting the same error as it still appears as though there are packages and directories that are missing per the backend module. |
|
I have a repro now as well. See #27503 |
Since @awanlin mentioned I then ran Unfortunately, I still reproduced the same error:
|
|
@cliffAtOsaic this command - |
I did run This is the command at the bottom of the Dockerfile. Is there a different CMD the Dockerfile should be running?
|
|
Hi @cliffAtOsaic, can we please keep this all in the issue you initially logged, trying to manage two issues is very challenging. |
Sorry, I had closed that one as a duplicate of this one hoping to simplify... I reopened the original and added new comments there. |
|
I'm also facing similar issues and would like to share what I've done to resolve it to this point. Let's say that I'm not a hero (yet) with dockerfiles, and I reckon that most people who face such an error (and can't solve) are also no warriors on this topic. I'm using the hardened wolfi-os image. See the docs Just implementing the dockerfile that is referenced, causes the following errors:
I also looked into the README.md that is provided with this hardened docker image, but no luck yet. Let's try to tackle the My dockerfile# This dockerfile builds an image for the backend package.
# It should be executed with the root of the repo as docker context.
#
# Before building this image, be sure to have run the following commands in the repo root:
#
# yarn install
# yarn tsc
# yarn build:backend
#
# Once the commands have been run, you can build the image using `yarn docker-build`
# syntax = docker/dockerfile:1.4
# Build Python environment in a separate builder stage
FROM cgr.dev/chainguard/python:latest-dev as python-builder
ENV PATH=/venv/bin:$PATH
RUN --mount=type=cache,target=/home/nonroot/.cache/pip,uid=65532,gid=65532 \
python3 -m venv /home/nonroot/venv && \
/home/nonroot/venv/bin/pip install mkdocs-techdocs-core==1.3.3
# Build Node environment in a separate builder stage
FROM cgr.dev/chainguard/wolfi-base:latest as node-builder
ENV NODE_VERSION="18=~18.16"
ENV NODE_ENV=production
RUN --mount=type=cache,target=/var/cache/apk,sharing=locked,uid=65532,gid=65532 \
--mount=type=cache,target=/var/lib/apk,sharing=locked,uid=65532,gid=65532 \
apk update && \
apk add nodejs-$NODE_VERSION yarn \
# Install isolate-vm dependencies, these are needed by the @backstage/plugin-scaffolder-backend.
openssl-dev brotli-dev c-ares-dev nghttp2-dev icu-dev zlib-dev gcc-12 libuv-dev build-base
WORKDIR /app
RUN chown -R nonroot:nonroot /app
RUN mkdir -p /home/nonroot/.yarn/berry && chown -R 65532:65532 /home/nonroot/.yarn/berry
USER nonroot
COPY --chown=65532:65532 .yarn ./.yarn
COPY --chown=65532:65532 .yarnrc.yml ./
# Added by me
RUN yarn tsc
RUN yarn --cwd packages/backend build && ls -l packages/backend/dist
# Added by me
COPY --chown=65532:65532 yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz
RUN --mount=type=cache,target=/home/nonroot/.yarn/berry/cache,sharing=locked,uid=65532,gid=65532 \
yarn workspaces focus --all --production && yarn cache clean --all
# Final stage to build the application image
FROM cgr.dev/chainguard/wolfi-base:latest
ENV PYTHON_VERSION="3.12=~3.12"
ENV NODE_VERSION="18=~18.16"
ENV NODE_ENV=production
RUN --mount=type=cache,target=/var/cache/apk,sharing=locked,uid=65532,gid=65532 \
--mount=type=cache,target=/var/lib/apk,sharing=locked,uid=65532,gid=65532 \
apk update && \
apk add \
# add node for backstage
nodejs-$NODE_VERSION \
# add python for backstage techdocs
python-$PYTHON_VERSION \
# add tini for init process
tini
WORKDIR /app
COPY --chown=65532:65532 package.json app-config.yaml ./
RUN chown -R 65532:65532 /app
RUN chown -R 65532:65532 /tmp
USER 65532:65532
COPY --from=node-builder --chown=65532:65532 /app/packages/backend/dist /app/packages/backend/dist
COPY --from=node-builder --chown=65532:65532 /app/node_modules ./node_modules
COPY --from=python-builder --chown=65532:65532 /home/nonroot/venv /home/nonroot/venv
ENV PATH=/home/nonroot/venv/bin:$PATH
ENV NODE_OPTIONS="--no-node-snapshot"
ENV GIT_PYTHON_REFRESH="quiet"
ENTRYPOINT ["tini", "--"]
CMD ["node", "packages/backend", "--config", "app-config.yaml"]How I tried resolving thisStep 1While searching this issue I got immediately directed to this issue, and saw what @benjdlambert suggested to resolve this issue.
I noticed that the dockerfile from the docs doesn't use the yarn commands mentioned above, so I added them to my dockerfile (see "My dockerfile"). However the missing /packages/backend/dists still isn't found and the same error Occurs Step 2So the next question: Is it not generated by my yarn commands or does the COPY command below it find nothing? I ran the It does generate what I need!: So why is it not the same behaviour when I build my Dockerfile (In my case via a pipeline. Not locally) Step 3Now I went on to take a look at my .gitignore and .dockerignore files My gitignore# macOS
.DS_Store
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
# Coverage directory generated when running tests with coverage
coverage
# Dependencies
node_modules/
# Yarn 3 files
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
# Node version directives
.nvmrc
# dotenv environment variables file
.env
.env.test
# Build output
dist
dist-types
# Temporary change files created by Vim
*.swp
# MkDocs build output
site
# Local configuration files
*.local.yaml
# Sensitive credentials
*-credentials.yaml
*-secrets.yaml
# vscode database functionality support files
*.session.sql
# E2E test reports
e2e-test-report/My dockerignore (same as Multi-stage build docs)dist-types
node_modules
packages/*/dist
packages/*/node_modules
plugins/*/dist
plugins/*/node_modulesSo there are two things I notice here:
Lets investigate changing the dockerignore. Step 4My changed dockerignorenode_modules
packages/*/node_modules
plugins/*/dist
plugins/*/node_modulesI stell get the same error at: So it doesn't seem that the dockerignore is the problem. Step 5Lets try to push my local My changed gitignore# macOS
.DS_Store
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
# Coverage directory generated when running tests with coverage
coverage
# Dependencies
node_modules/
# Yarn 3 files
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
# Node version directives
.nvmrc
# dotenv environment variables file
.env
.env.test
# Temporary change files created by Vim
*.swp
# MkDocs build output
site
# Local configuration files
*.local.yaml
# Sensitive credentials
*-credentials.yaml
*-secrets.yaml
# vscode database functionality support files
*.session.sql
# E2E test reports
e2e-test-report/I removed the build output section from the gitignore and committed the git files I generated locally with my If the COPY of the /dist doesn't work now, I don't know why. Step 6Yes finally. Some success. The COPY of the /dist directory seems to go correctly now, even tho it's not the way I would do it. (The yarn command in a dockerfile would be way better and is in my opinion preferred). Now only the Final sectionMy approach to solving these issues feel very counter intuitive. Please reach out if you have a better solution for generating the necessary yarn and dist files from the dockerfile to achieve building a container while still having clean source code. |
|
Is this issue resolve for anyone? I am getting same error with latest version of backstage on node-18 |
|
From what I know the latest Backstage version only supports node 20 and node 22 I resolved my comment from above by updating the OS image to node 20 and my yarn to yarn 4. At least it caused a creation of the necesarry files needed by the docker copy command. What you also always can try is a deletion of the node_modules and yarn.lock and then rerun your yarn install commands |
|
I'm experiencing this error as well, using the multi-stage-build Dockerfile Node - 20.0.9 I got a similar output to @twanbeeren's step 2 resolution, however when uncompressing these files it puts the index.cjs.js a few layers deeper. RUN yarn tsc
RUN yarn --cwd packages/backend build
RUN mkdir packages/backend/dist/skeleton packages/backend/dist/bundle \
&& tar xzf packages/backend/dist/skeleton.tar.gz -C packages/backend/dist/skeleton \
&& tar xzf packages/backend/dist/bundle.tar.gz -C packages/backend/dist/bundleUpdating |
|
@eddiehale3 the contents of both |
📜 Description
I'm currently experiencing the following when attempting to deploy Backstage via CI/CD pipeline per dockerfile leveraging a Node.js v18.20.2 image:
node:internal/modules/cjs/loader:451
throw err;
^
Error: Cannot find module '/appdir/packages/backend/dist/index.cjs.js'. Please verify that the package.json has a valid "main" entry
at tryPackage (node:internal/modules/cjs/loader:443:19)
at Module._findPath (node:internal/modules/cjs/loader:711:18)
at resolveMainPath (node:internal/modules/run_main:25:23)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:121:24)
at node:internal/main/run_main_module:28:49 {
code: 'MODULE_NOT_FOUND',
path: '/appdir/packages/backend/package.json',
requestPath: '/appdir/packages/backend'
}
Node.js v18.20.2
Within the same GitLab project, the application deploys successfully locally. Along with the image reference via FROM, the dockerfile is associated with the following config:
#Install all required utilities and packages
USER 0
RUN microdnf update -y && microdnf install -y git && microdnf install -y python3 && microdnf clean all
RUN microdnf install -y sqlite sqlite-devel && microdnf clean all
USER 1001
#Create node modules dir
USER 0
RUN mkdir -p /.cache/node/corepack/v1 && chmod -R 777 /.cache
USER 1001
#Set the specific yarn version
USER 0
RUN yarn set version 1.22.22
#Install dependencies using yarn
RUN npm install -g yarn
RUN mkdir ~/.yarn-global
RUN yarn config set prefix ~/.yarn-global
USER 1001
USER 0
RUN npm install -g typescript
RUN npm install -g npm
USER 1001
Copy app code to root
COPY --chown=1001:0 . .
EXPOSE 3000
Launch backstage app
CMD ["node", "packages/backend", "--config", "app-config.production.yaml"]
After running the following to ensure that I have the latest versions, I'm still hitting the same issue:
npx @backstage/cli versions:bump
yarn backstage-cli info
yarn run v1.22.22
OS: Darwin 23.4.0 - darwin/arm64
node: v18.20.2
yarn: 1.22.22
cli: 0.26.5 (installed)
backstage: 1.27.1
Dependencies:
@backstage/app-defaults 1.5.5
@backstage/backend-app-api 0.7.4
@backstage/backend-common 0.22.0
@backstage/backend-defaults 0.2.18
@backstage/backend-dev-utils 0.1.4
@backstage/backend-openapi-utils 0.1.11
@backstage/backend-plugin-api 0.6.18
@backstage/backend-tasks 0.5.23
@backstage/catalog-client 1.6.5
@backstage/catalog-model 1.5.0
@backstage/cli-common 0.1.13
@backstage/cli-node 0.2.5
@backstage/cli 0.26.5
@backstage/config-loader 1.8.0
@backstage/config 1.2.0
@backstage/core-app-api 1.12.5
@backstage/core-compat-api 0.2.5
@backstage/core-components 0.14.7
@backstage/core-plugin-api 1.9.2
@backstage/e2e-test-utils 0.1.1
@backstage/errors 1.2.4
@backstage/eslint-plugin 0.1.8
@backstage/frontend-plugin-api 0.6.5
@backstage/integration-aws-node 0.1.12
@backstage/integration-react 1.1.27
@backstage/integration 1.11.0
@backstage/plugin-api-docs 0.11.5
@backstage/plugin-app-backend 0.3.66
@backstage/plugin-app-node 0.1.18
@backstage/plugin-auth-backend-module-atlassian-provider 0.1.10
@backstage/plugin-auth-backend-module-aws-alb-provider 0.1.10
@backstage/plugin-auth-backend-module-azure-easyauth-provider 0.1.1
@backstage/plugin-auth-backend-module-bitbucket-provider 0.1.1
@backstage/plugin-auth-backend-module-cloudflare-access-provider 0.1.1
@backstage/plugin-auth-backend-module-gcp-iap-provider 0.2.13
@backstage/plugin-auth-backend-module-github-provider 0.1.15
@backstage/plugin-auth-backend-module-gitlab-provider 0.1.15
@backstage/plugin-auth-backend-module-google-provider 0.1.15
@backstage/plugin-auth-backend-module-guest-provider 0.1.4
@backstage/plugin-auth-backend-module-microsoft-provider 0.1.13
@backstage/plugin-auth-backend-module-oauth2-provider 0.1.15
@backstage/plugin-auth-backend-module-oauth2-proxy-provider 0.1.11
@backstage/plugin-auth-backend-module-oidc-provider 0.1.9
@backstage/plugin-auth-backend-module-okta-provider 0.0.11
@backstage/plugin-auth-backend 0.22.5
@backstage/plugin-auth-node 0.4.13
@backstage/plugin-auth-react 0.1.2
@backstage/plugin-catalog-backend-module-scaffolder-entity-model 0.1.16
@backstage/plugin-catalog-backend 1.22.0
@backstage/plugin-catalog-common 1.0.23
@backstage/plugin-catalog-graph 0.4.5
@backstage/plugin-catalog-import 0.11.0
@backstage/plugin-catalog-node 1.12.0
@backstage/plugin-catalog-react 1.12.0
@backstage/plugin-catalog 1.20.0
@backstage/plugin-events-node 0.3.4
@backstage/plugin-org 0.6.25
@backstage/plugin-permission-backend-module-allow-all-policy 0.1.15
@backstage/plugin-permission-backend 0.5.42
@backstage/plugin-permission-common 0.7.13
@backstage/plugin-permission-node 0.7.29
@backstage/plugin-permission-react 0.4.22
@backstage/plugin-proxy-backend 0.4.16
@backstage/plugin-scaffolder-backend-module-azure 0.1.10
@backstage/plugin-scaffolder-backend-module-bitbucket-cloud 0.1.8
@backstage/plugin-scaffolder-backend-module-bitbucket-server 0.1.8
@backstage/plugin-scaffolder-backend-module-bitbucket 0.2.8
@backstage/plugin-scaffolder-backend-module-gerrit 0.1.10
@backstage/plugin-scaffolder-backend-module-gitea 0.1.8
@backstage/plugin-scaffolder-backend-module-github 0.2.8
@backstage/plugin-scaffolder-backend-module-gitlab 0.4.0
@backstage/plugin-scaffolder-backend 1.22.6
@backstage/plugin-scaffolder-common 1.5.2
@backstage/plugin-scaffolder-node 0.4.4
@backstage/plugin-scaffolder-react 1.8.5
@backstage/plugin-scaffolder 1.20.0
@backstage/plugin-search-backend-module-catalog 0.1.24
@backstage/plugin-search-backend-module-techdocs 0.1.23
@backstage/plugin-search-backend-node 1.2.22
@backstage/plugin-search-backend 1.5.8
@backstage/plugin-search-common 1.2.11
@backstage/plugin-search-react 1.7.11
@backstage/plugin-search 1.4.11
@backstage/plugin-techdocs-backend 1.10.5
@backstage/plugin-techdocs-module-addons-contrib 1.1.10
@backstage/plugin-techdocs-node 1.12.4
@backstage/plugin-techdocs-react 1.2.4
@backstage/plugin-techdocs 1.10.5
@backstage/plugin-user-settings 0.8.6
@backstage/release-manifests 0.0.11
@backstage/repo-tools 0.9.0
@backstage/test-utils 1.5.5
@backstage/theme 0.5.4
@backstage/types 1.1.1
@backstage/version-bridge 1.0.8
👍 Expected behavior
Successful deployment via CI/CD pipeline deployment
👎 Actual Behavior with Screenshots
👟 Reproduction steps
Details are provided within the description.
📃 Provide the context for the Bug.
No response
🖥️ Your Environment
No response
👀 Have you spent some time to check if this bug has been raised before?
🏢 Have you read the Code of Conduct?
Are you willing to submit PR?
None
The text was updated successfully, but these errors were encountered: