chore(infra): Faster docker build #17050
Conversation
WalkthroughThe pull request introduces significant changes to the Changes
Possibly related PRs
Suggested labels
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #17050 +/- ##
=======================================
Coverage 35.69% 35.69%
=======================================
Files 6939 6939
Lines 147080 147081 +1
Branches 41832 41832
=======================================
+ Hits 52497 52498 +1
Misses 94583 94583
Flags with carried forward coverage won't be shown. Click here to find out more. see 2 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
Datadog ReportAll test runs ✅ 5 Total Test Services: 0 Failed, 5 Passed Test Services
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Outside diff range and nitpick comments (11)
scripts/ci/Dockerfile (11)
5-7: SimplifyDOCKER_REGISTRYassignment for clarityThe current definition of
DOCKER_REGISTRYmight be unnecessarily complex. SinceDOCKER_ECR_REGISTRYis already defined aspublic.ecr.aws/docker, the parameter expansion${DOCKER_ECR_REGISTRY%/docker}/dockeressentially reconstructs the same string. Consider simplifying the assignment for better readability.Apply this diff to simplify the code:
-ARG DOCKER_REGISTRY=${DOCKER_ECR_REGISTRY%/docker}/docker +ARG DOCKER_REGISTRY=${DOCKER_ECR_REGISTRY}
33-35: Remove unnecessary debug commands to streamline the buildThe
echoandlscommands are used for debugging purposes but may clutter the build output in a CI environment. Consider removing them to improve build efficiency and reduce noise in the logs.Apply this diff to remove the debug commands:
-RUN echo "PWD: $PWD" && \ - ls -lah && \ - ls -lah -d node_modules/
51-52: Remove unnecessary debug commands to streamline the buildSimilar to earlier, the
echoandlscommands can be removed to clean up the build process.Apply this diff to remove the debug commands:
-RUN echo "PWD: $PWD" && \ - ls -lah
76-77: Use--productionflag withyarn installto install only production dependenciesIn a production environment, it's best practice to install only the necessary dependencies. Using the
--productionflag withyarn installhelps in reducing the image size and potential attack surface.Apply this diff to modify the
yarn installcommand:-RUN yarn install && \ +RUN yarn install --production && \ yarn cache clean
89-90: Ensure the user creation commands are optimizedCombining user and group creation into a single command can improve efficiency and clarity.
Apply this diff to optimize user and group creation:
-RUN addgroup runners && \ - adduser -D runner -G runners +RUN adduser -D -G runners runner
117-120: Remove debug commands from the production imageThe
echoandlscommands increase the size of the image and may expose sensitive information about the build environment. It's recommended to remove them from the production stages.Apply this diff to remove the debug commands:
-RUN echo "PWD: $PWD" && \ - ls -lah ./ && \ - ls -lah main.js
142-144: Remove debug commands from the production imageAs with previous stages, removing unnecessary commands helps keep the image clean.
Apply this diff to remove the debug commands:
-RUN echo "PWD: $PWD" && \ - ls -lah ./ && \ - ls -lah main.js
219-222: Consolidate Yarn configuration copying and directory creationYou can streamline the commands to reduce the number of layers and improve efficiency.
Apply this diff to consolidate the commands:
-COPY --chown=pwuser:pwuser .yarnrc.yml ./ -RUN mkdir ./.yarn -COPY --chown=pwuser:pwuser .yarn/releases ./.yarn/releases +COPY --chown=pwuser:pwuser .yarnrc.yml ./ +COPY --chown=pwuser:pwuser .yarn/ ./.yarn/
235-237: Remove debug commands from the production imageAs previously noted, removing unnecessary
echoandlscommands is recommended.Apply this diff to remove the debug commands:
-RUN echo "PWD: $PWD" && \ - ls -lah ./ && \ - ls -lah ./entrypoint.sh
265-267: Remove debug commands from the production imageConsistently remove these commands across all stages to maintain a clean Docker image.
Apply this diff to remove the debug commands:
-RUN echo "PWD: $PWD" && \ - ls -lah ./ && \ - ls -lah ./entrypoint.sh
272-273: Implement the output-native stage or remove if unnecessaryThe
output-nativestage currently contains aRUN echo "not-implemented"command. Decide whether to implement this stage or remove it to avoid confusion.If you need assistance in implementing the
output-nativestage, please let me know.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
scripts/ci/Dockerfile(3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
scripts/ci/Dockerfile (1)
Pattern scripts/**/*: "Confirm that the code adheres to the following:
- Script efficiency, readability, and maintainability.
- Usage of environmental variables and configuration files for flexibility.
- Integration with NX workspace utilities and commands."
🔇 Additional comments (1)
scripts/ci/Dockerfile (1)
66-68: Ensure correct copying of Yarn configurations and dependencies
Copying Yarn configurations and .yarn directories to the ${APP_DIST_HOME} is crucial for the subsequent yarn install to work properly. Double-check that the paths are correct and that all necessary files are included.
Run the following script to verify that the required Yarn files are present in ${APP_DIST_HOME}:
✅ Verification successful
Let me verify the existence of these Yarn-related files in the repository first.
Yarn configuration files are correctly set up and copied
The verification shows that all required Yarn files are present in the repository root and are being correctly copied to ${APP_DIST_HOME}:
yarn.lockand.yarnrc.ymlexist in the root.yarn/patchesdirectory contains necessary patch files.yarn/releasescontains the Yarn binary (yarn-3.2.3.cjs)
The Dockerfile correctly sets up APP_DIST_HOME as a build argument and the copy commands are properly structured to move these files to their required locations. The subsequent yarn install command in the Dockerfile will have access to all necessary configuration files.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify that Yarn configuration and necessary files are copied correctly.
# Expected: Files yarn.lock, .yarnrc.yml, and directories .yarn/patches, .yarn/releases exist in ${APP_DIST_HOME}
if [ -d "${APP_DIST_HOME}" ]; then
ls "${APP_DIST_HOME}/yarn.lock" "${APP_DIST_HOME}/.yarnrc.yml" "${APP_DIST_HOME}/.yarn/patches" "${APP_DIST_HOME}/.yarn/releases"
else
echo "Directory ${APP_DIST_HOME} does not exist."
fi
Length of output: 78
Script:
#!/bin/bash
# Check if the required Yarn files exist in the repository root
ls -la yarn.lock .yarnrc.yml .yarn/patches .yarn/releases 2>/dev/null || echo "Some files are missing"
# Also verify the content of the Dockerfile to ensure the APP_DIST_HOME variable is properly set
grep -A 5 "APP_DIST_HOME" scripts/ci/Dockerfile
Length of output: 4632
| # TODO: remove awscli dependency (157 MB extra) | ||
| # hadolint ignore=DL3008 | ||
| RUN apt-get update -y && apt-get install -y zip awscli && apt-get purge | ||
|
|
||
| ARG APP_HOME | ||
| ARG APP_DIST_HOME | ||
| RUN apt-get update -y && \ | ||
| apt-get install --no-install-recommends -y zip awscli && \ | ||
| apt-get purge |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Address the TODO: Remove awscli dependency to reduce image size
The TODO comment indicates that awscli adds significant weight to the image. Consider removing it or using a slimmer alternative to optimize the image size.
Here's a possible approach to remove awscli:
- Use the AWS SDK for JavaScript if AWS functionalities are required.
- If
awscliis needed for specific commands, consider installingawscli-liteor using AWS's official Docker images for CLI operations.
Let me know if you would like assistance in implementing this change.
| RUN apt-get update -y && \ | ||
| apt-get install --no-install-recommends -y zip awscli && \ | ||
| apt-get purge |
There was a problem hiding this comment.
Avoid purging apt-get immediately after installation
The apt-get purge command is removing the entire package management system, which may cause issues if further apt commands are needed. Instead, use apt-get clean to remove cached package files.
Apply this diff to correct the cleanup step:
-RUN apt-get update -y && \
- apt-get install --no-install-recommends -y zip awscli && \
- apt-get purge
+RUN apt-get update -y && \
+ apt-get install --no-install-recommends -y zip awscli && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN apt-get update -y && \ | |
| apt-get install --no-install-recommends -y zip awscli && \ | |
| apt-get purge | |
| RUN apt-get update -y && \ | |
| apt-get install --no-install-recommends -y zip awscli && \ | |
| apt-get clean && \ | |
| rm -rf /var/lib/apt/lists/* |
| COPY ${APP_HOME}/package.json . | ||
| RUN chown -R pwuser:pwuser . | ||
| COPY --chown=pwuser:pwuser --from=builder /build/${APP_DIST_HOME} ./ | ||
| COPY --chown=pwuser:pwuser ${APP_HOME}/package.json ./ |
There was a problem hiding this comment.
Ensure ${APP_HOME} is correctly defined before using in COPY
The COPY command uses ${APP_HOME}/package.json. Verify that ${APP_HOME} is set and points to the correct directory to prevent build issues.
Apply this diff to define APP_HOME if not already set:
+# Ensure APP_HOME is defined
+ARG APP_HOME=your_app_directoryReplace your_app_directory with the appropriate path.
Committable suggestion skipped: line range outside the PR's diff.
| RUN apk --update --no-cache add git python3 build-base gcc && \ | ||
| ln -sf /usr/bin/python3 /usr/bin/python && \ | ||
| wget -O /tmp/jq-linux64 https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 && \ | ||
| chmod a+x /tmp/jq-linux64 && \ | ||
| mv /tmp/jq-linux64 /usr/bin/jq |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Use package manager to install jq for consistency and security
Instead of manually downloading jq, it's preferable to use the package manager apk to install it. This ensures that you receive a verified and up-to-date version from the repository, enhancing security and maintainability.
Apply this diff to install jq via apk:
-RUN apk --update --no-cache add git python3 build-base gcc && \
- ln -sf /usr/bin/python3 /usr/bin/python && \
- wget -O /tmp/jq-linux64 https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 && \
- chmod a+x /tmp/jq-linux64 && \
- mv /tmp/jq-linux64 /usr/bin/jq
+RUN apk --update --no-cache add git python3 build-base gcc jq && \
+ ln -sf /usr/bin/python3 /usr/bin/python📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN apk --update --no-cache add git python3 build-base gcc && \ | |
| ln -sf /usr/bin/python3 /usr/bin/python && \ | |
| wget -O /tmp/jq-linux64 https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 && \ | |
| chmod a+x /tmp/jq-linux64 && \ | |
| mv /tmp/jq-linux64 /usr/bin/jq | |
| RUN apk --update --no-cache add git python3 build-base gcc jq && \ | |
| ln -sf /usr/bin/python3 /usr/bin/python |
|
Affected services are: Deployed services: . |
Depends on #17051 and #17048
DOCKER_REGISTRYfor easier local developmentCherry-picking from #16882 stuff that's out-of-scope there.
Summary by CodeRabbit