Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(web): dockerize web application #1222

Merged
merged 1 commit into from
Nov 6, 2024
Merged

ci(web): dockerize web application #1222

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 73 additions & 2 deletions .github/workflows/build_web.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ jobs:
workflow: deploy-web-nightly
token: ${{ steps.app-token.outputs.token }}

build-docker-image:
# TODO: Remove later after migrating to Cloud Run
build-docker-image-old:
KeisukeYamashita marked this conversation as resolved.
Show resolved Hide resolved
name: Build and push Docker image
runs-on: ubuntu-latest
needs: [release-nightly]
Expand All @@ -105,7 +106,7 @@ jobs:
IMAGE_NAME: reearth/reearth-visualizer
defaults:
run:
working-directory: server
working-directory: web
steps:
- name: Checkout
uses: actions/checkout@v3
Expand Down Expand Up @@ -163,6 +164,76 @@ jobs:
cache-from: type=gha
cache-to: type=gha,mode=max

build-docker-image:
name: Build and push Docker image
runs-on: ubuntu-latest
needs: [release-nightly]
if: ${{ github.event.inputs.name != 'blank' || github.event.inputs.new_tag != 'blank' }}
env:
IMAGE_NAME: reearth/reearth-visualizer-web
defaults:
run:
working-directory: web
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Get options
id: options
env:
TAG: ${{ github.event.inputs.new_tag_short && github.event.inputs.new_tag_short != 'blank' && github.event.inputs.new_tag_short || '' }}
NAME: ${{ github.event.inputs.name }}
SHA: ${{ github.event.inputs.sha_short }}
run: |
if [[ -n $TAG ]]; then
PLATFORMS=linux/amd64,linux/arm64
VERSION=$TAG
TAGS=$IMAGE_NAME:$TAG
if [[ ! $TAG =~ '-' ]]; then
TAGS+=,${IMAGE_NAME}:${TAG%.*}
TAGS+=,${IMAGE_NAME}:${TAG%%.*}
TAGS+=,${IMAGE_NAME}:latest
fi
else
PLATFORMS=linux/amd64
VERSION=$SHA
TAGS=$IMAGE_NAME:$NAME
fi
echo "::set-output name=platforms::$PLATFORMS"
echo "::set-output name=version::$VERSION"
echo "::set-output name=tags::$TAGS"
KeisukeYamashita marked this conversation as resolved.
Show resolved Hide resolved

# TODO: Comment in after removing build-docker-image-old job.
# - name: Fetch reearth-web release
# uses: dsaltares/fetch-gh-release-asset@master
# with:
# repo: reearth/reearth
# version: tags/${{ github.event.inputs.name && github.event.inputs.name != 'blank' && github.event.inputs.name || github.event.inputs.new_tag }}
# file: reearth-web_${{ github.event.inputs.name && github.event.inputs.name != 'blank' && github.event.inputs.name || github.event.inputs.new_tag }}.tar.gz
# token: ${{ secrets.GITHUB_TOKEN }}
# target: server/reearth-web.tar.gz
# - name: Extract reearth/web
# run: tar -xvf reearth-web.tar.gz; mv reearth-web web; ls

- name: Build and push docker image
uses: docker/build-push-action@v5
with:
context: web
platforms: ${{ steps.options.outputs.platforms }}
push: true
build-args: VERSION=${{ steps.options.outputs.version }}
tags: ${{ steps.options.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max

release:
name: Release
runs-on: ubuntu-latest
Expand Down
64 changes: 62 additions & 2 deletions .github/workflows/ci_docker_build_push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ on:
description: "sha"
required: true
jobs:
build-docker-image:
name: Build and push Docker image
server:
name: Build and push server Docker image
runs-on: ubuntu-latest
if: github.event.inputs.name || github.event.inputs.new_tag
env:
Expand Down Expand Up @@ -63,6 +63,7 @@ jobs:
echo "::set-output name=platforms::$PLATFORMS"
echo "::set-output name=version::$VERSION"
echo "::set-output name=tags::$TAGS"
# TODO: Remove later after migrating to Cloud Run
- name: Fetch reearth-web release
uses: dsaltares/fetch-gh-release-asset@master
with:
Expand All @@ -73,6 +74,7 @@ jobs:
target: server/reearth-web.tar.gz
- name: Extract reearth/web
run: tar -xvf reearth-web.tar.gz; mv reearth-web web; ls

- name: Build and push docker image
uses: docker/build-push-action@v2
with:
Expand All @@ -83,3 +85,61 @@ jobs:
tags: ${{ steps.options.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max

web:
name: Build and push web Docker image
runs-on: ubuntu-latest
if: github.event.inputs.name || github.event.inputs.new_tag
env:
IMAGE_NAME: reearth/reearth-visualizer-web
defaults:
run:
working-directory: web
steps:
- name: Checkout
uses: actions/checkout@v2
Comment on lines +99 to +100
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Update Checkout Action version.

The actions/checkout@v2 is outdated. Update to the latest version for improved features and security.

Apply this change:

-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Checkout
uses: actions/checkout@v2
- name: Checkout
uses: actions/checkout@v4
🧰 Tools
🪛 actionlint

100-100: the runner of "actions/checkout@v2" action is too old to run on GitHub Actions. update the action's version to fix this issue

(action)

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Get options
id: options
env:
TAG: ${{ github.event.inputs.new_tag_short }}
NAME: ${{ github.event.inputs.name }}
SHA: ${{ github.event.inputs.sha_short }}
run: |
if [[ -n $TAG ]]; then
PLATFORMS=linux/amd64,linux/arm64
VERSION=$TAG
TAGS=$IMAGE_NAME:$TAG
if [[ ! $TAG =~ '-' ]]; then
TAGS+=,${IMAGE_NAME}:${TAG%.*}
TAGS+=,${IMAGE_NAME}:${TAG%%.*}
TAGS+=,${IMAGE_NAME}:latest
fi
else
PLATFORMS=linux/amd64
VERSION=$SHA
TAGS=$IMAGE_NAME:$NAME
fi
echo "::set-output name=platforms::$PLATFORMS"
echo "::set-output name=version::$VERSION"
echo "::set-output name=tags::$TAGS"
KeisukeYamashita marked this conversation as resolved.
Show resolved Hide resolved

- name: Build and push docker image
uses: docker/build-push-action@v4
with:
context: web
platforms: ${{ steps.options.outputs.platforms }}
push: true
build-args: VERSION=${{ steps.options.outputs.version }}
tags: ${{ steps.options.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
37 changes: 31 additions & 6 deletions .github/workflows/deploiy_aws.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@ on:
required: false
default:
env:
# server
IMAGE: reearth/reearth-visualizer:nightly
IMAGE_AWS: reearth-test-server:latest
# Server
IMAGE_API: reearth/reearth-visualizer:nightly
IMAGE_API_AWS: reearth-test-server:latest

# Web
IMAGE_WEB: reearth/reearth-visualizer-web:nightly
IMAGE_WEB_AWS: reearth-test-web:latest

jobs:
deploy_server:
Expand All @@ -33,6 +37,27 @@ jobs:

- name: docker pull, push to ECR
run: |
docker pull $IMAGE
docker tag $IMAGE ${{ steps.login-ecr.outputs.registry }}/$IMAGE_AWS
docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_AWS
docker pull $IMAGE_API
docker tag $IMAGE_API ${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS
docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS
Comment on lines +40 to +42
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add quotes around variables in shell commands for better safety.

While the commands work, they should follow shell scripting best practices by quoting variables to prevent word splitting and globbing issues.

Apply this diff:

-          docker pull $IMAGE_API
-          docker tag $IMAGE_API ${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS
-          docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS
+          docker pull "$IMAGE_API"
+          docker tag "$IMAGE_API" "${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS"
+          docker push "${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
docker pull $IMAGE_API
docker tag $IMAGE_API ${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS
docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS
docker pull "$IMAGE_API"
docker tag "$IMAGE_API" "${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS"
docker push "${{ steps.login-ecr.outputs.registry }}/$IMAGE_API_AWS"


deploy_web:
name: Deploy web to AWS
runs-on: ubuntu-latest
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID || github.events.inputs.keyId }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY || github.events.inputs.secretAccessKey }}
aws-region: us-west-2

- name: Log in to AWS ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

- name: docker pull, push to ECR
run: |
docker pull $IMAGE_WEB
docker tag $IMAGE_WEB ${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS
docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS
Comment on lines +61 to +63
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add quotes around variables in shell commands for better safety.

Similar to the server job, the shell commands should have proper quoting.

Apply this diff:

-          docker pull $IMAGE_WEB
-          docker tag $IMAGE_WEB ${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS
-          docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS
+          docker pull "$IMAGE_WEB"
+          docker tag "$IMAGE_WEB" "${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS"
+          docker push "${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
docker pull $IMAGE_WEB
docker tag $IMAGE_WEB ${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS
docker push ${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS
docker pull "$IMAGE_WEB"
docker tag "$IMAGE_WEB" "${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS"
docker push "${{ steps.login-ecr.outputs.registry }}/$IMAGE_WEB_AWS"

31 changes: 30 additions & 1 deletion .github/workflows/deploy_web_nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,15 @@ on:
env:
REEARTH_URL: visualizer.test.reearth.dev
GCS_DEST: gs://reearth-visualizer-oss-static-bucket/

IMAGE: reearth/reearth-visualizer-web:nightly
IMAGE_GCP: us-central1-docker.pkg.dev/reearth-oss/reearth/reearth-visualizer-web:nightly
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
jobs:
deploy_test:
# TODO: Remove later after migrating to Cloud Run
deploy_test_old:
name: Deploy test env
runs-on: ubuntu-latest
if: github.event.repository.full_name == 'reearth/reearth-visualizer'
Expand All @@ -27,3 +31,28 @@ jobs:
- run: tar -xvf reearth-web_nightly.tar.gz
- name: rsync
run: gsutil -m -h "Cache-Control:no-store" rsync -x "^reearth_config\\.json$|^cesium_ion_token\\.txt$" -dr reearth-web/ $GCS_DEST

deploy_test:
name: Deploy app to test env
runs-on: ubuntu-latest
if: github.event.repository.full_name == 'reearth/reearth-visualizer'
steps:
- uses: google-github-actions/auth@v0
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v0
- name: Configure docker
run: gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
- name: docker push
run: |
docker pull $IMAGE
docker tag $IMAGE $IMAGE_GCP
docker push $IMAGE_GCP
Comment on lines +45 to +51
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add error handling and fix unquoted variables.

The Docker configuration and push steps need some improvements:

Apply these changes to enhance reliability:

   - name: docker push
     run: |
-          docker pull $IMAGE
-          docker tag $IMAGE $IMAGE_GCP
-          docker push $IMAGE_GCP
+          set -eo pipefail
+          docker pull "$IMAGE" || exit 1
+          docker tag "$IMAGE" "$IMAGE_GCP" || exit 1
+          docker push "$IMAGE_GCP" || exit 1
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Configure docker
run: gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
- name: docker push
run: |
docker pull $IMAGE
docker tag $IMAGE $IMAGE_GCP
docker push $IMAGE_GCP
- name: Configure docker
run: gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
- name: docker push
run: |
set -eo pipefail
docker pull "$IMAGE" || exit 1
docker tag "$IMAGE" "$IMAGE_GCP" || exit 1
docker push "$IMAGE_GCP" || exit 1
🧰 Tools
🪛 actionlint

48-48: shellcheck reported issue in this script: SC2086:info:1:13: Double quote to prevent globbing and word splitting

(shellcheck)

48-48: shellcheck reported issue in this script: SC2086:info:2:12: Double quote to prevent globbing and word splitting

(shellcheck)

48-48: shellcheck reported issue in this script: SC2086:info:2:19: Double quote to prevent globbing and word splitting

(shellcheck)

48-48: shellcheck reported issue in this script: SC2086:info:3:13: Double quote to prevent globbing and word splitting

(shellcheck)

- name: Deploy to Cloud Run
run: |
gcloud run deploy reearth-visualizer-web \
--image $IMAGE_GCP \
--region $GCP_REGION \
--platform managed \
--quiet
13 changes: 13 additions & 0 deletions web/.dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
*

!docker/
!src/

!.yarnrc.yml
!index.html
!i18next-parser.config.js
!tsconfig.json
!package.json
!published.html
!vite.config.ts
!yarn.lock
69 changes: 69 additions & 0 deletions web/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
FROM node:20.18.0-slim AS builder
WORKDIR /app

ARG NODE_OPTIONS="--max-old-space-size=4096"
ARG GITHUB_SHA
ENV NODE_OPTIONS=$NODE_OPTIONS
ENV GITHUB_SHA=$GITHUB_SHA

RUN --mount=type=bind,source=package.json,target=package.json \
--mount=type=cache,target=/root/.npm,sharing=locked \
corepack enable

RUN --mount=type=bind,source=.yarnrc.yml,target=.yarnrc.yml \
--mount=type=bind,source=package.json,target=package.json \
--mount=type=bind,source=yarn.lock,target=yarn.lock \
--mount=type=cache,target=/root/.yarn,sharing=locked \
yarn install --frozen-lockfile

RUN --mount=type=bind,source=.yarnrc.yml,target=.yarnrc.yml \
--mount=type=bind,source=package.json,target=package.json \
--mount=type=bind,source=yarn.lock,target=yarn.lock \
--mount=type=bind,source=index.html,target=index.html \
--mount=type=bind,source=published.html,target=published.html \
--mount=type=bind,source=tsconfig.json,target=tsconfig.json \
--mount=type=bind,source=vite.config.ts,target=vite.config.ts \
--mount=type=bind,source=src,target=src \
--mount=type=cache,target=/root/.yarn,sharing=locked \
yarn build

FROM nginx:1.27-alpine
WORKDIR /usr/share/nginx/html

# Quite the Nginx startup logs.
ENV NGINX_ENTRYPOINT_QUIET_LOGS=true

# Default to Cloud Run port.
# Ref: https://cloud.google.com/run/docs/reference/container-contract#port
ENV PORT=8080

# Defaults Google Cloud Load Balancer header.
# Ref: https://cloud.google.com/load-balancing/docs/https#target-proxies
ENV REAL_IP_HEADER=X-Forwarded-For

# Default values.
ENV REEARTH_WEB_API=null
ENV REEARTH_WEB_AUTH0_AUDIENCE=null
ENV REEARTH_WEB_AUTH0_CLIENT_ID=null
ENV REEARTH_WEB_AUTH0_DOMAIN=null
ENV REEARTH_WEB_CESIUM_ION_ACCESS_TOKEN=null
ENV REEARTH_WEB_CLOUD_API=null
ENV REEARTH_WEB_CURRENT_TOS=null
ENV REEARTH_WEB_DEVELOPER_MODE=null
ENV REEARTH_WEB_DOCUMENTATION_URL=null
ENV REEARTH_WEB_EARLY_ACCESS_ADMINS=[]
ENV REEARTH_WEB_EXTENSION_URLS=[]
ENV REEARTH_WEB_IP=null
ENV REEARTH_WEB_MARKETPLACE_URL=null
ENV REEARTH_WEB_PASSWORD_POLICY=null
ENV REEARTH_WEB_PLUGINS=null
ENV REEARTH_WEB_POLICY=null
ENV REEARTH_WEB_PUBLISHED=null

COPY --from=builder --chown=nginx:nginx /app/dist /usr/share/nginx/html
COPY --chown=nginx:nginx docker/nginx.conf.template /etc/nginx/templates/nginx.conf.template
COPY --chown=nginx:nginx docker/40-envsubst-on-reearth-config.sh /docker-entrypoint.d
COPY --chown=nginx:nginx docker/reearth_config.json.template /opt/reearth/reearth_config.json.template

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]
25 changes: 25 additions & 0 deletions web/docker/40-envsubst-on-reearth-config.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/sh

set -e

# rewrite index.html to change title and favicon
_REEARTH_HTML_FILE="/usr/share/nginx/html/index.html"

# generate reearth_config.json
_REEARTH_CONFIG_TEMPLATE_FILE="/opt/reearth/reearth_config.json.template"
_REEARTH_CONFIG_OUTPUT_FILE="/usr/share/nginx/html/reearth_config.json"
KeisukeYamashita marked this conversation as resolved.
Show resolved Hide resolved

# Wrap with "" if the value doesn't start with '{[' and end with ']}' (JSON) or "null".
wrap_reearth_variables() {
for var in $(env | grep '^REEARTH_WEB' | cut -d= -f1); do
value=$(printenv "$var")
if [ -z "$value" ]; then
eval "export $var='\"\"'"
elif [ "$value" != "null" ] && [ "$value" != "true" ] && [ "$value" != "false" ] && ! echo "$value" | grep -qE '^\{.*\}$|^\[.*\]$'; then
eval "export $var='\"${value}\"'"
fi
done
}

wrap_reearth_variables "$@"
envsubst < "$_REEARTH_CONFIG_TEMPLATE_FILE" > "$_REEARTH_CONFIG_OUTPUT_FILE"
KeisukeYamashita marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading