Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ant frontend container #4531

Open
wants to merge 36 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
63cddd2
Initial Dockerfile build stage
amenasse Apr 4, 2023
f40fdcd
Build all internal packages for now
amenasse Apr 6, 2023
9939b48
Remove redundant packages
amenasse Apr 6, 2023
18f4549
Remove redundant layers (MKDIR)
amenasse Apr 6, 2023
fd5ab9e
Use workspaces focus instead of install
amenasse Apr 6, 2023
4a11abc
Run the app
amenasse Apr 6, 2023
d890634
Move to devops
amenasse Apr 6, 2023
b4b42d3
Combine layers
amenasse Apr 6, 2023
9718c2b
Shrink image
amenasse Apr 11, 2023
b8c3933
Reduce layers
amenasse Apr 11, 2023
68bd9c5
Combine config layers
amenasse Apr 11, 2023
904c62a
Use seperate stage for package.json copy
amenasse Apr 13, 2023
819c165
Improve comments
amenasse Apr 13, 2023
a04c991
Remove broken attempt at removing dev dependencies
amenasse Apr 13, 2023
53020fa
Don't copy everything
amenasse Apr 13, 2023
b9e757c
Fix production bundle build
amenasse Apr 13, 2023
8e113e9
Pull secrets from Lastpass on startup
amenasse Apr 13, 2023
32946e3
Move runtime docker scripts to avoid cache busting
amenasse Apr 18, 2023
4a0db45
Remove fetching env from LastPass
amenasse Apr 21, 2023
b510115
Fix CodeShip builds due to missing devops pacakge
amenasse Apr 27, 2023
5142436
Container to fetch environment from LastPass
amenasse Apr 21, 2023
e5f8849
Additional comments
amenasse May 4, 2023
71aaa13
Run apk once to install packages
amenasse May 4, 2023
51b4ab1
Move download env Dockerfile
amenasse May 16, 2023
0e8e694
Container to serve frontends
amenasse May 2, 2023
ffdd590
Add more frontends
amenasse May 2, 2023
c643571
Use foreach to do the build step
amenasse May 2, 2023
a79223c
Only build and copy server packages
amenasse May 2, 2023
8308b1c
Copy .env into package dirs prior to nginx startup
amenasse May 4, 2023
d250957
Explain /home/ubuntu
amenasse May 4, 2023
0a82f51
Combine layers
amenasse May 4, 2023
0e25da8
Fix env path
amenasse May 4, 2023
869d511
Cleanup comments
amenasse May 10, 2023
79c8a08
Convert env files into js objects
amenasse May 11, 2023
f815dcc
Write env files to directory served by nginx
amenasse May 18, 2023
f08306a
Selected packages can be passed to script
amenasse May 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,17 @@ COPY ./tsconfig* babel.config.json tsconfig-js.json jest.config-ts.json .eslintr
# Build and install internal dependencies
Copy link
Contributor

@IgorNadj IgorNadj May 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to consider index.html as well, it has some templating that looks at env vars

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, the following seems to work, but probably needs to be looked at by someone familair with tupaia.. suggest this be addressed in a seperate PR

diff --git a/packages/web-frontend/public/index.html b/packages/web-frontend/public/index.html
index 78279d1f8..7e0804fec 100644
--- a/packages/web-frontend/public/index.html
+++ b/packages/web-frontend/public/index.html
@@ -13,6 +13,7 @@
     />
     <meta property="og:url" content="https://tupaia.org" />
     <meta property="og:title" content="Tupaia" />
+    <script src="%PUBLIC_URL%/env-config.js"></script>

     <% if (process.env.REACT_APP_DEPLOYMENT_NAME === 'master' ||
     process.env.REACT_APP_DEPLOYMENT_NAME === 'main' || process.env.REACT_APP_DEPLOYMENT_NAME ===

RUN yarn build:internal-dependencies

COPY packages/ packages/
COPY packages/ ./packages/

RUN yarn build:non-internal-dependencies
# Build server packages
# Not using parrallel tasks (-P) as node eats all the ram
RUN yarn workspaces foreach --verbose --from '@tupaia/*-server' run build

# Build final production image
FROM node:14.19.3-alpine3.15 as dist
WORKDIR /tupaia

# We could copy just the servers here but the size difference is negligble and not worth the extra layers.
COPY --from=builder /tupaia/packages/ ./packages/
COPY --from=builder /tupaia/node_modules/ ./node_modules
# TODO: Remove dev dependencies here. Running yarn workspaces focus -A
Expand Down
73 changes: 73 additions & 0 deletions frontend.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
FROM node:14.19.3-alpine3.15 as base

# Install features not available in base alpine distro
RUN apk --no-cache add bash && \
yarn set version berry # set Yarn v3

# Copy *just the package.json* of each package in a separate stage so we don't cachebust it
WORKDIR /pre
FROM base as yarnprep
COPY .yarnrc.yml package.json yarn.lock ./
COPY packages/ pkgs/
RUN for pkg in $(ls pkgs); do if test -s pkgs/$pkg/package.json; then mkdir -p packages/$pkg && mv -v pkgs/$pkg/package.json packages/$pkg/; fi; done


FROM base as builder
WORKDIR /tupaia
COPY --from=yarnprep /pre/package.json /pre/yarn.lock /pre/.yarnrc.yml ./
COPY .yarn ./.yarn
COPY --from=yarnprep /pre/packages/ ./packages

# Run yarn without building, so we can cache node_modules without code changes
# invalidating this layer. Trying to install just the dependencies for the
# frontend packages doesn't work as root level dev dependencies are needed for
# the build. I can't figure out a way to install these deps without installing
# everything.
RUN SKIP_BUILD_INTERNAL_DEPENDENCIES=true yarn workspaces focus -A

## Add content of all internal dependency packages ready for internal dependencies to be built
COPY packages/access-policy/. ./packages/access-policy
COPY packages/utils/. ./packages/utils
COPY packages/tsutils/. ./packages/tsutils
COPY packages/types/. ./packages/types
COPY packages/ui-components/. ./packages/ui-components
COPY scripts/bash/ ./scripts/bash/

# Build tooling configuration files
COPY ./tsconfig* babel.config.json tsconfig-js.json jest.config-ts.json .eslintrc ./

# Build and install internal dependencies
RUN scripts/bash/buildInternalDependencies.sh --packagePath packages/web-frontend && \
scripts/bash/buildInternalDependencies.sh --packagePath packages/admin-panel && \
scripts/bash/buildInternalDependencies.sh --packagePath packages/psss

# Build the frontends
COPY packages/web-frontend packages/web-frontend
COPY packages/admin-panel packages/admin-panel
COPY packages/psss packages/psss

# Not using parrallel tasks (-P) as node eats all the ram
RUN yarn workspaces foreach --verbose -j 1 --from '{@tupaia/psss,@tupaia/admin-panel,@tupaia/web-frontend}' run build

FROM docker.io/bitnami/git:2.40.1 as h5bp
# add h5bp config
RUN git clone https://github.com/h5bp/server-configs-nginx.git && \
cd ./server-configs-nginx && \
git checkout tags/2.0.0

# Build final production image
FROM nginx:1.24.0-alpine3.17
RUN apk add bash
WORKDIR /home/ubuntu/tupaia

COPY packages/devops/configs/servers.conf /etc/nginx/conf.d/servers.conf
COPY --from=h5bp server-configs-nginx/h5bp /etc/nginx/h5bp
# copy .env files into package dirs prior to nginx startup
COPY scripts/docker/nginx/create-env.sh /docker-entrypoint.d/50-create-env.sh
# Use /home/ubuntu as workdir to be compatible with existing nginx config
# (packages/devops/configs/servers.conf)
WORKDIR "/home/ubuntu/tupaia"
COPY packages/devops/misc/error_page.html ./error_page.html
COPY --from=builder /tupaia/packages/web-frontend/build ./packages/web-frontend/build
COPY --from=builder /tupaia/packages/admin-panel/build ./packages/admin-panel/build
COPY --from=builder /tupaia/packages/psss/build ./packages/psss/build
53 changes: 53 additions & 0 deletions scripts/docker/nginx/create-env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/bin/bash

# Convert the env files generated by the downloadEnv container into js objects.
# Run by the frontend container prior to nginx startup

# Usage: create-env.sh <env source dir> <package dirs>
tupaia_env_dir=${1:-/env/packages}
shift
if [ -z $# ]; then
tupaia_package_dirs=(/home/ubuntu/tupaia/packages/*)
else
tupaia_package_dirs=("$@")
fi

function env-js() {
# Read $1 and write to $2 as a js object `window._env__`
# Adapted from
# https://www.freecodecamp.org/news/how-to-implement-runtime-environment-variables-with-create-react-app-docker-and-nginx-7f9d42a91d70
env_src=$1
env_js=$2

echo "window.env = {" > "$env_js"
# Read each line in .env file. Each line represents key=value pairs
while read -r line || [[ -n "$line" ]]; do
# Split env variables by character `=`
if printf '%s\n' "$line" | grep -q -e '='; then
varname=$(printf '%s\n' "$line" | sed -e 's/=.*//')
varvalue=$(printf '%s\n' "$line" | sed -e 's/^[^=]*=//')
fi

# Read value of current variable if exists as Environment variable
# otherwise use value from .env file
value=${!varname:-$varvalue}
# Append configuration property to JS file
echo " $varname: \"$value\"," >> "$env_js"
done < "$env_src"

echo "}" >> "$env_js"
}

for p in "${tupaia_package_dirs[@]}"; do
package=$(basename "$p")
if [ -f "$tupaia_env_dir/$package/.env" ]; then
echo "creating $p/.env-config.js"
if [[ $package == "web-frontend" ]]; then
# web frontend serves different bundles for desktop and mobile
env-js "$tupaia_env_dir/$package/.env" "$p/build/mobile/.env-config.js"
env-js "$tupaia_env_dir/$package/.env" "$p/build/desktop/.env-config.js"
else
env-js "$tupaia_env_dir/$package/.env" "$p/build/.env-config.js"
fi
fi
done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed
const baseUrl = process.env.REACT_APP_CONFIG_SERVER_BASE_URL || window.env.REACT_APP_CONFIG_SERVER_BASE_URL || '...'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good if this could be looked at by someone familiar with tupaia as part of a separate PR