fix(cmd-api-server): use ncc bundle in container image - CVE-2024-29415

Fixes the CVE mentioned and also improves our response time to future CVEs
by a very wide margin. Details below.

1. Fixing the mentioned vulnerability in the API server and doing so in
a way so that in the future our dependency upgrades automatically propagate
to the container builds as well.
2. The way we are achieving this is by making the container image build
use the pre-built bundle instead of pulling the package contents from npm.
3. This has the advantage of breaking the chicken egg problem with releases
to npm and container images, so from now on if we are adding a fix to the
API server in the code, the built container image will automatically contain
that fix when building on the CI for the pull request.
4. This is also a new pattern for how to create our container images that
has a couple more improvements to it:
4.1. The .dockerignore file is now specific to the particular package's
container image instead of the global one in the project root being used.
This was needed because we are copying files from the ./dist/ folder of the
package to the container image at build time but this was not possible while
the root dir .dockerignore file was in effect because it blanket ignores
the ./dist/ folders overall and so the image building was failing with errors
that it couldn't locate the bundle (which is inside the ./dist/ directory)
4.2. The healthcheck of the container is now 100% self-contained and needs
no external dependencies of any kind (neither npm nor operating system level ones)
This is beneficial because it reduces the attack-surface of the image and also
reduce the size of the image by at least a 100 MB.
4.3. With the introduction of the usage of the bundled version of the code
we have **dramatically** reduced the image size overall. The image built from
this revision of the code is 221 MB while the previous image versions were
hovering closer to a 0.5 GB.
5. Also updated the README of the package so that all the examples pertaining
to the container image are now fully functional once again.
6. Simplified the container image's definition: the custom docker entrypoint
script and the healthcheck bash script are no longer necessary.
7. Renamed the container image definition file from `Dockerfile` to
`cmd-api-server.Dockerfile` because this is mandated by Docker when building
images with custom .dockerignore files (it needs the custom filename to
disambiguate the .dockerignore files based on it)
8. Refactored how the CI executes the Trivy scan to reduce resource usage:
8.1. There is no separate image build job now. This was necessary because
with the new image definition we have to have the project compiled first
(since we no longer install directly from npm) so it would've been a lot of
duplicated compute time to recompile the project in yet another CI job for the
image build.

The image built from this revision is also published on the official repository
with the canary tag of:
`ghcr.io/hyperledger/cactus-cmd-api-server:2024-07-03T19-32-51-dev-3f5e97893`

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>

.cspell.json

@@ -182,6 +182,8 @@
     "tlsca",
     "tlscacerts",
     "Trivy",
+    "trivyignore",
+    "trivyignores",
     "txid",
     "txqueue",
     "Uisrs",

.github/workflows/ci.yaml

@@ -458,6 +458,30 @@ jobs:
 
       - run: ./tools/ci.sh
 
+      - name: build_ncc_bundle
+        run: yarn lerna run build:bundle --scope=@hyperledger/cactus-cmd-api-server
+
+      - name: ghcr.io/hyperledger/cactus-cmd-api-server
+        run: |
+          DOCKER_BUILDKIT=1 docker build \
+            --file ./packages/cactus-cmd-api-server/cmd-api-server.Dockerfile \
+            ./packages/cactus-cmd-api-server \
+            --tag cas \
+            --tag cmd-api-server \
+            --tag "ghcr.io/hyperledger/cactus-cmd-api-server:$(date +"%Y-%m-%dT%H-%M-%S" --utc)-dev-$(git rev-parse --short HEAD)"
+
+      - if: ${{ env.RUN_TRIVY_SCAN == 'true' }}
+        name: Run Trivy vulnerability scan for cmd-api-server
+        uses: aquasecurity/trivy-action@0.19.0
+        with:
+          image-ref: 'cmd-api-server'
+          format: 'table'
+          exit-code: '1'
+          ignore-unfixed: false
+          vuln-type: 'os,library'
+          severity: 'CRITICAL,HIGH'
+          trivyignores: ./.trivyignore
+
       - name: Ensure .tmp Directory Exists
         run: mkdir -p .tmp/benchmark-results/cmd-api-server/
 
@@ -484,7 +508,7 @@ jobs:
           auto-push: ${{ github.ref == 'refs/heads/main' }}
 
           # Show alert with commit comment on detecting possible performance regression
-          alert-threshold: '5%'
+          alert-threshold: '25%'
           comment-on-alert: true
           fail-on-alert: true
           alert-comment-cc-users: '@petermetz'
@@ -2199,25 +2223,6 @@ jobs:
       - uses: actions/checkout@v4.1.1
       - name: ghcr.io/hyperledger/cactus-besu-all-in-one
         run: DOCKER_BUILDKIT=1 docker build ./tools/docker/besu-all-in-one/ -f ./tools/docker/besu-all-in-one/Dockerfile
-  ghcr-cmd-api-server:
-    runs-on: ubuntu-22.04
-    needs:
-      - compute_changed_packages
-    if: needs.compute_changed_packages.outputs.cmd-api-server-changed == 'true'
-    steps:
-      - uses: actions/checkout@v4.1.1
-      - name: ghcr.io/hyperledger/cactus-cmd-api-server
-        run: DOCKER_BUILDKIT=1 docker build . -f ./packages/cactus-cmd-api-server/Dockerfile -t cactus-cmd-api-server
-      - if: ${{ env.RUN_TRIVY_SCAN == 'true' }}
-        name: Run Trivy vulnerability scan for cactus-cmd-api-server
-        uses: aquasecurity/trivy-action@0.19.0
-        with:
-          image-ref: 'cactus-cmd-api-server'
-          format: 'table'
-          exit-code: '1'
-          ignore-unfixed: false
-          vuln-type: 'os,library'
-          severity: 'CRITICAL,HIGH'
   ghcr-connector-besu:
     needs:
       - compute_changed_packages

.trivyignore

@@ -0,0 +1,3 @@
+# Misclassification by the Debian maintainers. Does not affect us
+# Source: https://github.com/aquasecurity/trivy/discussions/6722#discussioncomment-9518531
+CVE-2023-45853

packages/cactus-cmd-api-server/README.md

@@ -12,6 +12,7 @@
   - [Building the container image locally](#building-the-container-image-locally)
   - [Running the container image locally](#running-the-container-image-locally)
   - [Testing API calls with the container](#testing-api-calls-with-the-container)
+  - [Running a Security Scan on the Built Container Image](#running-a-security-scan-on-the-built-container-image)
 - [Prometheus Exporter](#prometheus-exporter)
   - [Usage Prometheus](#usage-prometheus)
   - [Prometheus Integration](#prometheus-integration)
@@ -216,15 +217,21 @@ of the machine they are hosted on:
 
 ### Building the container image locally
 
-In the Cactus project root say:
+In the project root directory run these commands on the terminal:
 
 ```sh
-DOCKER_BUILDKIT=1 docker build -f ./packages/cactus-cmd-api-server/Dockerfile . -t cas -t cactus-api-server
+yarn configure
+yarn lerna run build:bundle --scope=@hyperledger/cactus-cmd-api-server
+
 ```
 
-Build with a specific version of the npm package:
 ```sh
-DOCKER_BUILDKIT=1 docker build --build-arg NPM_PKG_VERSION=main -f ./packages/cactus-cmd-api-server/Dockerfile . -t cas -t cactus-api-server
+DOCKER_BUILDKIT=1 docker build \
+  --file ./packages/cactus-cmd-api-server/cmd-api-server.Dockerfile \
+  ./packages/cactus-cmd-api-server \
+   --tag cas \
+   --tag cmd-api-server \
+   --tag ghcr.io/hyperledger/cactus-cmd-api-server:$(date +"%Y-%m-%dT%H-%M-%S" --utc)-dev-$(git rev-parse --short HEAD)
 ```
 
 ### Running the container image locally
@@ -235,7 +242,6 @@ See section [Building the container image locally](#building-the-container-image
 Once you've built the container, the following commands should work:
 
 - Launch container - no plugins, default configuration
-
   ```sh
   docker run \
     --rm \
@@ -244,6 +250,13 @@ Once you've built the container, the following commands should work:
     --env AUTHORIZATION_PROTOCOL='NONE' \
     --env AUTHORIZATION_CONFIG_JSON='{}' \
     --env GRPC_TLS_ENABLED=false \
+    --env API_TLS_CERT_PEM=- \
+    --env API_TLS_CLIENT_CA_PEM=- \
+    --env API_TLS_KEY_PEM=- \
+    --env API_TLS_ENABLED=false \
+    --env API_MTLS_ENABLED=false \
+    --env API_HOST=0.0.0.0 \
+    --env LOG_LEVEL=INFO \
     cas
   ```
 
@@ -257,8 +270,15 @@ Once you've built the container, the following commands should work:
     --env AUTHORIZATION_PROTOCOL='NONE' \
     --env AUTHORIZATION_CONFIG_JSON='{}' \
     --env GRPC_TLS_ENABLED=false \
+    --env API_TLS_CERT_PEM=- \
+    --env API_TLS_CLIENT_CA_PEM=- \
+    --env API_TLS_KEY_PEM=- \
+    --env API_TLS_ENABLED=false \
+    --env API_MTLS_ENABLED=false \
+    --env API_HOST=0.0.0.0 \
+    --env LOG_LEVEL=INFO \
     cas \
-      node_modules/@hyperledger/cactus-cmd-api-server/dist/lib/main/typescript/cmd/cactus-api.js \
+      node index.js \
       --plugins='[{"packageName": "@hyperledger/cactus-plugin-ledger-connector-fabric", "type": "org.hyperledger.cactus.plugin_import_type.LOCAL", "action": "org.hyperledger.cactus.plugin_import_action.INSTALL",  "options": { "version": "dev", "peerBinary":"/fabric-samples/bin/peer", "connectionProfile": {}, "instanceId": "some-unique-instance-id"}}]'
   ```
 
@@ -271,6 +291,13 @@ Once you've built the container, the following commands should work:
     --env AUTHORIZATION_PROTOCOL='NONE' \
     --env AUTHORIZATION_CONFIG_JSON='{}' \
     --env GRPC_TLS_ENABLED=false \
+    --env API_TLS_CERT_PEM=- \
+    --env API_TLS_CLIENT_CA_PEM=- \
+    --env API_TLS_KEY_PEM=- \
+    --env API_TLS_ENABLED=false \
+    --env API_MTLS_ENABLED=false \
+    --env API_HOST=0.0.0.0 \
+    --env LOG_LEVEL=INFO \
     --env PLUGINS='[{"packageName": "@hyperledger/cactus-plugin-ledger-connector-besu", "type": "org.hyperledger.cactus.plugin_import_type.LOCAL", "action": "org.hyperledger.cactus.plugin_import_action.INSTALL",  "options": { "rpcApiWsHost": "http://127.0.0.1:8546", "rpcApiHttpHost": "http://127.0.0.1:8545", "instanceId": "some-unique-besu-connector-instance-id"}}]' \
     cas
   ```
@@ -284,15 +311,22 @@ Once you've built the container, the following commands should work:
     --env AUTHORIZATION_PROTOCOL='NONE' \
     --env AUTHORIZATION_CONFIG_JSON='{}' \
     --env GRPC_TLS_ENABLED=false \
+    --env API_TLS_CERT_PEM=- \
+    --env API_TLS_CLIENT_CA_PEM=- \
+    --env API_TLS_KEY_PEM=- \
+    --env API_TLS_ENABLED=false \
+    --env API_MTLS_ENABLED=false \
+    --env API_HOST=0.0.0.0 \
+    --env LOG_LEVEL=INFO \
     cas \
-      ./node_modules/@hyperledger/cactus-cmd-api-server/dist/lib/main/typescript/cmd/cactus-api.js \
+      node index.js \
       --plugins='[{"packageName": "@hyperledger/cactus-plugin-ledger-connector-besu", "type": "org.hyperledger.cactus.plugin_import_type.LOCAL", "action": "org.hyperledger.cactus.plugin_import_action.INSTALL",  "options": { "rpcApiWsHost": "http://127.0.0.1:8546", "rpcApiHttpHost": "http://127.0.0.1:8545", "instanceId": "some-unique-besu-connector-instance-id"}}]'
   ```
 
 - Launch container with **configuration file** mounted from host machine:
   ```sh
 
-  echo '{"plugins": [{"packageName": "@hyperledger/cactus-plugin-ledger-connector-besu", "type": "org.hyperledger.cactus.plugin_import_type.LOCAL", "action": "org.hyperledger.cactus.plugin_import_action.INSTALL",  "options": { "rpcApiWsHost": "http://127.0.0.1:8546", "rpcApiHttpHost": "http://127.0.0.1:8545", "instanceId": "some-unique-besu-connector-instance-id"}}]}' > cactus.json
+  echo '{"plugins": [{"packageName": "@hyperledger/cactus-plugin-ledger-connector-besu", "type": "org.hyperledger.cactus.plugin_import_type.LOCAL", "action": "org.hyperledger.cactus.plugin_import_action.INSTALL",  "options": { "rpcApiWsHost": "http://127.0.0.1:8546", "rpcApiHttpHost": "http://127.0.0.1:8545", "instanceId": "some-unique-besu-connector-instance-id"}}]}' > .cacti-config.json
 
   docker run \
     --rm \
@@ -301,10 +335,17 @@ Once you've built the container, the following commands should work:
     --env AUTHORIZATION_PROTOCOL='NONE' \
     --env AUTHORIZATION_CONFIG_JSON='{}' \
     --env GRPC_TLS_ENABLED=false \
-    --mount type=bind,source="$(pwd)"/cactus.json,target=/cactus.json \
+    --env API_TLS_CERT_PEM=- \
+    --env API_TLS_CLIENT_CA_PEM=- \
+    --env API_TLS_KEY_PEM=- \
+    --env API_TLS_ENABLED=false \
+    --env API_MTLS_ENABLED=false \
+    --env API_HOST=0.0.0.0 \
+    --env LOG_LEVEL=INFO \
+    --mount type=bind,source="$(pwd)"/.cacti-config.json,target=/.cacti-config.json \
     cas \
-      ./node_modules/@hyperledger/cactus-cmd-api-server/dist/lib/main/typescript/cmd/cactus-api.js \
-      --config-file=/cactus.json
+      node index.js \
+      --config-file=/.cacti-config.json
   ```
 
 ### Testing API calls with the container
@@ -314,9 +355,10 @@ Don't have a Besu network on hand to test with? Test or develop against our Besu
 1. Terminal Window 1 (Ledger)
     ```sh
     docker run \
+      --rm \
       --publish 8545:8545 \
       --publish 8546:8546 \
-      ghcr.io/hyperledger/cactus-besu-all-in-one:2023-11-16-89d9b93
+      ghcr.io/hyperledger/cactus-besu-all-in-one:2024-07-03-08925ff
     ```
 
 2. Terminal Window 2 (Cactus API Server)
@@ -329,6 +371,13 @@ Don't have a Besu network on hand to test with? Test or develop against our Besu
       --env AUTHORIZATION_PROTOCOL='NONE' \
       --env AUTHORIZATION_CONFIG_JSON='{}' \
       --env GRPC_TLS_ENABLED=false \
+      --env API_TLS_CERT_PEM=- \
+      --env API_TLS_CLIENT_CA_PEM=- \
+      --env API_TLS_KEY_PEM=- \
+      --env API_TLS_ENABLED=false \
+      --env API_MTLS_ENABLED=false \
+      --env API_HOST=0.0.0.0 \
+      --env LOG_LEVEL=INFO \
       --env PLUGINS='[{"packageName": "@hyperledger/cactus-plugin-ledger-connector-besu", "type": "org.hyperledger.cactus.plugin_import_type.LOCAL", "action": "org.hyperledger.cactus.plugin_import_action.INSTALL",  "options": { "rpcApiWsHost": "http://127.0.0.1:8546", "rpcApiHttpHost": "http://127.0.0.1:8545", "instanceId": "some-unique-besu-connector-instance-id"}}]' \
       cas
     ```
@@ -380,7 +429,18 @@ Don't have a Besu network on hand to test with? Test or develop against our Besu
     }
     ```
 
+### Running a Security Scan on the Built Container Image
+
+After having built the container image with the command(s) described above, you can
+run the Trivy tool to scan for security vulnerabilities.
 
+> The `trivy` binary is preinstalled within the VSCode dev container so make sure
+> to open the project in that if you would prefer not to install Trivy on your host
+> operating system.
+
+```sh
+trivy image --scanners=vuln --vuln-type=os,library --severity=CRITICAL --severity=HIGH cas
+```
 
 ## Prometheus Exporter
 This class creates a prometheus exporter, which scrapes the total Cactus node count.

packages/cactus-cmd-api-server/cmd-api-server.Dockerfile

@@ -0,0 +1,25 @@
+FROM node:22.4.0-bookworm-slim
+
+# CVE-2023-31484 - perl: CPAN.pm does not verify TLS certificates when downloading distributions over HTTPS...
+RUN apt-get remove -y --allow-remove-essential perl perl-base && apt-get autoremove -y
+
+ARG APP_DIR=/opt/cacti/cmd-api-server
+WORKDIR ${APP_DIR}
+
+COPY ./dist/bundle/ncc/ ${APP_DIR}
+COPY ./cmd-api-server.Dockerfile.healthcheck.mjs ${APP_DIR}
+CMD ["node", "index.js"]
+
+HEALTHCHECK --interval=5s --timeout=1s --start-period=1s --retries=60 CMD [ "node", "./cmd-api-server.Dockerfile.healthcheck.mjs", "http", "localhost", "4000" ]
+
+# FIXME: Stop hardcoding the less secure defaults once we've migrated to yarts
+# for CMD/ENV configuration file parsing.
+ENV COCKPIT_TLS_ENABLED=false
+ENV COCKPIT_CORS_DOMAIN_CSV=\*
+ENV COCKPIT_MTLS_ENABLED=false
+ENV COCKPIT_TLS_CERT_PEM=-
+ENV COCKPIT_TLS_KEY_PEM=-
+ENV COCKPIT_TLS_CLIENT_CA_PEM=-
+
+ENV TZ=Etc/UTC
+ENV NODE_ENV=production