Add PCCS Dockerfile and update docker-compose.yml to use it (taikoxyz#51

) This commit adds a PCCS service Dockerfile to support the latest v5 SGX quote format. Additionally, we have updated the README to provide detailed instructions for setting up the SGX infrastructure and dependencies, addressing the complexity and potential difficulties of the process.
pbeza · Mar 12, 2024 · e2d296a · e2d296a
1 parent ebcfdd4
commit e2d296a
Show file tree

Hide file tree

Showing 5 changed files with 293 additions and 40 deletions.
diff --git a/Dockerfile b/Dockerfile
@@ -13,13 +13,27 @@ RUN apt-get update && \
 RUN cargo build --release ${BUILD_FLAGS}
 
 FROM gramineproject/gramine:1.6-jammy as runtime
+ENV DEBIAN_FRONTEND=noninteractive
 WORKDIR /opt/raiko
 
-RUN apt-get update && \
-    apt-get install -y sudo && \
+RUN curl -o setup.sh -sL https://deb.nodesource.com/setup_18.x && \
+    chmod a+x setup.sh && \
+    ./setup.sh && \
+    apt-get update && \
+    apt-get install -y \
+        cracklib-runtime \
+        libsgx-dcap-default-qpl \
+        libsgx-dcap-ql \
+        libsgx-urts \
+        sgx-pck-id-retrieval-tool \
+        sudo && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
+RUN sed -i 's/#default quoting type = ecdsa_256/default quoting type = ecdsa_256/' /etc/aesmd.conf
+RUN sed -i 's/,"use_secure_cert": true/,"use_secure_cert": false/' /etc/sgx_default_qcnl.conf
+RUN sed -i 's/https:\/\/localhost:8081/https:\/\/pccs:8081/g' /etc/sgx_default_qcnl.conf
+
 RUN mkdir -p \
     ./bin \
     ./guests/sgx \

diff --git a/Dockerfile.pccs b/Dockerfile.pccs
@@ -0,0 +1,61 @@
+# https://raw.githubusercontent.com/intel/SGXDataCenterAttestationPrimitives/master/QuoteGeneration/pccs/container/Dockerfile
+
+# Use multi-stage builds to reduce final image size
+FROM ubuntu:23.04 AS builder
+
+# Define arguments used across multiple stages
+ARG DCAP_VERSION=DCAP_1.20
+ARG NODE_MAJOR=20
+
+# update and install packages, nodejs
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -yq \
+    && apt-get upgrade -yq \
+    && apt-get install -yq --no-install-recommends \
+    build-essential \
+    ca-certificates \
+    curl \
+    gnupg \
+    git \
+    zip \
+    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /usr/share/keyrings/nodesource.gpg \
+    && echo "deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_${NODE_MAJOR}.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list \
+    && apt-get update -yq \
+    && apt-get install -yq --no-install-recommends nodejs \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+# Clone the specific branch or tag
+RUN git clone --recurse-submodules https://github.com/intel/SGXDataCenterAttestationPrimitives.git # -b ${DCAP_VERSION} --depth 1
+
+# Build libPCKCertSelection library
+WORKDIR /SGXDataCenterAttestationPrimitives/tools/PCKCertSelection/
+RUN make \
+    && mkdir -p ../../QuoteGeneration/pccs/lib \
+    && cp ./out/libPCKCertSelection.so ../../QuoteGeneration/pccs/lib/ \
+    && make clean
+
+# Build PCCS
+WORKDIR /SGXDataCenterAttestationPrimitives/QuoteGeneration/pccs/
+RUN npm config set proxy $http_proxy \
+    && npm config set https-proxy $https_proxy \
+    && npm config set engine-strict true \
+    && npm install
+
+# Start final image build
+FROM ubuntu:23.04
+
+# Create user and group before copying files
+ARG USER=pccs
+RUN useradd -M -U -r ${USER} -s /bin/false
+
+# Copy only necessary files from builder stage
+COPY --from=builder /usr/bin/node /usr/bin/node
+COPY --from=builder --chown=${USER}:${USER} /SGXDataCenterAttestationPrimitives/QuoteGeneration/pccs/ /opt/intel/pccs/
+
+# Set the working directory and switch user
+WORKDIR /opt/intel/pccs/
+USER ${USER}
+
+# Define entrypoint
+ENTRYPOINT ["/usr/bin/node", "pccs_server.js"]
diff --git a/README_Docker.md b/README_Docker.md
@@ -1,50 +1,52 @@
-# Raiko Docker tutorial
+# Raiko Docker Setup Tutorial
 
-This tutorial was created to help you set up Raiko using a Docker container.
+This tutorial was created to assist you in setting up Raiko and its SGX dependencies using a Docker container. Configuring SGX can be complex without a detailed guide to walk you through each step. This tutorial strives to provide a comprehensive walkthrough, leaving no detail unaddressed.
 
-Raiko leverages [Intel SGX][sgx] through [Gramine][gramine]. Since Gramine supports only [a few distributions][gramine-distros] including Ubuntu, the Docker image is based on Ubuntu.
+## Prerequisites
+
+Raiko leverages [Intel SGX][sgx] via [Gramine][gramine]. As Gramine only supports [a limited number of distributions][gramine-distros], including Ubuntu. The Docker image is derived from Gramine's base image, which uses Ubuntu.
+
+Intel SGX is a technology that involves a considerable amount of configuration. Given its high level of configurability, the setup of your infrastructure may vary significantly depending on the attestation type (EPID, ECDSA) and other parameters. While we've strived to minimize the manual effort required to prepare the development environment, there are certain prerequisites that are challenging, if not impossible, to automate using Dockerfiles. This section outlines these prerequisites.
 
 [gramine-distros]: https://github.com/gramineproject/gramine/discussions/1555#discussioncomment-7016800
 [gramine]: https://gramineproject.io/
 
-## Prerequisites
+### Intel SGX-enabled CPU
 
-### SGX-enabled CPU
+Ensure that your machine has an [Intel SGX][sgx]-enabled CPU to run Raiko. You can verify if your CPU supports SGX (Software Guard Extensions) on Linux by using the [`cpuid`][cpuid] tool.
 
-Ensure your machine has an [SGX][sgx]-enabled CPU to run raiko. You can check if your CPU supports SGX (Software Guard Extensions) on Linux by using the [`cpuid`][cpuid] tool.
+1. If `cpuid` isn't already installed, you can install it. On Ubuntu, use the following command:
 
-1.  Install `cpuid` if it's not already installed. On Ubuntu, you can do this with the following command:
+        sudo apt-get install cpuid
 
-    sudo apt-get install cpuid
-
-1.  Run `cpuid` and `grep` for SGX:
+1. Run `cpuid` and `grep` for `sgx`:
 
         cpuid | grep -i sgx
 
-    If your CPU supports SGX, you should see output similar to this:
+    If your CPU supports SGX, the output should resemble the following:
 
     ```
     SGX: Software Guard Extensions supported = true
     ```
 
-    If you don't see this line, your CPU does not support SGX.
+    If this line doesn't appear, your CPU either doesn't support SGX, or it isn't enabled in the BIOS.
 
-Alternatively, you can run `grep sgx /proc/cpuinfo`. If the command returns no output, your CPU doesn't support SGX.
+As an alternative, you can execute `grep sgx /proc/cpuinfo`. If the command doesn't return any output, your CPU doesn't support SGX.
 
 [sgx]: https://www.intel.com/content/www/us/en/architecture-and-technology/software-guard-extensions.html
 [cpuid]: https://manpages.ubuntu.com/manpages/noble/en/man1/cpuid.1.html
 
 ### Modern Linux kernel
 
-Starting with Linux kernel version [`5.11`][kernel-5.11], the kernel provides out-of-the-box support for SGX. However, it doesn't support [EDMM][edmm] (Enclave Dynamic Memory Management), which Raiko requires. EDMM support first appeared in Linux `6.0`, so ensure that you have Linux kernel `6.0` or above.
+Starting with Linux kernel version [`5.11`][kernel-5.11], the kernel provides built-in support for SGX. However, it doesn't support one of its latest features, [EDMM][edmm] (Enclave Dynamic Memory Management), which Raiko requires. EDMM support was first introduced in Linux `6.0`, so ensure that your Linux kernel version is `6.0` or above.
 
-To check version of your kernel run:
+To check the version of your kernel, run:
 
 ```
 uname -a
 ```
 
-If you are using Ubuntu and you want to find what are the available Linux kernel versions, run:
+If you're using Ubuntu and want to see the available Linux kernel versions, run the following command:
 
 ```
 apt search linux-image
@@ -53,14 +55,49 @@ apt search linux-image
 [kernel-5.11]: https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/linux-overview.html
 [edmm]: https://gramine.readthedocs.io/en/stable/manifest-syntax.html#edmm
 
+### Generating PCCS Certificates
+
+Before running the Raiko Docker container, you need to fulfill some SGX-specific prerequisites, which include setting up the [PCCS][pccs-readme] (Provisioning Certificate Caching Service) configuration. The PCCS service is responsible for retrieving PCK Certificates and other collaterals on-demand from the internet at runtime, and then caching them in a local database. The PCCS exposes similar HTTPS interfaces as Intel's Provisioning Certificate Service.
+
+Begin the configuration process by [generating][pccs-cert-gen] an SSL certificate:
+
+```
+mkdir ~/.config/sgx-pccs
+cd ~/.config/sgx-pccs
+openssl genrsa -out private.pem 2048
+chmod 644 private.pem  # Docker container needs access
+openssl req -new -key private.pem -out csr.pem
+openssl x509 -req -days 365 -in csr.pem -signkey private.pem -out file.crt
+rm csr.pem
+```
+
+[pccs-readme]: https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/pccs/README.md
+[pccs-cert-gen]: https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/QuoteGeneration/pccs/container#2-generate-certificates-to-use-with-pccs
+
+### Subscribing to Intel PCS Service
+
+To use ECDSA Attestation, you need to subscribe to the Intel PCS service, following the steps in [Intel's how-to guide][intel-dcap-install-howto]. After subscribing to the service, you will get two keys: a primary API key and a secondary API key.
+
+To use these keys in your configuration file, copy the `config/default.json` [template config file][pccs-readme] to `$HOME/.config/sgx-pccs`. The `raiko` container will mount this as a volume. After copying the file, open it for editing and fill in the below listed parameters as recommended by [Intel's manual][pccs-cert-gen-config]:
+- `ApiKey`: The PCCS uses this API key to request collaterals from Intel's Provisioning Certificate Service. User needs to subscribe first to obtain an API key. For how to subscribe to Intel Provisioning Certificate Service and receive an API key, goto https://api.portal.trustedservices.intel.com/provisioning-certification and click on `Subscribe`.
+- `UserTokenHash`: SHA512 hash of the user token for the PCCS client user to register a platform. For example, PCK Cert ID retrieval tool will use the user token to send platform information to PCCS. (`echo -n "user_password" | sha512sum | tr -d '[:space:]-'`).
+- `AdminTokenHash`: SHA512 hash of the administrator token for the PCCS administrator to perform a manual refresh of cached artifacts (`echo -n "admin_password" | sha512sum | tr -d '[:space:]-'`).
+
+[intel-dcap-install-howto]: https://www.intel.com/content/www/us/en/developer/articles/guide/intel-software-guard-extensions-data-center-attestation-primitives-quick-install-guide.html
+[pccs-cert-gen-config]: https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/QuoteGeneration/pccs/container#3-fill-up-configuration-file
+
 ## Building Docker image
 
-Taiko doesn't provide prebuilt Docker image (yet). You need to build it yourself.
+Taiko doesn't currently offer a prebuilt Docker image. You will need to build it yourself using the `docker-compose` file we provide. Two Docker images need to be built: `raiko` and the SGX-specific `pccs` service, which manages the lifecycle of the certificates required for [ECDSA attestation][ecdsa].
 
 1. Clone `raiko` repository:
    ```
    git clone git@github.com:taikoxyz/raiko.git
    ```
+1. If you wish to use the latest `raiko` version, choose the `taiko/unstable` branch (which is the default option at the time of writing this document).
+   ```
+   git checkout taiko/unstable
+   ```
 1. Change active directory:
    ```
    cd raiko/docker
@@ -69,30 +106,102 @@ Taiko doesn't provide prebuilt Docker image (yet). You need to build it yourself
    ```
    docker compose build
    ```
-1. That's it! You should now be able to find the `raiko:latest` in the list of all Docker images:
+1. That's it! You should now see two Raiko images, `raiko` and `pccs`, in your Docker images list. You can view this list by running the following command:
    ```
    docker image ls
    ```
 
-## Running Docker container
+[ecdsa]: https://github.com/cloud-security-research/sgx-ra-tls/blob/master/README-ECDSA.md
+
+## Running PCCS service
+
+Before starting Raiko, the PCCS service must be up and running. To run the PCCS service, you must configure it with the SSL certificate that you generated in a previous section of this document:
+
+```
+docker compose up pccs -d
+```
 
-After successfully building Docker image, you are now able to bootstrap and run Raiko as a daemon.
+Verify the successful start of the PCCS service by executing:
+
+```
+docker logs pccs
+```
+
+If everything is set up correctly, you should see:
+
+```
+HTTPS Server is running on: https://localhost:8081
+```
+
+You can now bootstrap and run Raiko as a daemon.
+
+## Retrieving PCK Certs
+
+Now, we need to retrieve Intel's PCK Certificates and populate the PCCS service with them. To do this, open the file `/opt/intel/sgx-pck-id-retrieval-tool/network_setting.conf` and add the following lines:
+
+```
+PCCS_URL=https://localhost:8082/sgx/certification/v3/platforms
+user_token=<USER_TOKEN>
+USE_SECURE_CERT=FALSE
+```
+
+Replace `<USER_TOKEN>` with the user password you got when you subscribed to the Intel PCS Service, as described in a previous steps of this tutorial.
+
+At this point, you should be ready to fetch Intel's certificates. You can do this by running the `PCKIDRetrievalTool`, which you can install either from the Ubuntu repository:
+```
+$ echo ’deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu
+focal main’ | sudo tee /etc/apt/sources.list.d/intel-sgx.list > /dev/null
+$ wget -O - https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add -
+$ sudo apt update
+$ sudo apt install sgx-pck-id-retrieval-tool
+```
+Or, you can [build and install][sgx-pck-id-retrieval-tool] it yourself. After you have installed it, run the following command:
+
+```
+PCKIDRetrievalTool
+```
+
+Alternatively, you can skip editing the `network_setting.conf` configuration file and directly run the following command:
+
+```
+PCKIDRetrievalTool -url https://localhost:8082/sgx/certification/v3/platforms -user_token '<USER_TOKEN>' -use_secure_cert false
+```
+
+If everything was successful, you should receive a non-empty response when you make the following request:
+
+```
+curl -k -G "https://localhost:8081/sgx/certification/v3/rootcacrl"
+```
+
+Now, you're ready to bootstrap and run Raiko!
+
+[sgx-pck-id-retrieval-tool]: https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/tools/PCKRetrievalTool/installer
 
 ### Raiko bootstrapping
 
-Bootstrapping is the process of generating a public-private key pair, which will be used for doing signatures within the SGX enclave. The private key is stored in an [encrypted][gramine-encrypted-files] format in the `~/.config/raiko/secrets/priv.key` file. Encryption and decryption are performed inside the enclave, providing protection against malicious attacks.
+Bootstrapping involves generating a public-private key pair, which is used for signatures within the SGX enclave. The private key is stored in an [encrypted][gramine-encrypted-files] format in the `~/.config/raiko/secrets/priv.key` file. The encryption and decryption processes occur inside the enclave, offering protection against malicious attacks.
 
 1. Make sure you haven't generated Raiko's public-private key pair yet:
    ```
    ls ~/.config/raiko/secrets
    ```
-   If you `secrets` directory is not empty, you can skip Raiko bootstrapping.
+   If you `secrets` directory isn't empty, you can skip Raiko bootstrapping.
 1. Bootstrap Raiko:
    ```
    docker compose run --rm raiko --init
    ```
-   It creates a new, encrypted private key in `~/.config/raiko/secrets` directory. It also prints a public key that you need to send to the Taiko team for registration.
-   Register the "Instance address"(pinted by `--init` command) with the Taiko team. Once the Taiko team registers your instance, you will be able to use it to sign proofs.
+   It creates a new, encrypted private key in `~/.config/raiko/secrets` directory and couple more configuration files under `$HOME/.config/raiko` directory:
+   ```
+   $ tree ~/.config/raiko
+   /home/ubuntu/.config/raiko
+   ├── config
+   │   ├── bootstrap.json
+   │   ├── raiko-guest.manifest.sgx
+   │   └── raiko-guest.sig
+   └── secrets
+      └── priv.key
+   ```
+   You can inspect your public key, instance ID, and SGX quote in the file `$HOME/.config/raiko/bootstrap.json`.
 
 [gramine-encrypted-files]: https://gramine.readthedocs.io/en/stable/manifest-syntax.html#encrypted-files
 
@@ -118,19 +227,20 @@ Now, once you have Raiko up and running, you can test it to make sure it is serv
 1. Send a sample request to Raiko:
    ```
    curl --location --request POST 'http://localhost:8080' --header 'Content-Type: application/json' --data-raw '{
-     "jsonrpc": "2.0",
-     "id": 1,
-     "method": "proof",
-     "params": [
-       {
-         "type": "Sgx",
-         "l2Rpc": "https://rpc.internal.taiko.xyz",
-         "l1Rpc": "https://l1rpc.internal.taiko.xyz",
-         "block": 2,
-         "prover": "0x70997970C51812dc3A010C7d01b50e0d17dc79C8",
-         "graffiti": "0000000000000000000000000000000000000000000000000000000000000000"
-       }
-     ]
+      "jsonrpc": "2.0",
+      "method": "proof",
+      "params": [
+         {
+               "type": "Sgx",
+               "block": 3000,
+               "l2Rpc": "https://rpc.internal.taiko.xyz/",
+               "l1Rpc": "https://l1rpc.internal.taiko.xyz/",
+               "l1BeaconRpc": "https://l1beacon.internal.taiko.xyz/",
+               "prover": "0x70997970C51812dc3A010C7d01b50e0d17dc79C8",
+               "graffiti": "0x6162630000000000000000000000000000000000000000000000000000000000"
+         }
+      ],
+      "id": 0
    }'
    ```
    If the request was served correctly, you should see a lot of logs being produced in the log file and an SGX proof printed on the standard output:

diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
@@ -1,6 +1,9 @@
 version: "3.9"
 services:
   raiko:
+    build:
+      context: ..
+      dockerfile: Dockerfile
     image: gcr.io/evmchain/raiko:latest
     container_name: raiko
     command: --config-path=/etc/raiko/config.toml
@@ -11,7 +14,19 @@ services:
       - /tmp/sgx:/tmp/sgx
       - /var/log/raiko:/var/log/raiko
       - ${HOME}/.config/gramine:/root/.config/gramine
-      # make sure you have ${HOME}/.config/raiko directory
       - ${HOME}/.config/raiko:/root/.config/raiko
     ports:
       - "8080:8080"
+    depends_on:
+      - pccs
+  pccs:
+    build:
+      context: ..
+      dockerfile: Dockerfile.pccs
+    container_name: pccs
+    volumes:
+      - ${HOME}/.config/sgx-pccs/default.json:/opt/intel/pccs/config/default.json
+      - ${HOME}/.config/sgx-pccs/file.crt:/opt/intel/pccs/ssl_key/file.crt
+      - ${HOME}/.config/sgx-pccs/private.pem:/opt/intel/pccs/ssl_key/private.pem
+    ports:
+      - "8082:8081"