Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix docker build scripts and docker dependency #46

Merged
merged 3 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 6 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ Build all of the DSS artifacts and its dependency artifacts using one script:
Optionally, build only the dependencies artifacts:

```bash
./scripts/docker/build_gcc.sh
./scripts/docker/build_aws-sdk.sh
./scripts/docker/build_kernel.sh
./scripts/docker/build_mlnx-tools.sh
Expand Down Expand Up @@ -85,25 +84,22 @@ sudo yum install bc bison boost-devel cmake cmake3 CUnit-devel devtoolset-11 dpk
sudo python3 -m pip install pybind11 gcovr==5.0
```

**NOTE: User-built GCC and AWS-SDK-CPP RPMs must be installed on the build machine.**
**NOTE: User-built AWS-SDK-CPP RPM must be installed on the build machine.**

On initial build:

1. Build GCC: `./scripts/build_gcc.sh`
2. Install the resulting GCC RPM: `sudo yum install ./dss-ansible/artifacts/dss-gcc510*.rpm -y`
3. Build AWS-SDK-CPP: `./scripts/build_aws-sdk.sh`
4. Install the resulting AWS-SDK-CPP RPM: `sudo yum install ./dss-ansible/artifacts/aws-sdk-cpp-*.rpm -y`
5. Run the `build_all.sh` script: `./scripts/build_all.sh`
1. Build AWS-SDK-CPP: `./scripts/build_aws-sdk.sh`
2. Install the resulting AWS-SDK-CPP RPM: `sudo yum install ./dss-ansible/artifacts/aws-sdk-cpp-*.rpm -y`
3. Run the `build_all.sh` script: `./scripts/build_all.sh`

Once the GCC and AWS RPMs are installed, only the `build_all.sh` script needs to be run on subsequent builds.
Once the AWS RPM is installed, only the `build_all.sh` script needs to be run on subsequent builds.

Dependency artifacts for GCC, kernel, aws-sdk-cpp, and mlnx-tools are staged under `rpmbuilder` and `workspace` directories of your home directory by default. By leaving them in-place, re-build of these upstream components will be skipped on subsequent builds.
Dependency artifacts for kernel, aws-sdk-cpp, and mlnx-tools are staged under `rpmbuilder` and `workspace` directories of your home directory by default. By leaving them in-place, re-build of these upstream components will be skipped on subsequent builds.

### Optional: Build individual components

DSS Dependency build scripts:

- Build GCC: `./scripts/build_gcc.sh`
- Build aws-sdk-cpp: `./scripts/build_aws-sdk.sh`
- Build kernel: `./scripts/build_kernel.sh`
- Build mlnx-tools: `./scripts/build_mlnx-tools.sh`
Expand Down
1 change: 0 additions & 1 deletion scripts/build_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
. "$SCRIPT_DIR/utils.sh"

# Build dependency artificats
"$SCRIPT_DIR/build_gcc.sh"
"$SCRIPT_DIR/build_aws-sdk.sh"
"$SCRIPT_DIR/build_kernel.sh"
"$SCRIPT_DIR/build_mlnx-tools.sh"
Expand Down
4 changes: 1 addition & 3 deletions scripts/docker/DSS.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ FROM centos:centos7.8.2003
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8

COPY dss-ansible/artifacts/dss-gcc510-*.rpm ./
COPY dss-ansible/artifacts/aws-sdk-cpp-*.rpm ./
RUN set -eux \
&& yum install -y \
Expand Down Expand Up @@ -50,7 +49,6 @@ RUN set -eux \
tbb-devel \
wget \
zlib-devel \
/dss-gcc510*.rpm \
/aws-sdk-cpp*.rpm && \
rm -f ./*.rpm && \
yum clean all && \
Expand All @@ -65,7 +63,7 @@ RUN set -eux \
"ansible>=2.9,<2.10" \
ansible-lint==5.3.2 \
gcovr==5.0 \
pybind11 \
pybind11==2.11.1 \
pycodestyle==2.8.0 \
shellcheck-py==0.8.0.3 \
yamllint==1.26.3 && \
Expand Down
1 change: 0 additions & 1 deletion scripts/docker/build_all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ DOCKER_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
. "$DOCKER_DIR/../utils.sh"

# DSS Dependencies
"$DOCKER_DIR"/build_gcc.sh
"$DOCKER_DIR"/build_aws-sdk.sh
"$DOCKER_DIR"/build_kernel.sh
"$DOCKER_DIR"/build_mlnx-tools.sh
Expand Down
2 changes: 1 addition & 1 deletion scripts/docker/build_dss-client.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ set -e

# Build aws-sdk-cpp in Docker
DOCKER_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
"$DOCKER_DIR"/build_docker.sh -c dss-client -d DSS -t dssbuild -a 'dss_client-*.tgz' -u
"$DOCKER_DIR"/build_docker.sh -c dss-client -d DSS -t dssbuild -a 'dss_client-*.tgz'
2 changes: 1 addition & 1 deletion scripts/docker/build_dss-sdk.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ set -e

# Build dss-sdk in Docker
DOCKER_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
"$DOCKER_DIR"/build_docker.sh -c dss-sdk -d DSS -t dssbuild -a 'nkv-sdk-bin-*.tgz' -u
"$DOCKER_DIR"/build_docker.sh -c dss-sdk -d DSS -t dssbuild -a 'nkv-sdk-bin-*.tgz nkv-target-*.tgz'
2 changes: 1 addition & 1 deletion scripts/docker/build_minio.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ set -e

# Build dss-gcc in Docker
DOCKER_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
"$DOCKER_DIR"/build_docker.sh -c minio -d DSS -t dssbuild -a 'dss-minio-bin-' -u
"$DOCKER_DIR"/build_docker.sh -c minio -d DSS -t dssbuild -a 'dss-minio-bin-*'
Loading