forked from apache/pulsar
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[fix][C++ client] Fix rpm and deb packaging
### Motivation For branch-2.11 and master branches, the rpm and deb packaging is broken. > /usr/bin/env: 'python3': No such file or directory It's caused by [PIP-155](apache#15376), which removes the Python2 support. In addition, the Dockerfiles of `rpm` and `deb` subdirectories of `pulsar-client-cpp/pkg` don't install Boost for Python3 well. But the rpm build disables the `BUILD_PYTHON_WRAPPER` option while the deb build doesn't. ### Modifications - Fix Dockerfiles under `pkg` and `rpm` subdirectories and speed up the `docker build` process: - Install python3 to fix the bug - Download the CMake 3.24.0 binaries directly instead of building from source - Upgrade Boost to 1.79 so that only headers are required (the boost::regex module is required for CentOS 7 because the default GCC is 4.8 that there is something wrong with std::regex) - Add `-j8` option when compiling some large dependencies that uses 8 threads to speed the compilation - Remove the `-v` option of `tar` to avoid showing too many info - Check the `BUILD_IMAGE` environment variable in `docker-build-xxx.sh` under `rpm` and `deb` subdirectories. If it's defined, build the image rather than pull the image from DockerHub before executing the build script. Because currently only a few committers have the permission to push the Docker images. - Add two workflows to build RPM and DEB packages if the files under `pulsar-client-cpp` changed. They build the docker images and run the build script for RPM and DEB packaging. - Specify the platform to `linux/amd64` so that the scripts can be used in ARM64 machines like Mac M1.
- Loading branch information
1 parent
70020f1
commit 96fb9d0
Showing
9 changed files
with
260 additions
and
96 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
|
|
||
| name: CI - CPP DEB packaging | ||
| on: | ||
| pull_request: | ||
| branches: | ||
| - master | ||
| paths: | ||
| - '.github/workflows/**' | ||
| - 'pulsar-client-cpp/**' | ||
| push: | ||
| branches: | ||
| - branch-* | ||
| paths: | ||
| - '.github/workflows/**' | ||
| - 'pulsar-client-cpp/**' | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
|
|
||
| cpp-deb-packaging: | ||
| name: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 120 | ||
|
|
||
| steps: | ||
| - name: checkout | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Tune Runner VM | ||
| uses: ./.github/actions/tune-runner-vm | ||
|
|
||
| - name: Detect changed files | ||
| id: changes | ||
| uses: apache/pulsar-test-infra/paths-filter@master | ||
| with: | ||
| filters: .github/changes-filter.yaml | ||
| list-files: csv | ||
|
|
||
| - name: Check changed files | ||
| id: check_changes | ||
| run: echo "::set-output name=docs_only::${{ fromJSON(steps.changes.outputs.all_count) == fromJSON(steps.changes.outputs.docs_count) && fromJSON(steps.changes.outputs.docs_count) > 0 }}" | ||
|
|
||
| - name: Cache local Maven repository | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: | | ||
| ~/.m2/repository/*/*/* | ||
| !~/.m2/repository/org/apache/pulsar | ||
| key: ${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-m2-dependencies-core-modules- | ||
| - name: Set up JDK 17 | ||
| uses: actions/setup-java@v2 | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| with: | ||
| distribution: 'temurin' | ||
| java-version: 17 | ||
|
|
||
| - name: clean disk | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| run: | | ||
| sudo apt clean | ||
| docker rmi $(docker images -q) -f | ||
| df -h | ||
| - name: Package Pulsar source | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| run: mvn -B -ntp -q clean package -pl pulsar-client-api -am -DskipTests | ||
|
|
||
| - name: Build Debian packages | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| run: | | ||
| echo "Build Debian packages" | ||
| BUILD_IMAGE=1 pulsar-client-cpp/pkg/deb/docker-build-deb.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
|
|
||
| name: CI - CPP RPM packaging | ||
| on: | ||
| pull_request: | ||
| branches: | ||
| - master | ||
| paths: | ||
| - '.github/workflows/**' | ||
| - 'pulsar-client-cpp/**' | ||
| push: | ||
| branches: | ||
| - branch-* | ||
| paths: | ||
| - '.github/workflows/**' | ||
| - 'pulsar-client-cpp/**' | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
|
|
||
| cpp-rpm-packaging: | ||
| name: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 120 | ||
|
|
||
| steps: | ||
| - name: checkout | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Tune Runner VM | ||
| uses: ./.github/actions/tune-runner-vm | ||
|
|
||
| - name: Detect changed files | ||
| id: changes | ||
| uses: apache/pulsar-test-infra/paths-filter@master | ||
| with: | ||
| filters: .github/changes-filter.yaml | ||
| list-files: csv | ||
|
|
||
| - name: Check changed files | ||
| id: check_changes | ||
| run: echo "::set-output name=docs_only::${{ fromJSON(steps.changes.outputs.all_count) == fromJSON(steps.changes.outputs.docs_count) && fromJSON(steps.changes.outputs.docs_count) > 0 }}" | ||
|
|
||
| - name: Cache local Maven repository | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: | | ||
| ~/.m2/repository/*/*/* | ||
| !~/.m2/repository/org/apache/pulsar | ||
| key: ${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-m2-dependencies-core-modules- | ||
| - name: Set up JDK 17 | ||
| uses: actions/setup-java@v2 | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| with: | ||
| distribution: 'temurin' | ||
| java-version: 17 | ||
|
|
||
| - name: clean disk | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| run: | | ||
| sudo apt clean | ||
| docker rmi $(docker images -q) -f | ||
| df -h | ||
| - name: Package Pulsar source | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| run: mvn -B -ntp -q clean package -pl pulsar-client-api -am -DskipTests | ||
|
|
||
| - name: Build RPM packages | ||
| if: ${{ steps.check_changes.outputs.docs_only != 'true' }} | ||
| run: | | ||
| echo "Build RPM packages" | ||
| BUILD_IMAGE=1 pulsar-client-cpp/pkg/rpm/docker-build-rpm.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.