Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(build): link openssl statically #2984

Merged
merged 5 commits into from
Jun 5, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .config/hakari.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,6 @@ platforms = [

# Write out exact versions rather than a semver range. (Defaults to false.)
# exact-versions = true

[traversal-excludes]
workspace-members = ["workspace-static-link"]
3 changes: 1 addition & 2 deletions .github/aws-config/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,5 @@ CMD bash
ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update -y && apt-get install -yy ca-certificates lsb-release wget
RUN apt-get install -yy curl lcov git unzip zip libssl-dev zlib1g-dev cmake build-essential
RUN apt-get install -yy curl lcov git unzip zip zlib1g-dev cmake build-essential
RUN apt-get install -yy pkg-config make golang psmisc

4 changes: 1 addition & 3 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# ================= THIS FILE IS AUTOMATICALLY GENERATED =================
#
# To edit this file, please refer to the instructions in CONTRIBUTING.md.
Expand Down Expand Up @@ -641,7 +640,7 @@ jobs:
- name: Run rust clippy check
run: |
# If new CI checks are added, the one with `--locked` must be run first.
cargo clippy --all-targets --all-features --locked -- -D warnings
cargo clippy --all-targets --features failpoints --locked -- -D warnings
- name: Build documentation
run: |
cargo doc --document-private-items --no-deps
Expand Down Expand Up @@ -739,7 +738,6 @@ jobs:
asset_path: risingwave-${{ github.ref_name }}-x86_64-unknown-linux.tar.gz
asset_name: risingwave-${{ github.ref_name }}-x86_64-unknown-linux.tar.gz
asset_content_type: application/tar+gzip

# ================= THIS FILE IS AUTOMATICALLY GENERATED =================
#
# To edit this file, please refer to the instructions in CONTRIBUTING.md.
Expand Down
34 changes: 22 additions & 12 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ members = [
"src/utils/logging",
"src/utils/memcomparable",
"src/utils/pgwire",
"src/utils/static-link",
"src/utils/value-encoding",
"src/workspace-hack",
]
Expand Down
2 changes: 1 addition & 1 deletion Makefile.toml
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ script = """

set -e

for tool in cargo-llvm-cov cargo-nextest cargo-udeps cargo-hakari cargo-sort
for tool in cargo-llvm-cov cargo-nextest cargo-udeps cargo-hakari cargo-sort cargo-make
do
echo "install: $(tput setaf 4)$tool$(tput sgr0)"
cargo install $tool
Expand Down
2 changes: 1 addition & 1 deletion ci/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ ENV LANG en_US.utf8

RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get -y install make build-essential cmake protobuf-compiler curl \
openssl libssl-dev pkg-config bash openjdk-11-jdk wget unzip git tmux lld
openssl libssl-dev libcurl4-openssl-dev pkg-config bash openjdk-11-jdk wget unzip git tmux lld

SHELL ["/bin/bash", "-c"]

Expand Down
5 changes: 4 additions & 1 deletion ci/scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,14 @@ echo "--- Rust format check"
cargo fmt --all -- --check

echo "--- Build Rust components"
cargo build -p risingwave_cmd_all -p risedev -p risingwave_regress_test --profile "$profile"
cargo build -p risingwave_cmd_all -p risedev -p risingwave_regress_test --features static-link --profile "$profile"

echo "--- Compress RisingWave debug info"
objcopy --compress-debug-sections=zlib-gnu target/"$target"/risingwave

echo "--- Show link info"
ldd target/"$target"/risingwave

echo "--- Upload artifacts"
cp target/"$target"/risingwave ./risingwave-"$profile"
cp target/"$target"/risedev-playground ./risedev-playground-"$profile"
Expand Down
4 changes: 2 additions & 2 deletions ci/scripts/unit-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ rustup default "$(cat ./rust-toolchain)" && rustup component add llvm-tools-prev
cargo install cargo-llvm-cov

echo "--- Run clippy check"
cargo clippy --all-targets --all-features --locked -- -D warnings
cargo clippy --all-targets --features failpoints --locked -- -D warnings

echo "--- Build documentation"
cargo doc --document-private-items --no-deps

echo "--- Run unit tests with coverage"
cargo llvm-cov nextest --lcov --output-path lcov.info --features=failpoints -- --no-fail-fast
cargo llvm-cov nextest --lcov --output-path lcov.info --features failpoints -- --no-fail-fast

echo "--- Run doctest"
cargo test --doc
Expand Down
6 changes: 3 additions & 3 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM ubuntu:22.04 as builder

ENV LANG en_US.utf8

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install make build-essential cmake protobuf-compiler curl openssl libssl-dev pkg-config bash lld
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install make build-essential cmake protobuf-compiler curl pkg-config bash lld

SHELL ["/bin/bash", "-c"]

Expand All @@ -24,12 +24,12 @@ RUN rustup self update \

RUN cargo fetch
RUN mkdir -p /risingwave/bin
RUN cargo build -p risingwave_cmd -p risingwave_cmd_all --release && \
RUN cargo build -p risingwave_cmd -p risingwave_cmd_all --release --features static-link && \
mv /risingwave/target/release/{frontend,compute-node,meta-node,compactor,risingwave} /risingwave/bin/ && \
cargo clean

FROM ubuntu:22.04 as image-base
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install openssl libssl-dev ca-certificates && rm -rf /var/lib/{apt,dpkg,cache,log}/
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install ca-certificates && rm -rf /var/lib/{apt,dpkg,cache,log}/

FROM image-base as frontend-node
RUN mkdir -p /risingwave/bin
Expand Down
2 changes: 1 addition & 1 deletion docker/aws/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM ubuntu:22.04
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install openssl libssl-dev ca-certificates && rm -rf /var/lib/{apt,dpkg,cache,log}/
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install ca-certificates && rm -rf /var/lib/{apt,dpkg,cache,log}/

RUN mkdir -p /risingwave/bin
COPY risingwave /risingwave/bin/risingwave
Expand Down
2 changes: 1 addition & 1 deletion docker/aws/aws-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ if ! [[ "$DIR" =~ ^/ebs.* ]] ; then
fi

cd "$DIR/../.."
cargo build -p risingwave_cmd_all --release
cargo build -p risingwave_cmd_all --release --features static-link
objcopy --compress-debug-sections=zlib-gnu target/release/risingwave "$DIR/risingwave"

cd "$DIR"
Expand Down
2 changes: 1 addition & 1 deletion docs/developer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
To install the dependencies on Debian-based Linux systems, run:

```shell
sudo apt install make build-essential cmake protobuf-compiler curl openssl libssl-dev pkg-config postgresql-client tmux lld
sudo apt install make build-essential cmake protobuf-compiler curl openssl libssl-dev libcurl4-openssl-dev pkg-config postgresql-client tmux lld
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

Expand Down
4 changes: 4 additions & 0 deletions src/cmd/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ name = "risingwave_cmd"
version = "0.1.8"
edition = "2021"

[features]
static-link = ["workspace-static-link"]

[dependencies]
clap = { version = "3", features = ["derive"] }
log = { version = "0.4", features = ["release_max_level_info"] }
Expand All @@ -25,6 +28,7 @@ tokio = { version = "=0.2.0-alpha.3", package = "madsim-tokio", features = [
] }
tracing = { version = "0.1", features = ["release_max_level_info"] }
workspace-hack = { version = "0.1", path = "../workspace-hack" }
workspace-static-link = { path = "../utils/static-link", optional = true }

[[bin]]
name = "frontend"
Expand Down
4 changes: 4 additions & 0 deletions src/cmd_all/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ name = "risingwave_cmd_all"
version = "0.1.8"
edition = "2021"

[features]
static-link = ["workspace-static-link"]

[dependencies]
anyhow = "1"
clap = { version = "3", features = ["derive"] }
Expand All @@ -27,6 +30,7 @@ tokio = { version = "=0.2.0-alpha.3", package = "madsim-tokio", features = [
] }
tracing = { version = "0.1", features = ["release_max_level_info"] }
workspace-hack = { version = "0.1", path = "../workspace-hack" }
workspace-static-link = { path = "../utils/static-link", optional = true }

[[bin]]
name = "risingwave"
Expand Down
1 change: 1 addition & 0 deletions src/object_store/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ tokio = { version = "=0.2.0-alpha.3", package = "madsim-tokio", features = [
"fs",
] }
tracing = { version = "0.1" }
workspace-hack = { version = "0.1", path = "../workspace-hack" }
2 changes: 1 addition & 1 deletion src/risedevtool/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ console = "0.15"
dialoguer = "0.10"
enum-iterator = "1"
indicatif = "0.16"
isahc = "1"
isahc = { version = "1", default-features = false, features = ["text-decoding"] }
itertools = "0.10"
kafka = { version = "0.9", default-features = false }
regex = "1"
Expand Down
2 changes: 1 addition & 1 deletion src/utils/logging/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ edition = "2021"
[dependencies]
async-trait = "0.1"
futures = { version = "0.3", default-features = false, features = ["alloc"] }
isahc = "1" # isahc is the http client used for tracing. Always set it as the same version as opentelemetry-jaeger's.
isahc = { version = "1", default-features = false } # isahc is the http client used for tracing. Always set it as the same version as opentelemetry-jaeger's.
madsim = "=0.2.0-alpha.3"
opentelemetry = { version = "0.17", features = ["rt-tokio", "trace"] }
opentelemetry-jaeger = { version = "0.16", features = [
Expand Down
11 changes: 11 additions & 0 deletions src/utils/static-link/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[package]
name = "workspace-static-link"
version = "0.1.8"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
isahc = { version = "1", default-features = false, features = ["static-ssl", "static-curl"] }
openssl = { version = "0.10", features = ["vendored"] }
# workspace-hack = { version = "0.1", path = "../../workspace-hack" }
# Don't add workspace-hack into this crate!
8 changes: 8 additions & 0 deletions src/utils/static-link/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#[cfg(test)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[cfg(test)]
// Copyright 2022 Singularity Data
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#[cfg(test)]

mod tests {
#[test]
fn it_works() {
let result = 2 + 2;
assert_eq!(result, 4);
}
}
8 changes: 4 additions & 4 deletions src/workspace-hack/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ bstr = { version = "0.2", features = ["lazy_static", "regex-automata", "serde",
bytes = { version = "1", features = ["serde", "std"] }
chrono = { version = "0.4", features = ["clock", "libc", "oldtime", "std", "time", "winapi"] }
crossbeam-deque = { version = "0.8", features = ["crossbeam-epoch", "crossbeam-utils", "std"] }
crossbeam-utils-c38e5c1d305a1b54 = { package = "crossbeam-utils", version = "0.8", features = ["lazy_static", "std"] }
crossbeam-utils-ca01ad9e24f5d932 = { package = "crossbeam-utils", version = "0.7", features = ["lazy_static", "std"] }
crossbeam-utils = { version = "0.7", features = ["lazy_static", "std"] }
crypto-common = { version = "0.1", default-features = false, features = ["std"] }
digest = { version = "0.10", features = ["alloc", "block-buffer", "core-api", "mac", "std", "subtle"] }
either = { version = "1", features = ["use_std"] }
Expand All @@ -36,6 +35,7 @@ futures-util = { version = "0.3", features = ["alloc", "async-await", "async-awa
hashbrown = { version = "0.11", features = ["ahash", "inline-more", "raw"] }
hyper = { version = "0.14", features = ["client", "full", "h2", "http1", "http2", "runtime", "server", "socket2", "stream", "tcp"] }
indexmap = { version = "1", default-features = false, features = ["std"] }
isahc = { version = "1", default-features = false, features = ["encoding_rs", "mime", "text-decoding"] }
libc = { version = "0.2", features = ["std"] }
libz-sys = { version = "1", features = ["libc", "stock-zlib"] }
lock_api = { version = "0.4", default-features = false, features = ["arc_lock"] }
Expand Down Expand Up @@ -73,8 +73,7 @@ bytes = { version = "1", features = ["serde", "std"] }
cc = { version = "1", default-features = false, features = ["jobserver", "parallel"] }
chrono = { version = "0.4", features = ["clock", "libc", "oldtime", "std", "time", "winapi"] }
crossbeam-deque = { version = "0.8", features = ["crossbeam-epoch", "crossbeam-utils", "std"] }
crossbeam-utils-c38e5c1d305a1b54 = { package = "crossbeam-utils", version = "0.8", features = ["lazy_static", "std"] }
crossbeam-utils-ca01ad9e24f5d932 = { package = "crossbeam-utils", version = "0.7", features = ["lazy_static", "std"] }
crossbeam-utils = { version = "0.7", features = ["lazy_static", "std"] }
crypto-common = { version = "0.1", default-features = false, features = ["std"] }
digest = { version = "0.10", features = ["alloc", "block-buffer", "core-api", "mac", "std", "subtle"] }
either = { version = "1", features = ["use_std"] }
Expand All @@ -90,6 +89,7 @@ futures-util = { version = "0.3", features = ["alloc", "async-await", "async-awa
hashbrown = { version = "0.11", features = ["ahash", "inline-more", "raw"] }
hyper = { version = "0.14", features = ["client", "full", "h2", "http1", "http2", "runtime", "server", "socket2", "stream", "tcp"] }
indexmap = { version = "1", default-features = false, features = ["std"] }
isahc = { version = "1", default-features = false, features = ["encoding_rs", "mime", "text-decoding"] }
libc = { version = "0.2", features = ["std"] }
libz-sys = { version = "1", features = ["libc", "stock-zlib"] }
lock_api = { version = "0.4", default-features = false, features = ["arc_lock"] }
Expand Down