Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update linux_headers_image to support >= 6.3 kernels. Add 6.2 through 6.11 kernels #2036

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update linux_headers_image to support >= 6.3 kernels. Add 6.2 through 6.11 kernels #2036

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8b36ac9
Update linux_headers_image to support >= 6.3 kernels. Add additional …
ddelnano Oct 2, 2024
dde3f85
Ensure pre 6.3 dockerfile remains the same as before
ddelnano Oct 2, 2024
147d58d
Fix version detection for kernesl < 6 and fix arm64 builds
ddelnano Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 28 additions & 2 deletions tools/docker/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,13 @@ LINUX_KERNEL_VERSIONS := 4.14.304 \
5.18.19 \
5.19.17 \
6.0.19 \
6.1.8
6.1.8 \
6.2.16 \
6.3.13 \
6.6.53 \
6.8.12 \
6.10.12 \
6.11.1
Comment on lines +104 to +110
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an official process for how to pick these? Since I had conditional logic for pre 6.3 and post 6.3, I wanted the latest minor version on either side, in addition to some number of kernels between 6.3 and 6.11.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last time I did this I picked the newest patch version for each.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check the change in the size of the headers tar with these changes? I believe we were worried about the size increase here. cc @oazizi000

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should also update this comment https://github.com/pixie-io/pixie/blob/3c41d554215528e688328aef94192e696db617dc/src/stirling/BUILD.bazel#L68:72

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change adds 63MB to the existing header file and brings its total size to 315MB. That's a pretty big addition, so I'd be open to trimming down the added versions. I think we could select one version in between 6.1 and 6.7/6.8 and one from 6.10/6.11 (since #2035 is related to those versions).

Comment updated and will keep it in sync with what we decide as the right set of new headers to include.


LINUX_HEADER_X86_64_TEMPLATE := linux-headers-x86_64-%.tar.gz
LINUX_HEADER_ARM64_TEMPLATE := linux-headers-arm64-%.tar.gz
Expand Down Expand Up @@ -210,19 +216,39 @@ ifndef LINUX_HEADERS_REV
$(error set LINUX_HEADERS_REV to a valid value)
endif
@mkdir -p $(@D)
# Linux 6.3 and later has differences in the build process.
# We need to use a different Dockerfile for these versions.
KERN_MAJ=$$(echo $* | cut -d. -f1); \
KERN_MIN=$$(echo $* | cut -d. -f2); \
HEADERS_DOCKERFILE_SUFFIX=""; \
if [ $${KERN_MAJ} -le 6 ] && [ $${KERN_MIN} -lt 3 ]; then \
HEADERS_DOCKERFILE_SUFFIX=".bionic"; \
fi; \
linux_headers_image_tag="gcr.io/pixie-oss/pixie-dev-public:$*-$(LINUX_HEADERS_REV)"; \
$(DOCKER) build --build-arg KERN_VERSION=$* --build-arg ARCH=x86_64 linux_headers_image -t "$${linux_headers_image_tag}" && \
$(DOCKER) build --build-arg KERN_VERSION=$* \
--build-arg ARCH=x86_64 linux_headers_image \
-t "$${linux_headers_image_tag}" \
-f linux_headers_image/Dockerfile$${HEADERS_DOCKERFILE_SUFFIX} && \
$(DOCKER) run --rm -v $(PWD)/$(LINUX_HEADER_ASSETS_BUILD_DIR):/output "$${linux_headers_image_tag}"

$(LINUX_HEADER_ASSETS_BUILD_DIR)/linux-headers-arm64-%.tar.gz: linux_headers_image/Dockerfile linux_headers_image/arm64_config
ifndef LINUX_HEADERS_REV
$(error set LINUX_HEADERS_REV to a valid value)
endif
@mkdir -p $(@D)
# Linux 6.3 and later has differences in the build process.
# We need to use a different Dockerfile for these versions.
KERN_MAJ=$$(echo $* | cut -d. -f1); \
KERN_MIN=$$(echo $* | cut -d. -f2); \
HEADERS_DOCKERFILE_SUFFIX=""; \
if [ $${KERN_MAJ} -le 6 ] && [ $${KERN_MIN} -lt 3 ]; then \
HEADERS_DOCKERFILE_SUFFIX=".bionic"; \
fi; \
linux_headers_image_tag="gcr.io/pixie-oss/pixie-dev-public:$*-$(LINUX_HEADERS_REV)"; \
$(DOCKER) build --build-arg KERN_VERSION=$* \
--build-arg ARCH=arm64 \
--build-arg CROSS_COMPILE=aarch64-linux-gnu- \
-f linux_headers_image/Dockerfile$(HEADERS_DOCKERFILE_SUFFIX) \
linux_headers_image -t "$${linux_headers_image_tag}" && \
$(DOCKER) run --rm -v $(PWD)/$(LINUX_HEADER_ASSETS_BUILD_DIR):/output "$${linux_headers_image_tag}"

Expand Down
11 changes: 7 additions & 4 deletions tools/docker/linux_headers_image/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@
#
# SPDX-License-Identifier: Apache-2.0

FROM ubuntu:18.04@sha256:8aa9c2798215f99544d1ce7439ea9c3a6dfd82de607da1cec3a8a2fae005931b
# debhelper-compat (= 12) needed for kernels >= 6.3
FROM ubuntu:20.04@sha256:6d8d9799fe6ab3221965efac00b4c34a2bcc102c086a58dff9e19a08b913c7ef

# Install required packages
RUN apt-get update
RUN apt-get upgrade -y -q
RUN apt-get install -y -q build-essential \
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -q build-essential \
bc \
libelf-dev \
libssl-dev \
Expand All @@ -32,6 +33,7 @@ RUN apt-get install -y -q build-essential \
binutils-aarch64-linux-gnu \
gcc-aarch64-linux-gnu \
dwarves \
debhelper \
python3

ARG KERN_VERSION
Expand All @@ -54,11 +56,12 @@ WORKDIR /px/src/linux-${KERN_VERSION}
RUN cp /configs/${ARCH} .config
RUN make ARCH=${ARCH} olddefconfig
RUN make ARCH=${ARCH} clean
RUN make ARCH=${ARCH} -j $(nproc) deb-pkg LOCALVERSION=-pl
RUN make ARCH=${ARCH} -j $(nproc) bindeb-pkg LOCALVERSION=-pl

# Extract headers into a tarball
WORKDIR /px
RUN DEB_ARCH=$(echo ${ARCH} | sed 's/x86_64/amd64/g'); dpkg -x src/linux-headers-${KERN_VERSION}-pl_${KERN_VERSION}-pl-1_${DEB_ARCH}.deb .
# Linux 6.3 and later name the resulting package differently (there is no additional -pl suffix)
RUN DEB_ARCH=$(echo ${ARCH} | sed 's/x86_64/amd64/g'); dpkg -x src/linux-headers-${KERN_VERSION}-pl_${KERN_VERSION}-1_${DEB_ARCH}.deb .

# Remove broken symlinks
RUN find usr/src/linux-headers-${KERN_VERSION}-pl -xtype l -exec rm {} +
Expand Down
78 changes: 78 additions & 0 deletions tools/docker/linux_headers_image/Dockerfile.bionic
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Copyright 2018- The Pixie Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

FROM ubuntu:18.04@sha256:8aa9c2798215f99544d1ce7439ea9c3a6dfd82de607da1cec3a8a2fae005931b

# Install required packages
RUN apt-get update
RUN apt-get upgrade -y -q
RUN apt-get install -y -q build-essential \
bc \
libelf-dev \
libssl-dev \
flex \
bison \
kmod \
cpio \
rsync \
wget \
binutils-aarch64-linux-gnu \
gcc-aarch64-linux-gnu \
dwarves \
python3

ARG KERN_VERSION

# Download Linux sources
WORKDIR /px/src
RUN KERN_MAJ="$(echo "${KERN_VERSION}" | cut -d'.' -f1)"; \
wget -nv http://mirrors.edge.kernel.org/pub/linux/kernel/v${KERN_MAJ}.x/linux-${KERN_VERSION}.tar.gz
RUN tar zxf linux-${KERN_VERSION}.tar.gz

WORKDIR /configs
ADD x86_64_config /configs/x86_64
ADD arm64_config /configs/arm64

ARG ARCH
ARG CROSS_COMPILE

# Build Linux kernel
WORKDIR /px/src/linux-${KERN_VERSION}
RUN cp /configs/${ARCH} .config
RUN make ARCH=${ARCH} olddefconfig
RUN make ARCH=${ARCH} clean
RUN make ARCH=${ARCH} -j $(nproc) bindeb-pkg LOCALVERSION=-pl

# Extract headers into a tarball
WORKDIR /px
RUN DEB_ARCH=$(echo ${ARCH} | sed 's/x86_64/amd64/g'); dpkg -x src/linux-headers-${KERN_VERSION}-pl_${KERN_VERSION}-pl-1_${DEB_ARCH}.deb .

# Remove broken symlinks
RUN find usr/src/linux-headers-${KERN_VERSION}-pl -xtype l -exec rm {} +

# Remove uneeded files to reduce size
# Keep only:
# - usr/src/linux-headers-x.x.x-pl/include
# - usr/src/linux-headers-x.x.x-pl/arch/${ARCH}
# This reduces the size by a little over 2x.
RUN rm -rf usr/share
RUN find usr/src/linux-headers-${KERN_VERSION}-pl -maxdepth 1 -mindepth 1 ! -name include ! -name arch -type d \
-exec rm -rf {} +
RUN find usr/src/linux-headers-${KERN_VERSION}-pl/arch -maxdepth 1 -mindepth 1 ! -name $(echo ${ARCH} | sed 's/x86_64/x86/g') -type d -exec rm -rf {} +
RUN tar zcf linux-headers-${ARCH}-${KERN_VERSION}.tar.gz usr

VOLUME /output
CMD ["sh", "-c", "cp linux-headers-*.tar.gz /output/"]
Loading