Skip to content
This repository has been archived by the owner on Dec 7, 2023. It is now read-only.

Completely restructure & document the kernel image build #506

Merged
merged 5 commits into from
Feb 17, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions images/kernel/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,9 @@ RUN git fetch --tags
RUN git checkout v${KERNEL_VERSION} && \
make clean && make mrproper

COPY config-${KERNEL_VERSION}${KERNEL_EXTRA} .config
COPY generated/config-amd64-${KERNEL_VERSION}${KERNEL_EXTRA} .config

RUN make EXTRAVERSION=${KERNEL_EXTRA} LOCALVERSION= olddefconfig && \
make EXTRAVERSION=${KERNEL_EXTRA} LOCALVERSION= olddefconfig
RUN make EXTRAVERSION=${KERNEL_EXTRA} LOCALVERSION= olddefconfig

RUN make EXTRAVERSION=${KERNEL_EXTRA} LOCALVERSION= -j32
RUN make EXTRAVERSION=${KERNEL_EXTRA} LOCALVERSION= modules_install
Expand Down
27 changes: 19 additions & 8 deletions images/kernel/Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,22 @@
KERNEL_VERSION ?= 4.19.47
KERNEL_EXTRA ?=
# Check https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/refs/ for updates
KERNEL_VERSIONS ?= 4.14.166 4.19.97 5.4.13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting that these will not overwrite the old images @ https://hub.docker.com/r/weaveworks/ignite-kernel/tags since they are all version bumps 👍

Copy link

@taqtiqa-mark taqtiqa-mark Feb 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies in advance ... I'm just getting my head around Ignite - by spec'ing and writing a Cheff Cookbook to install ignite and ignited:
Is there a Linux kernel dependency to take into consideration when installing Ignite?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@taqtiqa-mark these kernel versions are for the VM guests that ignite runs. They're distributed as OCI images so you don't have to worry about installing them.

For installing ignite, you need a linux host capable of meeting Firecrackers minimum runtime dependencies.
Currently, that is a host with Linux Kernel 4.14 or newer and KVM enabled:

Please see our documented requirements:


all: build
build:
docker build -t weaveworks/ignite-kernel:${KERNEL_VERSION}${KERNEL_EXTRA} \
--build-arg KERNEL_VERSION=${KERNEL_VERSION} \
--build-arg KERNEL_EXTRA=${KERNEL_EXTRA} .

push:
docker push weaveworks/ignite-kernel:${KERNEL_VERSION}${KERNEL_EXTRA}
upgrade: $(addprefix upgrade-,$(KERNEL_VERSIONS))
./patch-config.sh
for file in generated/*; do \
./upgrade-config.sh $$file $$file; done
Comment on lines +8 to +9
Copy link
Contributor

@stealthybox stealthybox Feb 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me understand why this is here?
I'm reading that this runs

     make clean && make mrproper && cp /tmp/.config . && \
     make EXTRAVERSION="" LOCALVERSION= olddefconfig

on the patched config as the input.
That output then overwrites the input patched config.

Does this cleanup or do something to the generated/patched config that changes it?
Thanks 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This basically sanitizes, reorganizes and formats (overwrites) the kernel config for the final generated configs


upgrade-%:
for file in upstream/*; do \
./upgrade-config.sh $$file versioned/$$(basename $$file)-$*; done

build: $(addprefix build-,$(KERNEL_VERSIONS))
build-%:
docker build -t weaveworks/ignite-kernel:$* \
--build-arg KERNEL_VERSION=$* .

push: build $(addprefix push-,$(KERNEL_VERSIONS))
push-%:
docker push weaveworks/ignite-kernel:${KERNEL_VERSION}
41 changes: 41 additions & 0 deletions images/kernel/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Kernel Images

These kernel OCI images contain the kernel binary (at `/boot/vmlinux`) and supporting modules (in `/lib/modules`)
for guest VMs ran by Ignite.

## Building the Kernel Images

```console
$ make
```

## Versions

All LTS versions starting from 4.14 and above are supported by the Ignite team.
This means in practice:

- 4.14.x
- 4.19.x
- 5.4.x

The exact patch versions may be found in the [Makefile](Makefile).
The available versions exist in the [stable kernel git tree](https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/refs/).

## Upgrading to a new kernel version

It should be as easy as:
luxas marked this conversation as resolved.
Show resolved Hide resolved

```console
$ make upgrade
```

after you've upgraded the values in the Makefile.
Copy link
Contributor

@stealthybox stealthybox Feb 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
after you've upgraded the values in the Makefile.
after you've set new `KERNEL_VERSIONS` in the Makefile or changed `./config-patches`.
This will generate new patched configs for each version.
Older kernel versions should be deleted from `generated/` and `versioned/` when checking in new ones.


## Kernel Config Parameters we care about

Some options to the kernel are specifically important for making guest software work.

Please see: [config-patches](config-patches) for what kernel configs we've changed.
The base kernel config is the MicroVM-optimized config file from the Firecracker team.
We're storing it in [upstream/config-amd64](upstream/config-amd64). It's available online
at [firecracker/resources](https://github.com/firecracker-microvm/firecracker/tree/master/resources).
22 changes: 22 additions & 0 deletions images/kernel/config-patches
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# In this file, the recipe for patching all kernel configs (all versions & architectures) is

# For making Weave Net work
CONFIG_DUMMY=y
# Enable VXLAN support as a module so that e.g. Flannel works
CONFIG_VXLAN=m
# Enable support for soft shutdown of amd64 VMs
# See https://github.com/firecracker-microvm/firecracker/blob/master/docs/api_requests/actions.md#sendctrlaltdel
CONFIG_KEYBOARD_ATKBD=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_LIBPS2=y
# Make the guest's wall clock not drift
# https://github.com/firecracker-microvm/firecracker/blob/master/FAQ.md#my-guest-wall-clock-is-drifting-how-can-i-fix-it
CONFIG_PTP_1588_CLOCK=y
CONFIG_PTP_1588_CLOCK_KVM=y

# Some patches for keeping network functionalities that Kubernetes needs/might need
CONFIG_IPVLAN=y
CONFIG_IPVTAP=y
CONFIG_TAP=y
CONFIG_IP_VS_MH=m
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86 4.14.123 Kernel Configuration
# Linux/x86 4.14.166 Kernel Configuration
#
CONFIG_64BIT=y
CONFIG_X86_64=y
Expand Down Expand Up @@ -592,6 +592,9 @@ CONFIG_ARCH_RANDOM=y
CONFIG_X86_SMAP=y
# CONFIG_X86_INTEL_MPX is not set
CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS=y
CONFIG_X86_INTEL_TSX_MODE_OFF=y
# CONFIG_X86_INTEL_TSX_MODE_ON is not set
# CONFIG_X86_INTEL_TSX_MODE_AUTO is not set
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
Expand Down Expand Up @@ -1313,6 +1316,7 @@ CONFIG_DNS_RESOLVER=m
# CONFIG_BATMAN_ADV is not set
CONFIG_OPENVSWITCH=m
CONFIG_OPENVSWITCH_GRE=m
CONFIG_OPENVSWITCH_VXLAN=m
CONFIG_VSOCKETS=y
CONFIG_VIRTIO_VSOCKETS=y
CONFIG_VIRTIO_VSOCKETS_COMMON=y
Expand Down Expand Up @@ -1837,7 +1841,7 @@ CONFIG_I2C_ALGOBIT=m
# CONFIG_SPI is not set
# CONFIG_SPMI is not set
# CONFIG_HSI is not set
CONFIG_PPS=m
CONFIG_PPS=y
CONFIG_PPS_DEBUG=y

#
Expand All @@ -1854,12 +1858,12 @@ CONFIG_PPS_CLIENT_GPIO=m
#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK=m
CONFIG_PTP_1588_CLOCK=y

#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
CONFIG_PTP_1588_CLOCK_KVM=m
CONFIG_PTP_1588_CLOCK_KVM=y
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
# CONFIG_POWER_AVS is not set
Expand Down Expand Up @@ -3260,6 +3264,7 @@ CONFIG_ASSOCIATIVE_ARRAY=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT_MAP=y
CONFIG_HAS_DMA=y
# CONFIG_SGL_ALLOC is not set
# CONFIG_DMA_NOOP_OPS is not set
# CONFIG_DMA_VIRT_OPS is not set
CONFIG_CPU_RMAP=y
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86 4.19.47 Kernel Configuration
# Linux/x86 4.19.97 Kernel Configuration
#

#
# Compiler: gcc (Ubuntu 7.4.0-1ubuntu1~18.10) 7.4.0
# Compiler: gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
#
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=70400
CONFIG_CLANG_VERSION=0
CONFIG_CC_HAS_ASM_GOTO=y
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_EXTABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y
Expand Down Expand Up @@ -358,6 +359,9 @@ CONFIG_X86_SMAP=y
CONFIG_X86_INTEL_UMIP=y
# CONFIG_X86_INTEL_MPX is not set
CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS=y
CONFIG_X86_INTEL_TSX_MODE_OFF=y
# CONFIG_X86_INTEL_TSX_MODE_ON is not set
# CONFIG_X86_INTEL_TSX_MODE_AUTO is not set
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
Expand Down Expand Up @@ -758,7 +762,7 @@ CONFIG_PACKET_DIAG=m
CONFIG_UNIX=y
CONFIG_UNIX_DIAG=m
CONFIG_TLS=m
CONFIG_TLS_DEVICE=y
# CONFIG_TLS_DEVICE is not set
CONFIG_XFRM=y
CONFIG_XFRM_OFFLOAD=y
CONFIG_XFRM_ALGO=y
Expand Down Expand Up @@ -1338,6 +1342,7 @@ CONFIG_DNS_RESOLVER=m
# CONFIG_BATMAN_ADV is not set
CONFIG_OPENVSWITCH=m
CONFIG_OPENVSWITCH_GRE=m
CONFIG_OPENVSWITCH_VXLAN=m
CONFIG_VSOCKETS=y
CONFIG_VSOCKETS_DIAG=y
CONFIG_VIRTIO_VSOCKETS=y
Expand Down Expand Up @@ -1393,7 +1398,6 @@ CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_DST_CACHE=y
CONFIG_GRO_CELLS=y
CONFIG_SOCK_VALIDATE_XMIT=y
# CONFIG_NET_DEVLINK is not set
CONFIG_MAY_USE_DEVLINK=y
CONFIG_FAILOVER=y
Expand Down Expand Up @@ -1871,7 +1875,7 @@ CONFIG_I2C_ALGOBIT=m
# CONFIG_SPI is not set
# CONFIG_SPMI is not set
# CONFIG_HSI is not set
CONFIG_PPS=m
CONFIG_PPS=y
CONFIG_PPS_DEBUG=y

#
Expand All @@ -1888,12 +1892,12 @@ CONFIG_PPS_CLIENT_GPIO=m
#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK=m
CONFIG_PTP_1588_CLOCK=y

#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
CONFIG_PTP_1588_CLOCK_KVM=m
CONFIG_PTP_1588_CLOCK_KVM=y
# CONFIG_PINCTRL is not set
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
Expand Down Expand Up @@ -2165,10 +2169,10 @@ CONFIG_DRM_PANEL_ORIENTATION_QUIRKS=m
#
# Frame buffer Devices
#
CONFIG_FB=m
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_CMDLINE=y
CONFIG_FB_NOTIFY=y
CONFIG_FB=m
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_CFB_FILLRECT=m
CONFIG_FB_CFB_COPYAREA=m
CONFIG_FB_CFB_IMAGEBLIT=m
Expand Down Expand Up @@ -2266,7 +2270,7 @@ CONFIG_HID_GENERIC=m
# CONFIG_HID_LOGITECH is not set
# CONFIG_HID_MAGICMOUSE is not set
# CONFIG_HID_MAYFLASH is not set
# CONFIG_HID_REDRAGON is not set
CONFIG_HID_REDRAGON=y
# CONFIG_HID_MICROSOFT is not set
# CONFIG_HID_MONTEREY is not set
# CONFIG_HID_MULTITOUCH is not set
Expand Down Expand Up @@ -2404,7 +2408,6 @@ CONFIG_IOMMU_SUPPORT=y
#
# CONFIG_RPMSG_QCOM_GLINK_RPM is not set
# CONFIG_RPMSG_VIRTIO is not set
# CONFIG_SOUNDWIRE is not set

#
# SOC (System On Chip) specific Drivers
Expand Down
Loading