Skip to content

Commit 7e4491f

Browse files
woody-appledavidgooglerestyled-commitsmikaelhm
authored
Adds Dockerfile to build cross-platform images for binaries used by the certification test harness. (Take 2) (#21140) (#21262)
* Add Dockerfile for building multiplatform certification binaries. * Updates and documentation for chip-cert-bins image. * Adds all-clusters-minimal and other updates to chip-cert-bins. * Restyled by prettier-markdown * Fix spell-checking issues * Update .wordlist.txt (#18) * Update .wordlist.txt * Update README.md Co-authored-by: Restyled.io <commits@restyled.io> Co-authored-by: Mikael Møller <mikaelhm@hey.com> Co-authored-by: davidgoogle <49926720+davidgoogle@users.noreply.github.com> Co-authored-by: Restyled.io <commits@restyled.io> Co-authored-by: Mikael Møller <mikaelhm@hey.com>
1 parent 6293e39 commit 7e4491f

File tree

4 files changed

+350
-1
lines changed

4 files changed

+350
-1
lines changed

‎.github/.wordlist.txt

+3
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,7 @@ BeagleBone
126126
befc
127127
betaprogram
128128
BinaryInputBasic
129+
Binfmt
129130
bitbake
130131
bld
131132
BLE
@@ -160,7 +161,9 @@ btmgmt
160161
BTN
161162
BTP
162163
btvirt
164+
BuildKit
163165
buildwithmatter
166+
buildX
164167
burndown
165168
ButtonIsr
166169
BytesMain

‎integrations/docker/images/build-all.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
# https://github.com/project-chip/connectedhomeip/issues/710
2222
#
2323
set -e
24-
find "$(git rev-parse --show-toplevel)"/integrations/docker/images/ -name Dockerfile | while read -r dockerfile; do
24+
find "$(git rev-parse --show-toplevel)"/integrations/docker/images/ -name Dockerfile ! -path "*chip-cert-bins/*" | while read -r dockerfile; do
2525
pushd "$(dirname "$dockerfile")" >/dev/null
2626
./build.sh "$@"
2727
popd >/dev/null

‎integrations/docker/images/chip-cert-bins/Dockerfile

+278
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
# Stage 1: Setup dependencies (based on chip-build).
2+
FROM ubuntu:22.04 as chip-build-cert
3+
ARG TARGETPLATFORM
4+
# COMMITHASH defines the target commit to build from. May be passed in using --build-arg.
5+
ARG COMMITHASH=e556daac2e1ed3a141034a6dcc7e410e4cd1f8f6
6+
7+
# Ensure TARGETPLATFORM is set
8+
RUN case ${TARGETPLATFORM} in \
9+
"linux/amd64") \
10+
echo "Building for linux/amd64" \
11+
;; \
12+
"linux/arm64") \
13+
echo "Building for linux/arm64" \
14+
;; \
15+
*) \
16+
if [ -z "$TARGETPLATFORM" ] ;\
17+
then \
18+
echo "TARGETPLATFORM not defined! Please run from buildkit (buildx)." \
19+
&& return 1 ;\
20+
else \
21+
echo "Unsupported platform ${TARGETPLATFORM}." \
22+
&& return 1 ;\
23+
fi \
24+
;; \
25+
esac
26+
27+
# Below should be the same as chip-build except arm64 logic for cmake and node.
28+
29+
# base build and check tools and libraries layer
30+
RUN set -x \
31+
&& apt-get update \
32+
&& DEBIAN_FRONTEND=noninteractive apt-get install -fy \
33+
autoconf \
34+
automake \
35+
bison \
36+
bridge-utils \
37+
clang \
38+
clang-format \
39+
clang-tidy \
40+
curl \
41+
flex \
42+
g++ \
43+
git \
44+
gperf \
45+
iproute2 \
46+
jq \
47+
lcov \
48+
libavahi-client-dev \
49+
libavahi-common-dev \
50+
libcairo2-dev \
51+
libdbus-1-dev \
52+
libdbus-glib-1-dev \
53+
libgif-dev \
54+
libglib2.0-dev \
55+
libical-dev \
56+
libjpeg-dev \
57+
libdmalloc-dev \
58+
libmbedtls-dev \
59+
libncurses5-dev \
60+
libncursesw5-dev \
61+
libnspr4-dev \
62+
libpango1.0-dev \
63+
libpixman-1-dev \
64+
libreadline-dev \
65+
libssl-dev \
66+
libtool \
67+
libudev-dev \
68+
libusb-1.0-0 \
69+
libusb-dev \
70+
libxml2-dev \
71+
make \
72+
net-tools \
73+
ninja-build \
74+
openjdk-8-jdk \
75+
pkg-config \
76+
python3 \
77+
python3-dev \
78+
python3-venv \
79+
rsync \
80+
shellcheck \
81+
strace \
82+
systemd \
83+
udev \
84+
unzip \
85+
wget \
86+
git-lfs \
87+
zlib1g-dev \
88+
&& rm -rf /var/lib/apt/lists/ \
89+
&& git lfs install \
90+
&& : # last line
91+
92+
# Cmake (Mbed OS requires >=3.19.0-rc3 version which is not available in Ubuntu 20.04 repository)
93+
RUN case ${TARGETPLATFORM} in \
94+
"linux/amd64") \
95+
set -x \
96+
&& (cd /tmp \
97+
&& wget --progress=dot:giga https://github.com/Kitware/CMake/releases/download/v3.19.3/cmake-3.19.3-Linux-x86_64.sh \
98+
&& sh cmake-3.19.3-Linux-x86_64.sh --exclude-subdir --prefix=/usr/local \
99+
&& rm -rf cmake-3.19.3-Linux-x86_64.sh) \
100+
&& exec bash \
101+
;; \
102+
"linux/arm64") \
103+
set -x \
104+
&& (cd /tmp \
105+
&& wget --progress=dot:giga https://github.com/Kitware/CMake/releases/download/v3.19.3/cmake-3.19.3-Linux-aarch64.sh \
106+
&& sh cmake-3.19.3-Linux-aarch64.sh --exclude-subdir --prefix=/usr/local \
107+
&& rm -rf cmake-3.19.3-Linux-aarch64.sh) \
108+
&& exec bash \
109+
;; \
110+
*) \
111+
test -n "$TARGETPLATFORM" \
112+
echo "Unsupported platform ${TARGETPLATFORM}" \
113+
;; \
114+
esac
115+
116+
# Python 3 and PIP
117+
RUN set -x \
118+
&& DEBIAN_FRONTEND=noninteractive apt-get update \
119+
&& DEBIAN_FRONTEND=noninteractive apt-get install -y libgirepository1.0-dev \
120+
&& DEBIAN_FRONTEND=noninteractive apt-get install -y software-properties-common \
121+
&& add-apt-repository universe \
122+
&& curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py \
123+
&& python3 get-pip.py \
124+
&& rm -rf /var/lib/apt/lists/ \
125+
&& : # last line
126+
127+
RUN set -x \
128+
&& pip3 install attrs coloredlogs PyGithub pygit future portpicker mobly click cxxfilt ghapi pandas tabulate \
129+
&& : # last line
130+
131+
# build and install gn
132+
RUN set -x \
133+
&& git clone https://gn.googlesource.com/gn \
134+
&& cd gn \
135+
&& python3 build/gen.py \
136+
&& ninja -C out \
137+
&& cp out/gn /usr/local/bin \
138+
&& cd .. \
139+
&& rm -rf gn \
140+
&& : # last line
141+
142+
# Install bloat comparison tools
143+
RUN set -x \
144+
&& git clone https://github.com/google/bloaty.git \
145+
&& mkdir -p bloaty/build \
146+
&& cd bloaty/build \
147+
&& cmake ../ \
148+
&& make -j8 \
149+
&& make install \
150+
&& cd ../.. \
151+
&& rm -rf bloaty \
152+
&& : # last line
153+
154+
# NodeJS: install a newer version than what apt-get would read
155+
# This installs the latest LTS version of nodejs
156+
RUN case ${TARGETPLATFORM} in \
157+
"linux/amd64") \
158+
set -x \
159+
&& mkdir node_js \
160+
&& cd node_js \
161+
&& wget https://nodejs.org/dist/v12.19.0/node-v12.19.0-linux-x64.tar.xz \
162+
&& tar xfvJ node-v12.19.0-linux-x64.tar.xz \
163+
&& mv node-v12.19.0-linux-x64 /opt/ \
164+
&& ln -s /opt/node-v12.19.0-linux-x64 /opt/node \
165+
&& ln -s /opt/node/bin/* /usr/bin \
166+
&& cd .. \
167+
&& rm -rf node_js \
168+
;; \
169+
"linux/arm64")\
170+
set -x \
171+
&& mkdir node_js \
172+
&& cd node_js \
173+
&& wget https://nodejs.org/dist/v12.19.0/node-v12.19.0-linux-arm64.tar.xz \
174+
&& tar xfvJ node-v12.19.0-linux-arm64.tar.xz \
175+
&& mv node-v12.19.0-linux-arm64 /opt/ \
176+
&& ln -s /opt/node-v12.19.0-linux-arm64 /opt/node \
177+
&& ln -s /opt/node/bin/* /usr/bin \
178+
&& cd .. \
179+
&& rm -rf node_js \
180+
;; \
181+
*) ;; \
182+
esac
183+
184+
# Stage 1.5: Bootstrap Matter.
185+
RUN mkdir /root/connectedhomeip
186+
RUN git clone https://github.com/project-chip/connectedhomeip.git /root/connectedhomeip
187+
WORKDIR /root/connectedhomeip/
188+
RUN git checkout ${COMMITHASH}
189+
RUN scripts/build/gn_bootstrap.sh
190+
RUN gn gen out/debug --args='chip_mdns="platform" chip_inet_config_enable_ipv4=false'
191+
RUN ninja -C out/debug
192+
193+
# Stage 2: Build.
194+
from chip-build-cert as chip-build-cert-bins
195+
SHELL ["/bin/bash", "-c"]
196+
# Records Matter SDK commit hash to include in the image.
197+
RUN git rev-parse HEAD > /root/.sdk-sha-version
198+
RUN case ${TARGETPLATFORM} in \
199+
"linux/amd64") \
200+
set -x \
201+
&& source scripts/activate.sh \
202+
&& scripts/build/build_examples.py \
203+
--target linux-x64-all-clusters-ipv6only \
204+
--target linux-x64-all-clusters-minimal-ipv6only \
205+
--target linux-x64-bridge-ipv6only \
206+
--target linux-x64-tv-app-ipv6only \
207+
--target linux-x64-tv-casting-app-ipv6only \
208+
--target linux-x64-light-ipv6only \
209+
--target linux-x64-thermostat-ipv6only \
210+
--target linux-x64-ota-provider-ipv6only \
211+
--target linux-x64-ota-requestor-ipv6only \
212+
--target linux-x64-lock-ipv6only \
213+
build \
214+
&& mv out/linux-x64-all-clusters-ipv6only/chip-all-clusters-app out/chip-all-clusters-app \
215+
&& mv out/linux-x64-all-clusters-minimal-ipv6only/chip-all-clusters-minimal-app out/chip-all-clusters-minimal-app \
216+
&& mv out/linux-x64-bridge-ipv6only/chip-bridge-app out/chip-bridge-app \
217+
&& mv out/linux-x64-tv-app-ipv6only/chip-tv-app out/chip-tv-app \
218+
&& mv out/linux-x64-tv-casting-app-ipv6only/chip-tv-casting-app out/chip-tv-casting-app \
219+
&& mv out/linux-x64-light-ipv6only/chip-lighting-app out/chip-lighting-app \
220+
&& mv out/linux-x64-thermostat-ipv6only/thermostat-app out/thermostat-app \
221+
&& mv out/linux-x64-ota-provider-ipv6only/chip-ota-provider-app out/chip-ota-provider-app \
222+
&& mv out/linux-x64-ota-requestor-ipv6only/chip-ota-requestor-app out/chip-ota-requestor-app \
223+
&& mv out/linux-x64-lock-ipv6only/chip-lock-app out/chip-lock-app \
224+
;; \
225+
"linux/arm64")\
226+
set -x \
227+
&& source scripts/activate.sh \
228+
&& scripts/build/build_examples.py \
229+
--target linux-arm64-all-clusters-ipv6only \
230+
--target linux-arm64-all-clusters-minimal-ipv6only \
231+
--target linux-arm64-bridge-ipv6only \
232+
--target linux-arm64-tv-app-ipv6only \
233+
--target linux-arm64-tv-casting-app-ipv6only \
234+
--target linux-arm64-light-ipv6only \
235+
--target linux-arm64-thermostat-ipv6only \
236+
--target linux-arm64-ota-provider-ipv6only \
237+
--target linux-arm64-ota-requestor-ipv6only \
238+
--target linux-arm64-lock-ipv6only \
239+
build \
240+
&& mv out/linux-arm64-all-clusters-ipv6only/chip-all-clusters-app out/chip-all-clusters-app \
241+
&& mv out/linux-arm64-all-clusters-minimal-ipv6only/chip-all-clusters-minimal-app out/chip-all-clusters-minimal-app \
242+
&& mv out/linux-arm64-bridge-ipv6only/chip-bridge-app out/chip-bridge-app \
243+
&& mv out/linux-arm64-tv-app-ipv6only/chip-tv-app out/chip-tv-app \
244+
&& mv out/linux-arm64-tv-casting-app-ipv6only/chip-tv-casting-app out/chip-tv-casting-app \
245+
&& mv out/linux-arm64-light-ipv6only/chip-lighting-app out/chip-lighting-app \
246+
&& mv out/linux-arm64-thermostat-ipv6only/thermostat-app out/thermostat-app \
247+
&& mv out/linux-arm64-ota-provider-ipv6only/chip-ota-provider-app out/chip-ota-provider-app \
248+
&& mv out/linux-arm64-ota-requestor-ipv6only/chip-ota-requestor-app out/chip-ota-requestor-app \
249+
&& mv out/linux-arm64-lock-ipv6only/chip-lock-app out/chip-lock-app \
250+
;; \
251+
*) ;; \
252+
esac
253+
254+
RUN npm --prefix third_party/zap/repo/ ci
255+
RUN scripts/examples/gn_build_test_example.sh app1
256+
257+
# Stage 3: Copy relevant cert bins to a minimal image to reduce size.
258+
FROM ubuntu:22.04
259+
ENV TZ=Etc/UTC
260+
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
261+
RUN apt-get update -y
262+
RUN apt-get install -y libssl-dev libdbus-1-dev libglib2.0-dev libavahi-client-dev avahi-utils iproute2
263+
WORKDIR /root/
264+
COPY --from=chip-build-cert-bins /root/.sdk-sha-version .sdk-sha-version
265+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/debug/chip-tool chip-tool
266+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/debug/chip-shell chip-shell
267+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/debug/chip-cert chip-cert
268+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-all-clusters-app chip-all-clusters-app
269+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-all-clusters-minimal-app chip-all-clusters-minimal-app
270+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-lighting-app chip-lighting-app
271+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-tv-casting-app chip-tv-casting-app
272+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-tv-app chip-tv-app
273+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-bridge-app chip-bridge-app
274+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/thermostat-app thermostat-app
275+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-ota-provider-app chip-ota-provider-app
276+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-ota-requestor-app chip-ota-requestor-app
277+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/chip-lock-app chip-lock-app
278+
COPY --from=chip-build-cert-bins /root/connectedhomeip/out/app1/chip-app1 chip-app1

‎integrations/docker/images/chip-cert-bins/README.md

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Docker image for Matter Certification Test Harness
2+
3+
The Dockerfile here helps build multi-platform Docker images containing the
4+
executable binaries necessary for the Matter Test Harness. It utilizes the
5+
BuildKit toolkit and Buildx, included within Docker since version 18.06.
6+
7+
## Running
8+
9+
In order to properly run the binaries, avahi must be properly set up and passed
10+
to the container.
11+
12+
Prerequisites:
13+
14+
- Host must support and enable IPv6 and be on a network that has IPv6.
15+
- IPv6 must be enabled within avahi config on the host. `use-ipv6=yes` in
16+
avahi-daemon.conf
17+
- Sometimes there are stale avahi entries, so restarting avahi-daemon between
18+
runs may be necessary.
19+
20+
The host network and dbus must be exposed to the container for avahi to work
21+
properly. So for an interactive prompt, use:
22+
23+
```
24+
docker run -it --network host -v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket chip-cert-bins
25+
```
26+
27+
## Building
28+
29+
The Dockerfile requires building using the Buildx plugin, included within
30+
docker. It is used to build for both the amd64 and arm64 architectures, so the
31+
image may be cross-built and ran directly on a Raspberry Pi or other arm64 based
32+
environment. If your docker installation does not have the Buildx plugin, please
33+
update docker or install Buildx manually.
34+
35+
Prerequisites:
36+
37+
- A recent docker installation.
38+
- Create a Buildx builder: `docker buildx create --use --name mybuild`
39+
- Install the Binfmt cross-platform Docker emulators:
40+
`docker run --privileged --rm tonistiigi/binfmt --install all`
41+
42+
### Example: Building for the host platform and loading into Docker
43+
44+
```
45+
docker buildx build --load .
46+
```
47+
48+
The above command will build the image and load them into your local Docker
49+
instance.
50+
51+
### Example: Building for another platform and exporting to a tar
52+
53+
```
54+
docker buildx build --platform linux/arm64 --output "dest=/full/path/to/dest/chipcertbins.tar,type=docker" .
55+
```
56+
57+
The above command will build the image and export it to a tar file. You may copy
58+
the tar file to a RaspberryPi and import the image by using:
59+
60+
```
61+
docker load --input chipcertbins.tar
62+
```
63+
64+
### Example: Creating a multi-platform image and pushing to the Docker registry
65+
66+
```
67+
docker buildx build --platform linux/amd64,linux/arm64 --tag chip-cert-bins:tag1 --push .
68+
```

0 commit comments

Comments
 (0)