Skip to content

Commit 536bdcb

Browse files
lucasssvazme-no-dev
lucasssvaz
and
me-no-dev
authoredMay 13, 2024··
Add lib-builder docker image (#155)
* Test docker Image * Fix Dockerfile * Fix file permissions * Fix bugs and add run script * Update tools/docker/Dockerfile * Add textual to docker image * Reorder * Fix * Test * Fix * Add UI checks * Fix docker image * Fix exec in docker * Move argument * Add workflow * Add checks to entrypoint * Improvements to Dockerfile * Test example run scripts * Fix powershell script * Change parameter name * Add comment * Add Readme * Fix comparison * Remove cache * Fix comment * Improve permission settings * Add warning * Update readme * Fix repositories --------- Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>
·
idf-release_v5.5idf-master
1 parent 0c37f25 commit 536bdcb

File tree

8 files changed

+388
-3
lines changed

8 files changed

+388
-3
lines changed
 

Diff for: ‎.github/workflows/docker.yml

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Build and push Docker image
2+
3+
on:
4+
push:
5+
branches:
6+
- 'master'
7+
- 'release/*'
8+
tags:
9+
- 'v*.*'
10+
11+
env:
12+
# Build the image for amd64 and arm64
13+
BUILD_PLATFORMS: linux/amd64,linux/arm64
14+
DOCKERHUB_REPO: ${{ github.repository_owner }}/esp32-arduino-lib-builder
15+
16+
jobs:
17+
docker:
18+
# Disable the job in forks
19+
if: ${{ github.repository_owner == 'espressif' }}
20+
21+
runs-on: ubuntu-latest
22+
steps:
23+
# Depending on the branch/tag, set CLONE_BRANCH_OR_TAG variable (used in the Dockerfile
24+
# as a build arg) and TAG_NAME (used when tagging the image).
25+
#
26+
# The following 3 steps cover the alternatives (tag, release branch, master branch):
27+
- name: Set variables (tags)
28+
if: ${{ github.ref_type == 'tag' }}
29+
run: |
30+
echo "CLONE_BRANCH_OR_TAG=$GITHUB_REF_NAME" >> $GITHUB_ENV
31+
echo "TAG_NAME=$GITHUB_REF_NAME" >> $GITHUB_ENV
32+
- name: Set variables (release branches)
33+
if: ${{ github.ref_type == 'branch' && startsWith(github.ref_name, 'release/') }}
34+
run: |
35+
echo "CLONE_BRANCH_OR_TAG=$GITHUB_REF_NAME" >> $GITHUB_ENV
36+
echo "TAG_NAME=release-${GITHUB_REF_NAME##release/}" >> $GITHUB_ENV
37+
- name: Set variables (main branch)
38+
if: ${{ github.ref_type == 'branch' && github.ref_name == 'master' }}
39+
run: |
40+
echo "CLONE_BRANCH_OR_TAG=master" >> $GITHUB_ENV
41+
echo "TAG_NAME=latest" >> $GITHUB_ENV
42+
43+
# Display the variables set above, just in case.
44+
- name: Check variables
45+
run: |
46+
echo "CLONE_BRANCH_OR_TAG: $CLONE_BRANCH_OR_TAG"
47+
echo "CHECKOUT_REF: $CHECKOUT_REF"
48+
echo "TAG_NAME: $TAG_NAME"
49+
50+
# The following steps are the standard boilerplate from
51+
# https://github.com/marketplace/actions/build-and-push-docker-images
52+
- name: Checkout
53+
uses: actions/checkout@v4
54+
- name: Login to Docker Hub
55+
uses: docker/login-action@v3
56+
with:
57+
username: ${{ secrets.DOCKERHUB_USERNAME }}
58+
password: ${{ secrets.DOCKERHUB_TOKEN }}
59+
- name: Set up QEMU for multiarch builds
60+
uses: docker/setup-qemu-action@v3
61+
- name: Set up Docker Buildx
62+
uses: docker/setup-buildx-action@v3
63+
- name: Build and push
64+
uses: docker/build-push-action@v5
65+
with:
66+
context: tools/docker
67+
push: true
68+
tags: ${{ env.DOCKERHUB_REPO }}:${{ env.TAG_NAME }}
69+
platforms: ${{ env.BUILD_PLATFORMS }}
70+
build-args: |
71+
LIBBUILDER_CLONE_URL=${{ github.server_url }}/${{ github.repository }}.git
72+
LIBBUILDER_CLONE_BRANCH_OR_TAG=${{ env.CLONE_BRANCH_OR_TAG }}
73+
74+
- name: Update Docker Hub repository description (master branch)
75+
if: ${{ github.ref_type == 'branch' && github.ref_name == 'master' }}
76+
uses: peter-evans/dockerhub-description@v4
77+
with:
78+
username: ${{ secrets.DOCKERHUB_USERNAME }}
79+
password: ${{ secrets.DOCKERHUB_TOKEN }}
80+
repository: ${{ env.DOCKERHUB_REPO }}
81+
readme-filepath: ./tools/docker/README.md

Diff for: ‎tools/config_editor/app.py

+14-2
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,6 @@ def compose(self) -> ComposeResult:
8686

8787
def on_mount(self) -> None:
8888
# Event handler called when the app is mounted for the first time
89-
self.title = "Configurator"
9089
self.sub_title = "Main Menu"
9190
print("Main screen mounted.")
9291

@@ -104,6 +103,7 @@ class ConfigEditorApp(App):
104103
# Options to be set by the command line arguments
105104
setting_target = ""
106105
setting_arduino_path = ""
106+
setting_output_permissions = ""
107107
setting_arduino_branch = ""
108108
setting_idf_branch = ""
109109
setting_idf_commit = ""
@@ -131,6 +131,7 @@ def on_mount(self) -> None:
131131
print("IDF Branch: " + str(self.setting_idf_branch))
132132
print("IDF Commit: " + str(self.setting_idf_commit))
133133
print("IDF Debug Level: " + str(self.setting_debug_level))
134+
self.title = "Configurator"
134135
self.push_screen("main")
135136

136137
def arduino_default_path():
@@ -199,6 +200,13 @@ def main() -> None:
199200
required=False,
200201
help="Path to arduino-esp32 directory. Default: " + arduino_default_path())
201202

203+
parser.add_argument("--output-permissions",
204+
metavar="<uid:gid>",
205+
type=str,
206+
default="",
207+
required=False,
208+
help=argparse.SUPPRESS) # Hidden option. It is only supposed to be used by the docker container
209+
202210
parser.add_argument("-A", "--arduino-branch",
203211
metavar="<arduino branch>",
204212
type=str,
@@ -256,14 +264,18 @@ def main() -> None:
256264
elif args.arduino_path == arduino_default_path():
257265
print("Warning: Default Arduino path not found. Disabling copy to Arduino.")
258266
app.setting_enable_copy = False
267+
elif args.arduino_path == "/arduino-esp32": # Docker mount point
268+
print("Warning: Docker mount point not found. Disabling copy to Arduino.")
269+
app.setting_enable_copy = False
259270
else:
260-
print("Invalid path to Arduino core: " + os.path.abspath(args.arduino_path))
271+
print("Error: Invalid path to Arduino core: " + os.path.abspath(args.arduino_path))
261272
exit(1)
262273
else:
263274
app.setting_enable_copy = False
264275

265276
# Set the other options
266277
app.setting_arduino_path = os.path.abspath(args.arduino_path)
278+
app.setting_output_permissions = args.output_permissions
267279
app.setting_arduino_branch = args.arduino_branch
268280
app.setting_idf_branch = args.idf_branch
269281
app.setting_idf_commit = args.idf_commit

Diff for: ‎tools/config_editor/compile.py

+41-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import sys
22
import subprocess
33
import os
4+
import re
45

56
from rich.console import RenderableType
67

@@ -60,6 +61,11 @@ def print_success(self, message: str) -> None:
6061
self.button_widget.add_class("-success")
6162
#print("Success: " + message) # For debugging
6263

64+
def print_warning(self, message: str) -> None:
65+
# Print warning message to the RichLog widget
66+
self.log_widget.write("[b bright_yellow]" + message)
67+
#print("Warning: " + message) # For debugging
68+
6369
def print_info(self, message: str) -> None:
6470
# Print info message to the RichLog widget
6571
self.log_widget.write("[b bright_cyan]" + message)
@@ -72,7 +78,12 @@ def compile_libs(self) -> None:
7278

7379
label = self.query_one("#compile-title", Static)
7480
self.child_process = None
75-
if self.app.setting_target == ",".join(self.app.supported_targets):
81+
82+
if not self.app.setting_target:
83+
self.print_error("No target selected")
84+
label.update("No target selected")
85+
return
86+
elif self.app.setting_target == ",".join(self.app.supported_targets):
7687
target = "all targets"
7788
else:
7889
target = self.app.setting_target.replace(",", ", ").upper()
@@ -133,6 +144,30 @@ def compile_libs(self) -> None:
133144
print("Error reading child process errors: " + str(e))
134145
label.update("Compilation failed for " + target)
135146
else:
147+
if self.app.setting_output_permissions:
148+
regex = r"^[1-9][0-9]*:[1-9][0-9]*$" # Regex to match the uid:gid format. Note that 0:0 (root) is not allowed
149+
if re.match(regex, self.app.setting_output_permissions):
150+
print_info("Setting permissions to: " + self.app.setting_output_permissions)
151+
chown_process = None
152+
try:
153+
chown_process = subprocess.run(["chown", "-R", self.app.setting_output_permissions, self.app.setting_arduino_path])
154+
chown_process.wait()
155+
except Exception as e:
156+
print("Error changing permissions: " + str(e))
157+
158+
if chown_process and chown_process.returncode != 0:
159+
self.print_error("Error changing permissions")
160+
self.print_error("Please change the ownership of generated files manually")
161+
else:
162+
self.print_success("Permissions changed successfully")
163+
elif self.app.setting_output_permissions == "0:0":
164+
self.print_warning("Permissions settings are set to root (0:0)")
165+
self.print_warning("Please change the ownership of generated files manually")
166+
self.print_warning("If you are compiling for Windows, you may ignore this warning")
167+
else:
168+
self.print_error("Invalid permissions format: " + self.app.setting_output_permissions)
169+
self.print_error("Please change the ownership of generated files manually")
170+
136171
self.print_success("Compilation successful for " + target)
137172
label.update("Compilation successful for " + target)
138173

@@ -161,3 +196,8 @@ def compose(self) -> ComposeResult:
161196
self.button_widget = Button("Back", id="compile-back-button")
162197
yield self.button_widget
163198
yield Footer()
199+
200+
def on_mount(self) -> None:
201+
# Event handler called when the screen is mounted
202+
print("Compile screen mounted")
203+
self.sub_title = "Compilation"

Diff for: ‎tools/docker/Dockerfile

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# To Do: Check if it is worth to use espressif/idf as base image (image size will be much bigger)
2+
FROM ubuntu:22.04
3+
4+
# switch to root, let the entrypoint drop back to host user
5+
USER root
6+
SHELL ["/bin/bash", "-c"]
7+
8+
ARG DEBIAN_FRONTEND=noninteractive
9+
10+
RUN : \
11+
&& apt-get update \
12+
&& apt-get install -y --no-install-recommends \
13+
bison \
14+
ccache \
15+
cmake \
16+
curl \
17+
flex \
18+
git \
19+
gperf \
20+
jq \
21+
libncurses-dev \
22+
libssl-dev \
23+
libusb-1.0 \
24+
ninja-build \
25+
patch \
26+
python3 \
27+
python3-click \
28+
python3-cryptography \
29+
python3-future \
30+
python3-pip \
31+
python3-pyelftools \
32+
python3-pyparsing \
33+
python3-serial \
34+
python3-setuptools \
35+
python3-venv \
36+
wget \
37+
&& pip3 install --upgrade pip textual-dev \
38+
&& apt-get autoremove -y \
39+
&& rm -rf /var/lib/apt/lists/* \
40+
&& :
41+
42+
# To build the image for a branch or a tag of the lib-builder, pass --build-arg LIBBUILDER_CLONE_BRANCH_OR_TAG=name.
43+
# To build the image with a specific commit ID of lib-builder, pass --build-arg LIBBUILDER_CHECKOUT_REF=commit-id.
44+
# It is possibe to combine both, e.g.:
45+
# LIBBUILDER_CLONE_BRANCH_OR_TAG=release/vX.Y
46+
# LIBBUILDER_CHECKOUT_REF=<some commit on release/vX.Y branch>.
47+
# Use LIBBUILDER_CLONE_SHALLOW=1 to peform shallow clone (i.e. --depth=1 --shallow-submodules)
48+
# Use LIBBUILDER_CLONE_SHALLOW_DEPTH=X to define the depth if LIBBUILDER_CLONE_SHALLOW is used (i.e. --depth=X)
49+
50+
ARG LIBBUILDER_CLONE_URL=https://github.com/espressif/esp32-arduino-lib-builder
51+
ARG LIBBUILDER_CLONE_BRANCH_OR_TAG=master
52+
ARG LIBBUILDER_CHECKOUT_REF=
53+
ARG LIBBUILDER_CLONE_SHALLOW=
54+
ARG LIBBUILDER_CLONE_SHALLOW_DEPTH=1
55+
56+
ENV LIBBUILDER_PATH=/opt/esp/lib-builder
57+
# Ccache is installed, enable it by default
58+
ENV IDF_CCACHE_ENABLE=1
59+
60+
RUN echo LIBBUILDER_CHECKOUT_REF=$LIBBUILDER_CHECKOUT_REF LIBBUILDER_CLONE_BRANCH_OR_TAG=$LIBBUILDER_CLONE_BRANCH_OR_TAG && \
61+
git clone --recursive \
62+
${LIBBUILDER_CLONE_SHALLOW:+--depth=${LIBBUILDER_CLONE_SHALLOW_DEPTH} --shallow-submodules} \
63+
${LIBBUILDER_CLONE_BRANCH_OR_TAG:+-b $LIBBUILDER_CLONE_BRANCH_OR_TAG} \
64+
$LIBBUILDER_CLONE_URL $LIBBUILDER_PATH && \
65+
git config --system --add safe.directory $LIBBUILDER_PATH && \
66+
if [ -n "$LIBBUILDER_CHECKOUT_REF" ]; then \
67+
cd $LIBBUILDER_PATH && \
68+
if [ -n "$LIBBUILDER_CLONE_SHALLOW" ]; then \
69+
git fetch origin --depth=${LIBBUILDER_CLONE_SHALLOW_DEPTH} --recurse-submodules ${LIBBUILDER_CHECKOUT_REF}; \
70+
fi && \
71+
git checkout $LIBBUILDER_CHECKOUT_REF && \
72+
git submodule update --init --recursive; \
73+
fi
74+
75+
COPY entrypoint.sh $LIBBUILDER_PATH/entrypoint.sh
76+
77+
WORKDIR /opt/esp/lib-builder
78+
ENTRYPOINT [ "/opt/esp/lib-builder/entrypoint.sh" ]
79+
CMD [ "python3", "tools/config_editor/app.py" ]

Diff for: ‎tools/docker/README.md

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
<!-- This is a brief version of the Arduino Core for ESP32 documentation (add specific link)
2+
intended to be displayed on the Docker Hub page: https://hub.docker.com/r/espressif/esp32-arduino-lib-builder.
3+
When changing this page, please keep the documentation in sync.
4+
(Keep the differences between Markdown and restructuredText in mind.)
5+
-->
6+
7+
# ESP-IDF Docker Image
8+
9+
This is a Docker image for the [ESP32 Arduino Lib Builder](https://github.com/espressif/esp32-arduino-lib-builder). It is intended for building the static libraries of ESP-IDF components for use in Arduino projects.
10+
11+
This image contains a copy of the esp32-arduino-lib-builder repository and already include or will obtain all the required tools and dependencies to build the Arduino static libraries.
12+
13+
Currently supported architectures are:
14+
- `amd64`
15+
- `arm64`
16+
17+
## Tags
18+
19+
Multiple tags of this image are maintained:
20+
21+
- `latest`: tracks `master` branch of esp32-arduino-lib-builder
22+
- `release-vX.Y`: tracks `release/vX.Y` branch of esp32-arduino-lib-builder
23+
24+
## Basic Usage
25+
26+
```bash
27+
docker run --rm -it -e "TERM=xterm-256color" -v <path to arduino-esp32>:/arduino-esp32 espressif/esp32-arduino-lib-builder:latest
28+
```
29+
30+
The above command explained:
31+
32+
- `docker run`: Runs a command in a new container.
33+
- `--rm`: Optional. Automatically removes the container when it exits. Remove this flag if you plan to use the container multiple times.
34+
- `-i`: Runs the container in interactive mode.
35+
- `-t`: Allocates a pseudo-TTY.
36+
- `-e "TERM=xterm-256color"`: Optional. Sets the terminal type to `xterm-256color` to display colors correctly.
37+
- `-v <path to arduino-esp32>:/arduino-esp32`: Optional. Mounts the Arduino Core for ESP32 repository at `/arduino-esp32` inside the container. Replace `<path to arduino-esp32>` with the path to the repository on the host machine. If not provided, the container will not copy the compiled libraries to the host machine.
38+
- `espressif/esp32-arduino-lib-builder:latest`: The Docker image to use.
39+
40+
After running the above command, you will be inside the container and can build the libraries using the user interface.
41+
42+
By default the docker container will run the user interface script. If you want to run a specific command, you can pass it as an argument to the docker run command. For example, to run a terminal inside the container, you can run:
43+
44+
```bash
45+
docker run -it espressif/esp32-arduino-lib-builder:latest /bin/bash
46+
```
47+
48+
## Documentation
49+
50+
<!-- Pending: Add link to the full documentation. -->
51+
52+
For more information about this image and the detailed usage instructions, please refer to the Arduino Core for ESP32 documentation.

Diff for: ‎tools/docker/entrypoint.sh

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/usr/bin/env bash
2+
set -e
3+
4+
# LIBBUILDER_GIT_SAFE_DIR has the same format as system PATH environment variable.
5+
# All path specified in LIBBUILDER_GIT_SAFE_DIR will be added to user's
6+
# global git config as safe.directory paths. For more information
7+
# see git-config manual page.
8+
if [ -n "${LIBBUILDER_GIT_SAFE_DIR+x}" ]
9+
then
10+
echo "Adding following directories into git's safe.directory"
11+
echo "$LIBBUILDER_GIT_SAFE_DIR" | tr ':' '\n' | while read -r dir
12+
do
13+
git config --global --add safe.directory "$dir"
14+
echo " $dir"
15+
done
16+
fi
17+
18+
# Check if the mount point /arduino-esp32 exists
19+
if [ -d "/arduino-esp32" ] && [[ "$@" == "python3 tools/config_editor/app.py"* ]]; then
20+
# Running UI with mount point detected, adding -c and --output-permissions arguments
21+
echo "Output folder permissions: `stat -c "%u:%g" /arduino-esp32`"
22+
exec "$@" -c /arduino-esp32 --output-permissions `stat -c "%u:%g" /arduino-esp32`
23+
else
24+
# Running UI without mount point detected or running another command
25+
exec "$@"
26+
fi

0 commit comments

Comments
 (0)
Please sign in to comment.