Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esp32: improve toolchain management #38540

Merged
merged 3 commits into from
Sep 29, 2021
Merged

esp32: improve toolchain management #38540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a20ef7e
toolchain: esp32: define target toolchain
sylvioalves Sep 6, 2021
de6bfaa
docs: esp32: update environment setup content
sylvioalves Sep 6, 2021
0c4917b
cmake: esp32: use zephyr defined python tool
sylvioalves Sep 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions boards/riscv/esp32c3_devkitm/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ if(CONFIG_BOOTLOADER_ESP_IDF)
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_ASM_COMPILER=${CMAKE_ASM_COMPILER}
-DCMAKE_SYSTEM_NAME=${CMAKE_SYSTEM_NAME}
-DPYTHON=${PYTHON_EXECUTABLE}
BUILD_COMMAND
${CMAKE_COMMAND} --build .
INSTALL_COMMAND "" # This particular build system has no install command
Expand All @@ -35,20 +36,20 @@ if(CONFIG_BOOTLOADER_ESP_IDF)
BINARY_DIR ${espidf_build_dir}
CONFIGURE_COMMAND ""
BUILD_COMMAND
python ${ESP_IDF_PATH}/components/partition_table/gen_esp32part.py -q
--offset 0x1000 --flash-size 4MB ${ESP_IDF_PATH}/components/partition_table/partitions_singleapp.csv ${espidf_build_dir}/partitions_singleapp.bin
${PYTHON_EXECUTABLE} ${ESP_IDF_PATH}/components/partition_table/gen_esp32part.py -q
--offset 0x8000 --flash-size 4MB ${ESP_IDF_PATH}/components/partition_table/partitions_singleapp.csv ${espidf_build_dir}/partitions_singleapp.bin
INSTALL_COMMAND ""
)

if(CONFIG_BUILD_OUTPUT_BIN)
set_property(GLOBAL APPEND PROPERTY extra_post_build_commands
COMMAND python ${ESP_IDF_PATH}/components/esptool_py/esptool/esptool.py
COMMAND ${PYTHON_EXECUTABLE} ${ESP_IDF_PATH}/components/esptool_py/esptool/esptool.py
ARGS --chip esp32c3 elf2image --flash_mode dio --flash_freq 40m
-o ${CMAKE_BINARY_DIR}/zephyr/${CONFIG_KERNEL_BIN_NAME}.bin
${CMAKE_BINARY_DIR}/zephyr/${CONFIG_KERNEL_BIN_NAME}.elf)
endif()

set_property(TARGET bintools PROPERTY disassembly_flag_inline_source)
set_property(TARGET bintools PROPERTY disassembly_flag_inline_source)

add_dependencies(app EspIdfBootloader EspPartitionTable)

Expand Down
8 changes: 3 additions & 5 deletions boards/riscv/esp32c3_devkitm/doc/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,21 +51,19 @@ On Linux and macOS:
.. code-block:: console

export ZEPHYR_TOOLCHAIN_VARIANT="espressif"
export ESPRESSIF_TOOLCHAIN_PATH="${HOME}/.espressif/tools/riscv32-esp-elf/1.24.0.123_64eb9ff-8.4.0/riscv32-esp-elf"
export ESPRESSIF_TOOLCHAIN_PATH="${HOME}/.espressif/tools/zephyr"
Copy link
Collaborator

@tejlmand tejlmand Sep 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious.

It seems the espressif toolchain is now expected to be installed under a different folder per default.
(/.espressif/tools/zephyr/riscv32-esp-elf/... vs. /.espressif/tools/riscv32-esp-elf/...)
Is that part of the espressif installer itself, or is there some missing doc update on toolchain installation ?

Also what about users already having the espressif toolchain installed ?
How will they be aware that they must now move their current
.espressif/tools/riscv32-esp-elf into .espressif/tools/zephyr/riscv32-esp-elf instead.

Also, is 1.24.0.123_64eb9ff-8.4.0 no longer part of the toolchain installation path ?

Copy link
Collaborator Author

@sylvioalves sylvioalves Sep 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejlmand I have explained that already in this PR. Can you check previous comments? The main reason is having the same export PATH for Espressif toolchain to all SoCs, not depending on folder names and versioning as it is right now.

So, as you can see in the document update in this PR, no more 1.24.0.123_64eb9ff-8.4.0 or toolchain name explicit in folder path.

west espressif install is the download agent responsible to download and extract the toolchain in proper place (or user can manually download it as well).

Also what about users already having the espressif toolchain installed ?

When building, it will give them cmake error with a message to re-configure toolchain path.
Is there any recommendation on how to inform users about this change besides changing the ESP32 documentation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the output when running west espressif install. It also informs the user how to export path into the environment.

sylvio@pc:~/projects/espressif/zephyr/zephyr$ west espressif install
=== downloading ESP-IDF tools..
Installing tools: xtensa-esp32-elf, xtensa-esp32s2-elf, riscv32-esp-elf, openocd-esp32
Skipping xtensa-esp32-elf@esp-2020r3-8.4.0 (already installed)
Skipping xtensa-esp32s2-elf@esp-2020r3-8.4.0 (already installed)
Skipping riscv32-esp-elf@1.24.0.123_64eb9ff-8.4.0 (already installed)
Skipping openocd-esp32@v0.10.0-esp32-20200709 (already installed)
=== downloading ESP-IDF tools completed
The toolchain has been downloaded to /home/sylvio/.espressif/tools/zephyr
Export or set the following variables into the environment:
export ESPRESSIF_TOOLCHAIN_PATH="/home/sylvio/.espressif/tools/zephyr"
export ZEPHYR_TOOLCHAIN_VARIANT="espressif"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejlmand I have explained that already in this PR.

If you refer to this comment #38540 (comment), then you only describe why, but my question was about who / how that path was ensured.
The installer or a guide ?

Now that question you answer with the west espressif install sentence.

Also what about users already having the espressif toolchain installed ?

When building, it will give them cmake error with a message to re-configure toolchain path.

No, you give them an error message requesting users to re-install something that is already installed.
I think we should ensure that if users have a setup with the old style then that will still be supported.

That will allow users to keep working and update the toolchain when a new revision is released and they update anyway.

By the way, your west espressif install fails on my machine.

west espressif install
=== downloading ESP-IDF tools..
/usr/bin/env: ‘python’: No such file or directory
FATAL ERROR: command exited with status 127: ./tools/idf_tools.py --tools-json=tools/zephyr_tools.json install

because I only have python3 and no python link.
Why not use the python that was already invoked with west ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then we can remove support for the old style at a later release, similar to what is done for other things we deprecate.


On Windows:

.. code-block:: console

# on CMD:
set ESPRESSIF_TOOLCHAIN_PATH=%USERPROFILE%\.espressif\tools\riscv32-esp-elf\1.24.0.123_64eb9ff-8.4.0\riscv32-esp-elf
set ESPRESSIF_TOOLCHAIN_PATH=%USERPROFILE%\.espressif\tools\zephyr
set ZEPHYR_TOOLCHAIN_VARIANT=espressif
set PATH=%PATH%;%ESPRESSIF_TOOLCHAIN_PATH%\bin

# on PowerShell
$env:ESPRESSIF_TOOLCHAIN_PATH="$env:USERPROFILE\.espressif\tools\riscv32-esp-elf\1.24.0.123_64eb9ff-8.4.0\riscv32-esp-elf"
$env:ESPRESSIF_TOOLCHAIN_PATH="$env:USERPROFILE\.espressif\tools\zephyr"
$env:ZEPHYR_TOOLCHAIN_VARIANT="espressif"
$env:Path += "$env:ESPRESSIF_TOOLCHAIN_PATH\bin"

Finally, retrieve required submodules to build this port. This might take a while for the first time:

Expand Down
7 changes: 4 additions & 3 deletions boards/xtensa/esp32/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ if(CONFIG_BOOTLOADER_ESP_IDF)
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_ASM_COMPILER=${CMAKE_ASM_COMPILER}
-DCMAKE_SYSTEM_NAME=${CMAKE_SYSTEM_NAME}
-DPYTHON=${PYTHON_EXECUTABLE}
BUILD_COMMAND
${CMAKE_COMMAND} --build .
INSTALL_COMMAND "" # This particular build system has no install command
Expand All @@ -34,14 +35,14 @@ if(CONFIG_BOOTLOADER_ESP_IDF)
BINARY_DIR ${espidf_build_dir}
CONFIGURE_COMMAND ""
BUILD_COMMAND
python ${ESP_IDF_PATH}/components/partition_table/gen_esp32part.py -q
--offset 0x1000 --flash-size 4MB ${ESP_IDF_PATH}/components/partition_table/partitions_singleapp.csv ${espidf_build_dir}/partitions_singleapp.bin
${PYTHON_EXECUTABLE} ${ESP_IDF_PATH}/components/partition_table/gen_esp32part.py -q
--offset 0x8000 --flash-size 4MB ${ESP_IDF_PATH}/components/partition_table/partitions_singleapp.csv ${espidf_build_dir}/partitions_singleapp.bin
INSTALL_COMMAND ""
)

if(CONFIG_BUILD_OUTPUT_BIN)
set_property(GLOBAL APPEND PROPERTY extra_post_build_commands
COMMAND python ${ESP_IDF_PATH}/components/esptool_py/esptool/esptool.py
COMMAND ${PYTHON_EXECUTABLE} ${ESP_IDF_PATH}/components/esptool_py/esptool/esptool.py
ARGS --chip esp32 elf2image --flash_mode dio --flash_freq 40m
-o ${CMAKE_BINARY_DIR}/zephyr/${CONFIG_KERNEL_BIN_NAME}.bin
${CMAKE_BINARY_DIR}/zephyr/${CONFIG_KERNEL_BIN_NAME}.elf)
Expand Down
9 changes: 3 additions & 6 deletions boards/xtensa/esp32/doc/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ ESP32
Overview
********


ESP32 is a series of low cost, low power system on a chip microcontrollers
with integrated Wi-Fi & dual-mode Bluetooth. The ESP32 series employs a
Tensilica Xtensa LX6 microprocessor in both dual-core and single-core
Expand Down Expand Up @@ -83,21 +82,19 @@ On Linux and macOS:
.. code-block:: console

export ZEPHYR_TOOLCHAIN_VARIANT="espressif"
export ESPRESSIF_TOOLCHAIN_PATH="${HOME}/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf"
export ESPRESSIF_TOOLCHAIN_PATH="${HOME}/.espressif/tools/zephyr"

On Windows:

.. code-block:: console

# on CMD:
set ESPRESSIF_TOOLCHAIN_PATH=%USERPROFILE%\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf
set ESPRESSIF_TOOLCHAIN_PATH=%USERPROFILE%\.espressif\tools\zephyr
set ZEPHYR_TOOLCHAIN_VARIANT=espressif
set PATH=%PATH%;%ESPRESSIF_TOOLCHAIN_PATH%\bin

# on PowerShell
$env:ESPRESSIF_TOOLCHAIN_PATH="$env:USERPROFILE\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf"
$env:ESPRESSIF_TOOLCHAIN_PATH="$env:USERPROFILE\.espressif\tools\zephyr"
$env:ZEPHYR_TOOLCHAIN_VARIANT="espressif"
$env:Path += "$env:ESPRESSIF_TOOLCHAIN_PATH\bin"

Finally, retrieve required submodules to build this port. This might take a while for the first time:

Expand Down
7 changes: 4 additions & 3 deletions boards/xtensa/esp32s2_saola/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ if(CONFIG_BOOTLOADER_ESP_IDF)
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_ASM_COMPILER=${CMAKE_ASM_COMPILER}
-DCMAKE_SYSTEM_NAME=${CMAKE_SYSTEM_NAME}
-DPYTHON=${PYTHON_EXECUTABLE}
BUILD_COMMAND
${CMAKE_COMMAND} --build .
INSTALL_COMMAND "" # This particular build system has no install command
Expand All @@ -34,14 +35,14 @@ if(CONFIG_BOOTLOADER_ESP_IDF)
BINARY_DIR ${espidf_build_dir}
CONFIGURE_COMMAND ""
BUILD_COMMAND
python ${ESP_IDF_PATH}/components/partition_table/gen_esp32part.py -q
--offset 0x1000 --flash-size 4MB ${ESP_IDF_PATH}/components/partition_table/partitions_singleapp.csv ${espidf_build_dir}/partitions_singleapp.bin
${PYTHON_EXECUTABLE} ${ESP_IDF_PATH}/components/partition_table/gen_esp32part.py -q
--offset 0x8000 --flash-size 4MB ${ESP_IDF_PATH}/components/partition_table/partitions_singleapp.csv ${espidf_build_dir}/partitions_singleapp.bin
INSTALL_COMMAND ""
)

if(CONFIG_BUILD_OUTPUT_BIN)
set_property(GLOBAL APPEND PROPERTY extra_post_build_commands
COMMAND python ${ESP_IDF_PATH}/components/esptool_py/esptool/esptool.py
COMMAND ${PYTHON_EXECUTABLE} ${ESP_IDF_PATH}/components/esptool_py/esptool/esptool.py
ARGS --chip esp32s2 elf2image --flash_mode dio --flash_freq 40m
-o ${CMAKE_BINARY_DIR}/zephyr/${CONFIG_KERNEL_BIN_NAME}.bin
${CMAKE_BINARY_DIR}/zephyr/${CONFIG_KERNEL_BIN_NAME}.elf)
Expand Down
8 changes: 3 additions & 5 deletions boards/xtensa/esp32s2_saola/doc/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,19 @@ On Linux and macOS:
.. code-block:: console

export ZEPHYR_TOOLCHAIN_VARIANT="espressif"
export ESPRESSIF_TOOLCHAIN_PATH="${HOME}/.espressif/tools/xtensa-esp32s2-elf/esp-2020r3-8.4.0/xtensa-esp32s2-elf"
export ESPRESSIF_TOOLCHAIN_PATH="${HOME}/.espressif/tools/zephyr"

On Windows:

.. code-block:: console

# on CMD:
set ESPRESSIF_TOOLCHAIN_PATH=%USERPROFILE%\.espressif\tools\xtensa-esp32s2-elf\esp-2020r3-8.4.0\xtensa-esp32s2-elf
set ESPRESSIF_TOOLCHAIN_PATH=%USERPROFILE%\.espressif\tools\zephyr
set ZEPHYR_TOOLCHAIN_VARIANT=espressif
set PATH=%PATH%;%ESPRESSIF_TOOLCHAIN_PATH%\bin

# on PowerShell
$env:ESPRESSIF_TOOLCHAIN_PATH="$env:USERPROFILE\.espressif\tools\xtensa-esp32s2-elf\esp-2020r3-8.4.0\xtensa-esp32s2-elf"
$env:ESPRESSIF_TOOLCHAIN_PATH="$env:USERPROFILE\.espressif\tools\zephyr"
$env:ZEPHYR_TOOLCHAIN_VARIANT="espressif"
$env:Path += "$env:ESPRESSIF_TOOLCHAIN_PATH\bin"

Finally, retrieve required submodules to build this port. This might take a while for the first time:

Expand Down
43 changes: 34 additions & 9 deletions cmake/toolchain/espressif/generic.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,23 +10,48 @@ set(COMPILER gcc)
set(LINKER ld)
set(BINTOOLS gnu)

# find some toolchain
file(GLOB toolchain_paths
LIST_DIRECTORIES true
${TOOLCHAIN_HOME}
${TOOLCHAIN_HOME}/*-esp32*/*-elf
${TOOLCHAIN_HOME}/*-esp*/*-elf
)

# Old toolchain installation path has been deprecated in 2.7.
# This code and related code depending on ESPRESSIF_DEPRECATED_PATH can be removed after two releases.
if(NOT toolchain_paths)
# find some toolchain
file(GLOB toolchain_paths
LIST_DIRECTORIES true
${TOOLCHAIN_HOME}
)

set(ESPRESSIF_DEPRECATED_PATH TRUE)
endif()

if(toolchain_paths)
list(GET toolchain_paths 0 soc_toolchain_path)
list(GET toolchain_paths 0 some_toolchain_path)

get_filename_component(one_toolchain_root "${some_toolchain_path}" DIRECTORY)
get_filename_component(one_toolchain "${some_toolchain_path}" NAME)

get_filename_component(soc_toolchain "${soc_toolchain_path}" NAME)
set(CROSS_COMPILE_TARGET ${one_toolchain})
set(SYSROOT_TARGET ${one_toolchain})

if(ESPRESSIF_DEPRECATED_PATH)
set(CROSS_COMPILE ${ESPRESSIF_TOOLCHAIN_PATH}/bin/${CROSS_COMPILE_TARGET}-)
set(SYSROOT_DIR ${ESPRESSIF_TOOLCHAIN_PATH}/${SYSROOT_TARGET})
else()
set(CROSS_COMPILE ${one_toolchain_root}/bin/${CROSS_COMPILE_TARGET}-)
set(SYSROOT_DIR ${one_toolchain_root}/${SYSROOT_TARGET})
endif()

set(CROSS_COMPILE_TARGET ${soc_toolchain})
set(SYSROOT_TARGET ${soc_toolchain})
endif()

set(CROSS_COMPILE ${TOOLCHAIN_HOME}/bin/${CROSS_COMPILE_TARGET}-)
set(SYSROOT_DIR ${TOOLCHAIN_HOME}/${SYSROOT_TARGET})
if(NOT CROSS_COMPILE_TARGET)
message(FATAL_ERROR
"Unable to find toolchain in ${TOOLCHAIN_HOME} "
"Run `west espressif install` to download it. Then, export to path accordingly.")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejlmand This is current error if toolchain not found by either missing its installation or having previous installation version.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I see that.
But this just ask user to re-install, but a user already having the toolchain installed should not need to re-install it.

endif()

set(TOOLCHAIN_HAS_NEWLIB ON CACHE BOOL "True if toolchain supports newlib")

message(STATUS "Found toolchain: espressif (${ESPRESSIF_TOOLCHAIN_PATH})")
28 changes: 27 additions & 1 deletion cmake/toolchain/espressif/target.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@
# SPDX-License-Identifier: Apache-2.0

# This file intentionally left blank.
set_ifndef(ESPRESSIF_TOOLCHAIN_PATH "$ENV{ESPRESSIF_TOOLCHAIN_PATH}")
set( ESPRESSIF_TOOLCHAIN_PATH ${ESPRESSIF_TOOLCHAIN_PATH} CACHE PATH "")
assert( ESPRESSIF_TOOLCHAIN_PATH "ESPRESSIF_TOOLCHAIN_PATH is not set")

set(COMPILER gcc)
set(LINKER ld)
set(BINTOOLS gnu)

set(CROSS_COMPILE_TARGET_xtensa_esp32 xtensa-esp32-elf)
set(CROSS_COMPILE_TARGET_xtensa_esp32s2 xtensa-esp32s2-elf)
set(CROSS_COMPILE_TARGET_riscv_esp32c3 riscv32-esp-elf)

set(CROSS_COMPILE_TARGET ${CROSS_COMPILE_TARGET_${ARCH}_${CONFIG_SOC}})
set(SYSROOT_TARGET ${CROSS_COMPILE_TARGET})

if(ESPRESSIF_DEPRECATED_PATH)
set(TOOLCHAIN_HOME ${ESPRESSIF_TOOLCHAIN_PATH})
else()
set(TOOLCHAIN_HOME ${ESPRESSIF_TOOLCHAIN_PATH}/${CROSS_COMPILE_TARGET})
endif()

set(CROSS_COMPILE ${TOOLCHAIN_HOME}/bin/${CROSS_COMPILE_TARGET}-)
set(SYSROOT_DIR ${TOOLCHAIN_HOME}/${SYSROOT_TARGET})

set(TOOLCHAIN_HAS_NEWLIB ON CACHE BOOL "True if toolchain supports newlib")

message(STATUS "Found toolchain: espressif (${ESPRESSIF_TOOLCHAIN_PATH})")
2 changes: 1 addition & 1 deletion west.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ manifest:
groups:
- hal
- name: hal_espressif
revision: 0d30edf01a41b2c12e6715ff909f81421e96127e
revision: 6cb37487b6ee42f96b4ba0d4719bb5759b098822
path: modules/hal/espressif
west-commands: west/west-commands.yml
groups:
Expand Down