cpu/esp8266: build the SDK bootloader from source

[gschorcht]

Contribution description

This PR is a takeover of PR #17043, which is rebased to the current master and includes some corrections that became necessary after rebasing.

Copied from description of PR #17043:

We had four versions of pre-built bootloaders for the esp8266 with different settings of logging and color logging. These bootloaders were manually built from the SDK and shipped with RIOT-OS source code. However there are more settings that affect the bootloader build that are relevant to the app or final board that uses this bootloader. In particular, flash size and flash speed is important for the bootloader to be able to load an app from a large partition table at the fastest speed supported by the board layout and flash chip.

Another example is the UART baudrate of the logging output from the bootloader. The boot ROM will normally start at a baud rate of 74880 (depending on the crystal installed), so it might make sense to keep the UART output at the same speed so we can debug boot modes and bootloader with the same terminal.

This patch builds the bootloader.bin file from the ESP8266 SDK source code. The code is built as a module (esp8266_bootloader) which at the moment doesn't generate any object code for the application and only produces a bootloader.bin file set to the BOOTLOADER_BIN make variable for the esptool.inc.mk to flash.

The code needs to be compiled and linked with custom rules defined in the module's Makefile since the bootloader.bin is its own separate application.

The BOOTLOADER_BIN variable is changed from a path relative to the $(RIOTCPU)/$(CPU)/bin/ directory to be full path. This makes it easier for applications or board to provide their own bootloader binary if needed.

As a result of building the bootloader from source we fixed the issue of having a large partition table.

Testing procedure

Use following command to flash the application with STDIO UART baudrate of 115200 baud.

BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash

Connect with a terminal programm of your choice (unfortunatly picocom and socat don't support a baudrate close to 74880), for example:

python -m serial.tools.miniterm /dev/ttyUSB0 74880

On reset, the esp8266-esp-12x node shows the ROM bootloader log output

 ets Jan  8 2013,rst cause:2, boot mode:(3,7) 

load 0x40100000, len 6152, room 16 
tail 8
chksum 0x6f
load 0x3ffe8008, len 24, room 0 
tail 8
chksum 0x86
load 0x3ffe8020, len 3408, room 0 
tail 0
chksum 0x79

as well as the second-stage bootloader built by this PR (ESP-IDF v3.1-51-g913a06a9ac3) at 74880 baudrate.

I (42) boot: ESP-IDF v3.1-51-g913a06a9ac3 2nd stage bootloader
I (42) boot: compile time 11:25:03
I (42) boot: SPI Speed      : 26.7MHz
...
I (151) boot: Loaded app from partition at offset 0x10000

The application output is seen as garbage since the esp8266-esp-12x uses 115200 as baurate by default.

To see all output at a baudrate of 74880 baud, you can use the following command:

CFLAGS='-DSTDIO_UART_BAUDRATE=74880' BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash

If the application is built without options, the ROOM bootloader output will be 74880 baud and the second stage bootloader and application output will be 115200 baud.

Issues/PRs references

Fixes issue #16402

[riot-ci]

Murdock results

✔️ PASSED

f92295d pkg/esp8266_sdk: add esp_bootloader to Kconfig

Success	Failures	Total	Runtime
6764	0	6765	12m:45s

Artifacts

• Documentation preview

[benpicco]

looks good to me!

bors merge

[bors]

Build failed:

• Murdock

[benpicco]

bors merge

[bors]

🕐 Waiting for PR status (GitHub check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

[gschorcht]

bors merge

Hm, bors didn't restart the build

Waiting for PR status (GitHub check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

whatever it means.

[kaspar030]

If bors got stuck, a force push might help.

[gschorcht]

If bors got stuck, a force push might help.

Pushing something only triggers a fast build.

[benpicco]

bors merge

[bors]

Build succeeded:

• Murdock
• static-tests
• tools-build

[kaspar030]

Pushing something only triggers a fast build.

Pushing also makes bors re-evaluate the status checks. IIUC bors doesn't realize if a status changes to "failed" and then to later to "success" (like the label checker changing its mind after changing labels).

[bors]

Already running a review

[maribu]

It turns out that sitting behind a 6 Mbps DSL line is no fun with dynamic web pages where you never know if the state shown is already synced...

[gschorcht]

Thanks 😄