Dealing with 2nd stage bootloader iram_loader_seg

[bjoernQ]

The 2nd stage bootloader is loaded to RAM and we cannot use that area for data or code.
For some chips we account for that by configuring memory accordingly - on some we haven't yet (#1071)

When lowering the amount of available memory, we are wasting some RAM since we could use it for stack (or .bss)

To use it as stack we probably could just configure stack-top in the linker scripts (but be aware of the flip-link feature on H2/C6)

[MabezDev]

The 2nd stage bootloader is loaded to RAM and we cannot use that area for data or code.

Why not? I understand needing to keep ROM code/ ROM bootloader memory safe, but once the 2nd stage bootloader jumps to the main app the memory is free to use, right? I might be missing something though.

[bjoernQ]

The problem is, the bootloader cannot load data there (or code) while it's still running (before jumping into the application).

As soon as our application runs, we can use the memory which was previously occupied by the 2nd stage bootloader. We can use that region for stack, heap or .bss

If we lower the available RAM so we never place data/code there then we get "nice" linker errors but the way we currently place the stack leaves that memory unused/wasted. (same for esp-alloc where we show people to use .bss for the heap)

If we pretend that we can use all the memory it happens that users end up placing data/code there and get an error when trying to boot

In the past we went with option B but for some chips the linker scripts got adjusted to option A. I guess option A where the linker complains about something not fitting into a section is easier to understand than the bootloader (the message looks like esp_image: Segment 2 0x403aeb10-0x403aec10 invalid: overlaps loader IRAM)

[MabezDev]

Ah so sorry for the misunderstanding! That makes perfect sense, thanks for explaining.

[wuwbobo2021]

I'm redirected to this issue from #2001. According to bjoernQ's words, some RAM in ESP32 is wasted. But it's probably not only about the 2nd stage bootloader, which has a maximum size 28,672B (0x8000 - 0x1000 = 0x7000).

I found a clearer explanation about DRAM here: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/esp-idf-en-v5.3-esp32.pdf, 4.23.1 (Page 2726). It's so strange that it differs from the web version of the same documentation: https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32s2/api-guides/memory-types.html, which doesn't describe for these things:

"The available size of the internal DRAM region is reduced by 64 KB (by shifting start address to 0x3FFC0000) if Bluetooth stack is used. Length of this region is also reduced by 16 KB or 32 KB if trace memory is used. Due to some memory fragmentation issues caused by ROM, it is also not possible to use all available DRAM for static allocations - however the remaining DRAM is still available as heap at runtime."

"Note: There is 520 KB of available SRAM (320 KB of DRAM and 200 KB of IRAM) on the ESP32. However, due to a technical limitation, the maximum statically allocated DRAM usage is 160 KB. The remaining 160 KB (for a total of 320 KB of DRAM) can only be allocated at runtime as heap."

Available internal SRAM size of ESP32-S2 is 320KB, not 520KB (that's for ESP32). On the other hand, even the default usage of esp-alloc allocate its managed memory region statically. That's probably the reason why the memory left for user app programmer is so small: a little more than 16KB, while it should be appr. 130KB in ESP-IDF C project.

Looking into the memory mapping table in ESP32-S2 technical reference (Table 23), I understand that the ROM itself isn't mapped into the middle of SRAM (either IRAM or DRAM), but the middle of SRAM is probably used by ROM functions. Unfortunately I don't know where to find detailed info about its exact address and size, and I don't know if it's a black box. I'd like to know whether or not esp-hal (not based on ESP-IDF) is officially supported by Espressif.

[wuwbobo2021]

Sorry, I've got it wrong, and must make the correction before measuring "reserved memory regions" in the memory layout: IDF manuals for ESP32 and ESP32-S2 are not the same. https://docs.espressif.com/projects/esp-idf/en/stable/esp32/esp-idf-en-v5.3-esp32.pdf corresponds to https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32/api-guides/memory-types.html, not for ESP32-S2.

Chip-specific memory layout information for the ESP-IDF heap allocator can be found at components/heap/port in the IDF folder, which looks useful.

In memory_layout.c for ESP32 (not S2) it's described that "ROM Stack" is located in "0x3ffe0000 region", "after the scheduler has started, the ROM stack is not used anymore by anything". "The 0x3ffe0000 region also contains static RAM for various ROM functions." For ESP32-S2, such notes were not written in memory_layout.c, but there are also reserved memory regions. Both seems to be reserving some memory regions from heap allocator by some paramters at runtime.

In esp-hal, dram_seg defined for ESP32-S2 (esp-hal/ld/esp32s2/memory.x) is defined as if the size of entire DRAM is 188KB, not 320KB: [0x3FFB0000, 0x3FFDF000). region [0x3FFDF000, 0x40000000) is not included in the segment.

When talking about possible equivalent of ESP-IDF function esp_get_minimum_free_heap_size(), Anthony Grondin told me: "esp-wifi currently doesn't support a global allocator. It uses its own allocated heap, on the stack. The equivalent of esp_get_minimum_free_heap_size() is https://github.com/esp-rs/esp-hal/blob/f829c8f2acc1069fd503d827073d31afa3b3e87b/esp-wifi/src/wifi/os\_adapter.rs#L884-L899 but currently it's not exposed publicly, so you will need to patch the library to use it." It's probably about esp-wifi's own allocated heap...

I guess it's good to get clear of these memory regions and make esp-alloc handle multiple inconsistant regions just like ESP-IDF, that might solve this problem (along with some other issues).