Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heap: Improve debug caller address reporting #8720

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4ec3c81
Lite refactoring and expanded comments
mhightower83 Nov 11, 2022
6088147
Improve caller address reporting to indicate address in
mhightower83 Nov 18, 2022
a716193
missed items
mhightower83 Nov 21, 2022
1102256
improve comments
mhightower83 Nov 22, 2022
6dfd46a
Merge branch 'master' into pr-heap-refactor3
mhightower83 Dec 5, 2022
8b3bb1f
Merge branch 'master' into pr-heap-refactor3
mhightower83 Dec 17, 2022
ea9befa
Finish merge cleanuo
mhightower83 Dec 17, 2022
3dd08be
requested changes
mhightower83 Dec 17, 2022
45f1c3b
Added missing DRAM fallback to pvPortCallocIram, pvPortZallocIram, and
mhightower83 Dec 18, 2022
57eb2b3
Add nothrow to aliases
mhightower83 Dec 18, 2022
c2590a7
Merge branch 'master' into pr-heap-refactor3
mhightower83 Dec 19, 2022
0619034
Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…
mhightower83 Dec 19, 2022
19da675
Merge branch 'master' into pr-heap-refactor3
mhightower83 Dec 19, 2022
af7b962
Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…
mhightower83 Dec 19, 2022
9423c8e
Merge branch 'master' into pr-heap-refactor3
mhightower83 Dec 22, 2022
471f838
Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…
mhightower83 Dec 22, 2022
d829540
Merge branch 'master' into pr-heap-refactor3
mhightower83 Jan 20, 2023
4c96365
Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…
mhightower83 Jan 20, 2023
840e55a
Merge branch 'master' into pr-heap-refactor3
mhightower83 Apr 18, 2023
95c7e2c
Merge branch 'master' into pr-heap-refactor3
mhightower83 Apr 27, 2023
0ce295e
Merge branch 'master' into pr-heap-refactor3
mhightower83 Apr 28, 2023
4717dcf
Use the same format for printing caller, file, line details
mhightower83 Apr 29, 2023
b083043
refactored print_loc()
mhightower83 Apr 29, 2023
ecc1b14
Merge branch 'master' into pr-heap-refactor3
mhightower83 May 2, 2023
a0044d6
Merge branch 'master' into pr-heap-refactor3
mhightower83 Jun 16, 2023
01b238a
Merge branch 'master' into pr-heap-refactor3
mhightower83 Jul 21, 2023
b5aa1de
removed unused variable from umm_local.c
mhightower83 Jul 21, 2023
1c99daf
Merge branch 'master' into pr-heap-refactor3
mhightower83 Feb 6, 2024
2bbadca
Merge branch 'master' into pr-heap-refactor3
mhightower83 Aug 30, 2024
49b48fb
Requested changes
mhightower83 Sep 3, 2024
3786ce6
Added missing "new" operators
mhightower83 Sep 5, 2024
b9a0e55
Added missing delete array operators
mhightower83 Sep 6, 2024
a5f6d7d
malloc align build cleanup - fully builds
mhightower83 Sep 8, 2024
54b034b
Correct possition of UMM_ENABLE_MEMALIGN handling in umm_malloc/umm_m…
mhightower83 Sep 12, 2024
e948329
remove static_assert checks on alignment
mhightower83 Sep 15, 2024
5653249
Updated changes to match upstream style using uncrustify.
mhightower83 Sep 17, 2024
d303d01
update comments
mhightower83 Oct 15, 2024
01a89fd
Added test Sketch for "new" and "delete" operators
mhightower83 Oct 21, 2024
0574c1a
Oops, remove external debug libary from example.
mhightower83 Oct 22, 2024
31217f6
Improved build coverage of DEV_DEBUG_ABI_CPP option and operator dele…
mhightower83 Oct 24, 2024
ecc7eed
Changed printf format from %S => %s
mhightower83 Nov 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 39 additions & 29 deletions cores/esp8266/abi.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -24,57 +24,67 @@
using __cxxabiv1::__guard;

// Debugging helper, last allocation which returned NULL
extern void *umm_last_fail_alloc_addr;
extern int umm_last_fail_alloc_size;
extern "C" void *_heap_abi_malloc(size_t size, bool unhandled, const void* const caller);

extern "C" void __cxa_pure_virtual(void) __attribute__ ((__noreturn__));
extern "C" void __cxa_deleted_virtual(void) __attribute__ ((__noreturn__));

#if defined(__cpp_exceptions) && (defined(DEBUG_ESP_OOM) || defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_WITHINISR))
Copy link
Collaborator

@mcspr mcspr Sep 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't 'new_handler' be independent of debugging?
since the idea is to allow plain new to sometimes avoid throwing (regardless of whether it builds w/ -fexceptions or without). exception-less builds obviously just call abort for would-be exception
ref new_op.cc and new_opnt.cc, nothrow variant just wraps the other in try catch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't 'new_handler' be independent of debugging?

It has been a while since I did this, I think the increased build size for non-debug builds was a consideration.
It would add 136 bytes of IROM to the non-debug build so that it always builds with our revised new_handler.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will have to recheck too, then. Debug=enabled without exceptions are not handled though?

Plus there are more missing overloads from -std=c++17
https://en.cppreference.com/w/cpp/memory/new/operator_new

Copy link
Contributor Author

@mhightower83 mhightower83 Sep 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plus there are more missing overloads from -std=c++17

The current master does not support "new" align operations. Compiler reports:
/workdir/repo/gcc-gnu/libstdc++-v3/libsupc++/new_opa.cc:86: undefined reference tomemalign'`

No hits on a quick issue check for memalign.

The library umm_malloc needs an enhancement to support an allocate-aligned option. Also, things will get complicated when incorporating the poison check feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've worked through adding an alignment option to the umm_malloc library. In the process, I have addressed an unreported alignment issue. umm_malloc allocated data addresses always landed on an 8-byte aligned - 4-byte.
Now I need to finish working through all the build options in abi.cpp and heap.cpp.

Copy link
Collaborator

@mcspr mcspr Sep 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(edit: comment thread from above)

Can this be a config error? GCC / newlib / hal / etc.

Reading cppreference and some of the internal GCC stuff from '-faligned-new=X', 'Fundamental alignment' and max_align_t seem to come from 'long double' and 'long long' which are afaik 4-byte ones.

Currently, it does indeed show 8-byte

constexpr auto a = alignof(long long); // 8
constexpr auto b = alignof(long double); // 8

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(edit: comment thread from above)

Can this be a config error? GCC / newlib / hal / etc.

Reading cppreference and some of the internal GCC stuff from '-faligned-new=X', 'Fundamental alignment' and max_align_t seem to come from 'long double' and 'long long' which are afaik 4-byte ones.

Currently, it does indeed show 8-byte

constexpr auto a = alignof(long long); // 8
constexpr auto b = alignof(long double); // 8

...which is sort-of expected, as run-of-the-mill xtensa isa pdf does specify alignment req for long double and long long as 8. compiler here works according to spec, stack and heap alignment would stay consistent vs. current approach

no mention / explicit alignment things mentioned in the machine config, besides the BIGGEST_ALIGNMENT of 128 (bits)
https://github.com/gcc-mirror/gcc/tree/master/gcc/config/xtensa /ALIGN/

esp32 variants adhere to a similar standard. however, esp8266 rtos does use 4-byte heap alignment
(which I would assume works just fine for the most part)

/*
When built with C++ Exceptions: "enabled", track caller address of Last OOM.
* For debug build, force enable Last OOM tracking.
* With the option "DEBUG_ESP_OOM," always do Last OOM tracking.
* Otherwise, disable Last OOM tracking. The build relies on the weak link to
the default C++ exception handler.
*/

// Debug replacement adaptation from ".../new_op.cc".
using std::new_handler;
using std::bad_alloc;

void * operator new (std::size_t size)
{
void *p;
mcspr marked this conversation as resolved.
Show resolved Hide resolved

/* malloc (0) is unpredictable; avoid it. */
if (__builtin_expect(size == 0, false)) {
size = 1;
}

#if !defined(__cpp_exceptions)
while (0 == (p = _heap_abi_malloc(size, false, __builtin_return_address(0)))) {
new_handler handler = std::get_new_handler();
if (!handler) {
throw(bad_alloc());
}
handler();
}

return p;
}
#elif !defined(__cpp_exceptions)
// When doing builds with C++ Exceptions "disabled", always save details of
// the last OOM event.

// overwrite weak operators new/new[] definitions

void* operator new(size_t size)
{
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
__unhandled_exception(PSTR("OOM"));
}
return ret;
return _heap_abi_malloc(size, true, __builtin_return_address(0));
}

void* operator new[](size_t size)
{
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
__unhandled_exception(PSTR("OOM"));
}
return ret;
return _heap_abi_malloc(size, true, __builtin_return_address(0));
}

void* operator new (size_t size, const std::nothrow_t&)
{
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
}
return ret;
return _heap_abi_malloc(size, false, __builtin_return_address(0));
}

void* operator new[] (size_t size, const std::nothrow_t&)
{
void *ret = malloc(size);
if (0 != size && 0 == ret) {
umm_last_fail_alloc_addr = __builtin_return_address(0);
umm_last_fail_alloc_size = size;
}
return ret;
return _heap_abi_malloc(size, false, __builtin_return_address(0));
}

#endif // !defined(__cpp_exceptions)
Expand Down
116 changes: 61 additions & 55 deletions cores/esp8266/core_esp8266_postmortem.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -36,41 +36,47 @@

extern "C" {

// These will be pointers to PROGMEM const strings
static const char* s_panic_file = 0;
static int s_panic_line = 0;
static const char* s_panic_func = 0;
static const char* s_panic_what = 0;

// Our wiring for abort() and C++ exceptions
static bool s_abort_called = false;
static const char* s_unhandled_exception = NULL;

// Common way to notify about where the stack smashing happened
// (but, **only** if caller uses our handler function)
static uint32_t s_stacksmash_addr = 0;

void abort() __attribute__((noreturn));
static void uart_write_char_d(char c);
static void uart0_write_char_d(char c);
static void uart1_write_char_d(char c);
static void print_stack(uint32_t start, uint32_t end);

// using numbers different from "REASON_" in user_interface.h (=0..6)
enum rst_reason_sw
{
REASON_USER_STACK_SMASH = 253,
REASON_USER_SWEXCEPTION_RST = 254
};
static int s_user_reset_reason = REASON_DEFAULT_RST;

static struct PostmortemInfo {
int user_reset_reason; // REASON_DEFAULT_RST;

// These will be pointers to PROGMEM const strings
const char* panic_file; // 0;
int panic_line; // 0;
const char* panic_func; // 0;
const char* panic_what; // 0;

// Our wiring for abort() and C++ exceptions
bool abort_called; // false;
const char* unhandled_exception; // NULL;

// Common way to notify about where the stack smashing happened
// (but, **only** if caller uses our handler function)
uint32_t stacksmash_addr; // 0;
} s_pm = {REASON_DEFAULT_RST, 0, 0, 0, 0, false, NULL, 0};
mhightower83 marked this conversation as resolved.
Show resolved Hide resolved

// From UMM, the last caller of a malloc/realloc/calloc which failed:
extern void *umm_last_fail_alloc_addr;
extern int umm_last_fail_alloc_size;
extern struct umm_last_fail_alloc {
const void *addr;
size_t size;
#if defined(DEBUG_ESP_OOM)
extern const char *umm_last_fail_alloc_file;
extern int umm_last_fail_alloc_line;
const char *file;
int line;
#endif
} _umm_last_fail_alloc;


void abort() __attribute__((noreturn));
static void uart_write_char_d(char c);
static void uart0_write_char_d(char c);
static void uart1_write_char_d(char c);
static void print_stack(uint32_t start, uint32_t end);

static void raise_exception() __attribute__((noreturn));

Expand Down Expand Up @@ -116,7 +122,7 @@ void __wrap_system_restart_local() {

struct rst_info rst_info;
memset(&rst_info, 0, sizeof(rst_info));
if (s_user_reset_reason == REASON_DEFAULT_RST)
if (s_pm.user_reset_reason == REASON_DEFAULT_RST)
{
system_rtc_mem_read(0, &rst_info, sizeof(rst_info));
if (rst_info.reason != REASON_SOFT_WDT_RST &&
Expand All @@ -127,23 +133,23 @@ void __wrap_system_restart_local() {
}
}
else
rst_info.reason = s_user_reset_reason;
rst_info.reason = s_pm.user_reset_reason;

ets_install_putc1(&uart_write_char_d);

cut_here();

if (s_panic_line) {
ets_printf_P(PSTR("\nPanic %S:%d %S"), s_panic_file, s_panic_line, s_panic_func);
if (s_panic_what) {
ets_printf_P(PSTR(": Assertion '%S' failed."), s_panic_what);
if (s_pm.panic_line) {
ets_printf_P(PSTR("\nPanic %S:%d %S"), s_pm.panic_file, s_pm.panic_line, s_pm.panic_func);
if (s_pm.panic_what) {
ets_printf_P(PSTR(": Assertion '%S' failed."), s_pm.panic_what);
}
ets_putc('\n');
}
else if (s_unhandled_exception) {
ets_printf_P(PSTR("\nUnhandled C++ exception: %S\n"), s_unhandled_exception);
else if (s_pm.unhandled_exception) {
ets_printf_P(PSTR("\nUnhandled C++ exception: %S\n"), s_pm.unhandled_exception);
}
else if (s_abort_called) {
else if (s_pm.abort_called) {
ets_printf_P(PSTR("\nAbort called\n"));
}
else if (rst_info.reason == REASON_EXCEPTION_RST) {
Expand All @@ -159,8 +165,8 @@ void __wrap_system_restart_local() {
else if (rst_info.reason == REASON_USER_STACK_SMASH) {
ets_printf_P(PSTR("\nStack smashing detected.\n"));
ets_printf_P(PSTR("\nException (%d):\nepc1=0x%08x epc2=0x%08x epc3=0x%08x excvaddr=0x%08x depc=0x%08x\n"),
5 /* Alloca exception, closest thing to stack fault*/, s_stacksmash_addr, 0, 0, 0, 0);
}
5 /* Alloca exception, closest thing to stack fault*/, s_pm.stacksmash_addr, 0, 0, 0, 0);
}
else {
ets_printf_P(PSTR("\nGeneric Reset\n"));
}
Expand Down Expand Up @@ -211,22 +217,22 @@ void __wrap_system_restart_local() {
ets_printf_P(PSTR("<<<stack<<<\n"));

// Use cap-X formatting to ensure the standard EspExceptionDecoder doesn't match the address
if (umm_last_fail_alloc_addr) {
if (_umm_last_fail_alloc.addr) {
#if defined(DEBUG_ESP_OOM)
ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)@%S:%d\n"),
(uint32_t)umm_last_fail_alloc_addr, umm_last_fail_alloc_size,
umm_last_fail_alloc_file, umm_last_fail_alloc_line);
(uint32_t)_umm_last_fail_alloc.addr, _umm_last_fail_alloc.size,
_umm_last_fail_alloc.file, _umm_last_fail_alloc.line);
#else
ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)\n"), (uint32_t)umm_last_fail_alloc_addr, umm_last_fail_alloc_size);
ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)\n"), (uint32_t)_umm_last_fail_alloc.addr, _umm_last_fail_alloc.size);
#endif
}

cut_here();

if (s_unhandled_exception && umm_last_fail_alloc_addr) {
if (s_pm.unhandled_exception && _umm_last_fail_alloc.addr) {
// now outside from the "cut-here" zone, print correctly the `new` caller address,
// idf-monitor.py will be able to decode this one and show exact location in sources
ets_printf_P(PSTR("\nlast failed alloc caller: 0x%08x\n"), (uint32_t)umm_last_fail_alloc_addr);
ets_printf_P(PSTR("\nlast failed alloc caller: 0x%08x\n"), (uint32_t)_umm_last_fail_alloc.addr);
}

custom_crash_callback( &rst_info, sp_dump + offset, stack_end );
Expand Down Expand Up @@ -275,45 +281,45 @@ static void raise_exception() {
if (gdb_present())
__asm__ __volatile__ ("syscall"); // triggers GDB when enabled

s_user_reset_reason = REASON_USER_SWEXCEPTION_RST;
s_pm.user_reset_reason = REASON_USER_SWEXCEPTION_RST;
ets_printf_P(PSTR("\nUser exception (panic/abort/assert)"));
__wrap_system_restart_local();

while (1); // never reached, needed to satisfy "noreturn" attribute
}

void abort() {
s_abort_called = true;
s_pm.abort_called = true;
raise_exception();
}

void __unhandled_exception(const char *str) {
s_unhandled_exception = str;
s_pm.unhandled_exception = str;
raise_exception();
}

void __assert_func(const char *file, int line, const char *func, const char *what) {
s_panic_file = file;
s_panic_line = line;
s_panic_func = func;
s_panic_what = what;
s_pm.panic_file = file;
s_pm.panic_line = line;
s_pm.panic_func = func;
s_pm.panic_what = what;
gdb_do_break(); /* if GDB is not present, this is a no-op */
raise_exception();
}

void __panic_func(const char* file, int line, const char* func) {
s_panic_file = file;
s_panic_line = line;
s_panic_func = func;
s_panic_what = 0;
s_pm.panic_file = file;
s_pm.panic_line = line;
s_pm.panic_func = func;
s_pm.panic_what = 0;
gdb_do_break(); /* if GDB is not present, this is a no-op */
raise_exception();
}

uintptr_t __stack_chk_guard = 0x08675309 ^ RANDOM_REG32;
void __stack_chk_fail(void) {
s_user_reset_reason = REASON_USER_STACK_SMASH;
s_stacksmash_addr = (uint32_t)__builtin_return_address(0);
s_pm.user_reset_reason = REASON_USER_STACK_SMASH;
s_pm.stacksmash_addr = (uint32_t)__builtin_return_address(0);

if (gdb_present())
__asm__ __volatile__ ("syscall"); // triggers GDB when enabled
Expand Down
Loading