popup “Exception: wasm auxiliary stack overflow”

[tkernelcn]

linux iwasm build

wasm-micro-runtime-main/product-mini/platforms/linux/build$ cmake -DWAMR_BUILD_MULTI_MODULE=1 -DWAMR_BUILD_LIB_PTHREAD=1 -DWAMR_BUILD_LIB_PTHREAD_SEMAPHORE=1 ..

running

./iwasm --map-dir="/::./" --addr-pool=127.0.0.1/15,0.0.0.0/15 --allow-resolve=* --max-threads=20 --heap-size=65535 ./test-apps.wasm

test-apps.zip

result:

....
[==========] Running (test_cases) 1 test(s)
[  RUN     ] 1: test_lookup_host
Host: www.wshifen.com
IPv4 address: 103.235.46.40 (TCP)
[  OK      ] 1: test_lookup_host
[==========] finish (test_cases), total: 1, passed: 1, failed: 0, errors: 0, skipped: 0
[==========] Running (test_cases) 1 test(s)
[  RUN     ] 1: test_select_stdin
Exception: wasm auxiliary stack overflow

failed case source: (a very simple select delay test)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>
#include <stdint.h>
#include <unistd.h>
#include <assert.h>
#include <pthread.h>
#include "test.h"
#include <errno.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <sys/socket.h>
#ifdef __wasi__
#include <wasi_socket_ext.h>
#endif
#include <sys/time.h>

static int test_select_stdin(void **state)
{
    int ret;
    struct timeval tv;
    fd_set readfds;
    int max_fd = STDIN_FILENO;

    FD_ZERO(&readfds);
    FD_SET(STDIN_FILENO, &readfds);

    tv.tv_sec = 1;
    tv.tv_usec = 0;

    ret = select(max_fd + 1, &readfds, NULL, NULL, &tv);
    if(ret == -1) {
        perror("select");
        return TEST_FAILED;
    } else if (ret == 0) {
        printf("delay 1s\n");
    } else {
        printf("stdin read ready\n");
    }

    return TEST_PASSED;
}

int test_select(void)
{
    const struct UnitTest test_cases[] = {
        unit_test(test_select_stdin),
    };
    return run_group_tests(test_cases, NULL, NULL);
}

[tkernelcn]

backtrace

#0  0x0000555555561d10 in wasm_set_exception ()
#1  0x000055555556d6c4 in wasm_interp_call_func_bytecode ()
#2  0x000055555556e9f2 in wasm_interp_call_wasm ()
#3  0x0000555555564f6f in call_wasm_with_hw_bound_check ()
#4  0x000055555556596c in wasm_call_function ()
#5  0x0000555555561c81 in wasm_runtime_call_wasm ()
#6  0x000055555555f861 in wasm_application_execute_main ()
#7  0x000055555555d5f4 in main ()

[tkernelcn]

one comments:
even I add -DWAMR_DISABLE_HW_BOUND_CHECK=1 to build iwasm, also have this error message.

[yamt]

in case of WAMR_BUILD_LIB_PTHREAD, the aux stack is divided by the max number of threads.
i guess you should:

• enlarge the stack size (-z stack-size=xxx when building your test-apps.wasm)
• or use a smaller number for --max-threads=xxx
• or switch to another pthread implementation (see https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/pthread_impls.md)

[tkernelcn]

if set -z stack-size=65535 also have the problem
if remove --max-threads=xxx or smaller to --max-threads=6, the test pass

my question is, where to set the aux stack pool size, if my test is more threads enabled?
Thanks.

[tkernelcn]

one more question: (this is not linux host, is a embeded freertos host.)
if WAMR_BUILD_LIB_WASI_THREADS=1 start up errors:

[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed                                                                                                                        
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread

failed source

static bool
allocate_aux_stack(WASMExecEnv *exec_env, uint32 *start, uint32 *size)
{
    WASMCluster *cluster = wasm_exec_env_get_cluster(exec_env);
#if WASM_ENABLE_HEAP_AUX_STACK_ALLOCATION != 0
    WASMModuleInstanceCommon *module_inst =
        wasm_exec_env_get_module_inst(exec_env);
    uint32 stack_end;

    stack_end = wasm_runtime_module_malloc_internal(module_inst, exec_env,
                                                    cluster->stack_size, NULL);
    *start = stack_end + cluster->stack_size;
    *size = cluster->stack_size;

    return stack_end != 0;
#else

my allocator config: (use my host os heap)

    init_args.mem_alloc_type = Alloc_With_Allocator;
    init_args.mem_alloc_option.allocator.malloc_func = malloc;
    init_args.mem_alloc_option.allocator.realloc_func = realloc;
    init_args.mem_alloc_option.allocator.free_func = free;

    /* initialize runtime environment */
    if (!wasm_runtime_full_init(&init_args)) {
        printf("Init runtime environment failed.\n");
        return;
    }

my question is which memory heap it use, I will enlarge the heap.

Thanks.

[yamt]

if set -z stack-size=65535 also have the problem if remove --max-threads=xxx or smaller to --max-threads=6, the test pass

iirc 65536 is the default. you should use even larger.

65536 / (20 + 1) = ~3KB.
note that fd_set is about 4KB by default.

my question is, where to set the aux stack pool size, if my test is more threads enabled? Thanks.

it's proportional to max threads.

[yamt]

one more question: (this is not linux host, is a embeded freertos host.) if WAMR_BUILD_LIB_WASI_THREADS=1 start up errors:

[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed                                                                                                                        
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread

failed source

static bool
allocate_aux_stack(WASMExecEnv *exec_env, uint32 *start, uint32 *size)
{
    WASMCluster *cluster = wasm_exec_env_get_cluster(exec_env);
#if WASM_ENABLE_HEAP_AUX_STACK_ALLOCATION != 0
    WASMModuleInstanceCommon *module_inst =
        wasm_exec_env_get_module_inst(exec_env);
    uint32 stack_end;

    stack_end = wasm_runtime_module_malloc_internal(module_inst, exec_env,
                                                    cluster->stack_size, NULL);
    *start = stack_end + cluster->stack_size;
    *size = cluster->stack_size;

    return stack_end != 0;
#else

my allocator config: (use my host os heap)

    init_args.mem_alloc_type = Alloc_With_Allocator;
    init_args.mem_alloc_option.allocator.malloc_func = malloc;
    init_args.mem_alloc_option.allocator.realloc_func = realloc;
    init_args.mem_alloc_option.allocator.free_func = free;

    /* initialize runtime environment */
    if (!wasm_runtime_full_init(&init_args)) {
        printf("Init runtime environment failed.\n");
        return;
    }

my question is which memory heap it use, I will enlarge the heap.

Thanks.

in case of wasi-threads, aux stack is allocated from libc heap in the module itself.
i guess you need to use larger -Wl,--max-memory=xxx.

[tkernelcn]

if set -z stack-size=65535 also have the problem if remove --max-threads=xxx or smaller to --max-threads=6, the test pass

iirc 65536 is the default. you should use even larger.

65536 / (20 + 1) = ~3KB. note that fd_set is about 4KB by default.

my question is, where to set the aux stack pool size, if my test is more threads enabled? Thanks.

it's proportional to max threads.

actually -z stack-size=65536 is enough, I even set -z stack-size=98304; but if not set --max-threads= or set smaller to --max-threads=6 will pass, otherwise failed, so maybe --max-threads divider have higher priority than -z stack-size=xxx

[tkernelcn]

one more question: (this is not linux host, is a embeded freertos host.) if WAMR_BUILD_LIB_WASI_THREADS=1 start up errors:

[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed                                                                                                                        
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread
[00:00:00:000 - 567AC350]: warning: allocate 65536 bytes memory failed
[00:00:00:000 - 567AC350]: thread manager error: failed to allocate aux stack space for new thread

failed source

static bool
allocate_aux_stack(WASMExecEnv *exec_env, uint32 *start, uint32 *size)
{
    WASMCluster *cluster = wasm_exec_env_get_cluster(exec_env);
#if WASM_ENABLE_HEAP_AUX_STACK_ALLOCATION != 0
    WASMModuleInstanceCommon *module_inst =
        wasm_exec_env_get_module_inst(exec_env);
    uint32 stack_end;

    stack_end = wasm_runtime_module_malloc_internal(module_inst, exec_env,
                                                    cluster->stack_size, NULL);
    *start = stack_end + cluster->stack_size;
    *size = cluster->stack_size;

    return stack_end != 0;
#else

my allocator config: (use my host os heap)

    init_args.mem_alloc_type = Alloc_With_Allocator;
    init_args.mem_alloc_option.allocator.malloc_func = malloc;
    init_args.mem_alloc_option.allocator.realloc_func = realloc;
    init_args.mem_alloc_option.allocator.free_func = free;

    /* initialize runtime environment */
    if (!wasm_runtime_full_init(&init_args)) {
        printf("Init runtime environment failed.\n");
        return;
    }

my question is which memory heap it use, I will enlarge the heap.
Thanks.

in case of wasi-threads, aux stack is allocated from libc heap in the module itself. i guess you need to use larger -Wl,--max-memory=xxx.

Thanks, it works with -Wl,--max-memory=262144 -z stack-size=65536

[yamt]

if set -z stack-size=65535 also have the problem if remove --max-threads=xxx or smaller to --max-threads=6, the test pass

iirc 65536 is the default. you should use even larger.
65536 / (20 + 1) = ~3KB. note that fd_set is about 4KB by default.

my question is, where to set the aux stack pool size, if my test is more threads enabled? Thanks.

it's proportional to max threads.

actually -z stack-size=65536 is enough, I even set -z stack-size=98304; but if not set --max-threads= or set smaller to --max-threads=6 will pass, otherwise failed, so maybe --max-threads divider have higher priority than -z stack-size=xxx

if you have stack-size=S and max-threads=N, stack size for a thread is about S / (N + 1).
note: this calculation only applies to WAMR_BUILD_LIB_PTHREAD version of pthread.

[tkernelcn]

if set -z stack-size=65535 also have the problem if remove --max-threads=xxx or smaller to --max-threads=6, the test pass

iirc 65536 is the default. you should use even larger.
65536 / (20 + 1) = ~3KB. note that fd_set is about 4KB by default.

my question is, where to set the aux stack pool size, if my test is more threads enabled? Thanks.

it's proportional to max threads.

actually -z stack-size=65536 is enough, I even set -z stack-size=98304; but if not set --max-threads= or set smaller to --max-threads=6 will pass, otherwise failed, so maybe --max-threads divider have higher priority than -z stack-size=xxx

if you have stack-size=S and max-threads=N, stack size for a thread is about S / (N + 1). note: this calculation only applies to WAMR_BUILD_LIB_PTHREAD version of pthread.

Thanks, clear!!

[tkernelcn]

one more question, is there any approach can set different thread different stack size?
for embedded system no enough memory, this feature is helpful.

[yamt]

one more question, is there any approach can set different thread different stack size? for embedded system no enough memory, this feature is helpful.

for WAMR_BUILD_LIB_PTHREAD, no.
for wasi-threads, you can use pthread_attr_setstacksize.

[tkernelcn]

is it support dynamic stack? that means before stack overflow VMCore will extend stack size
if support that, how to enable it.
thanks.

[yamt]

is it support dynamic stack? that means before stack overflow VMCore will extend stack size if support that, how to enable it. thanks.

for some platforms like linux, the memory might be demand-paged.
otherwise, no.

[tkernelcn]

Thanks a lot!

[tkernelcn]

one more question, is there any approach can set different thread different stack size? for embedded system no enough memory, this feature is helpful.

for WAMR_BUILD_LIB_PTHREAD, no. for wasi-threads, you can use pthread_attr_setstacksize.

I try to set pthread stack size but popup below exception, is there any missing configurations or wasi-sdk not support?

[00:00:00:000 - 567CDFF0]: warning: failed to link import function (env, pthread_attr_init)
[00:00:00:000 - 567CDFF0]: warning: failed to link import function (env, pthread_attr_setstacksize)
[00:00:00:000 - 567CDFF0]: warning: failed to link import function (env, pthread_attr_destroy)
Exception: failed to call unlinked import function (env, pthread_attr_init)

[yamt]

one more question, is there any approach can set different thread different stack size? for embedded system no enough memory, this feature is helpful.

for WAMR_BUILD_LIB_PTHREAD, no. for wasi-threads, you can use pthread_attr_setstacksize.

I try to set pthread stack size but popup below exception, is there any missing configurations or wasi-sdk not support?

[00:00:00:000 - 567CDFF0]: warning: failed to link import function (env, pthread_attr_init)
[00:00:00:000 - 567CDFF0]: warning: failed to link import function (env, pthread_attr_setstacksize)
[00:00:00:000 - 567CDFF0]: warning: failed to link import function (env, pthread_attr_destroy)
Exception: failed to call unlinked import function (env, pthread_attr_init)

i'm sure the latest wasi-sdk has pthread_attr_setstacksize.
i suspect you have some issues in your build procedure.

[tkernelcn]

below is my build options，would you please share your build cmdline for comparing, Thanks.
PS: a lot of -D and -I that passed from top level of CMakeLists, please just focus on last line.
the thread can be created and run, only issue is attribute related api.

[ 87%] Building C object lib/wasm/wasi-apps/CMakeFiles/test-apps.wasm.dir/test_thread.c.o
cd /home/peter/git_workspace/eval_wasm/build/lib/wasm/wasi-apps && /opt/wasi-sdk/bin/clang 
--sysroot=/opt/wasi-sdk/share/wasi-sysroot 
-DBH_FREE=wasm_runtime_free 
-DBH_MALLOC=wasm_runtime_malloc 
-DBH_PLATFORM_FREERTOS=1 
-DBUILD_ALL 
-DBUILD_LIBC_WASI 
-DBUILD_NX_VFS 
-DBUILD_TARGET_X86_32 
-DBUILD_TLSF 
-DDEBUG 
-DMY_DEBUG=1 
-DWASM_DISABLE_HW_BOUND_CHECK=1 
-DWASM_DISABLE_STACK_HW_BOUND_CHECK=1 
-DWASM_DISABLE_WAKEUP_BLOCKING_OP=0 
-DWASM_ENABLE_AOT=1 
-DWASM_ENABLE_BULK_MEMORY=1 
-DWASM_ENABLE_DEBUG_AOT=1 
-DWASM_ENABLE_DEBUG_INTERP=1 
-DWASM_ENABLE_DUMP_CALL_STACK=1 
-DWASM_ENABLE_FAST_INTERP=0 
-DWASM_ENABLE_INTERP=1 
-DWASM_ENABLE_LIBC_BUILTIN=1 
-DWASM_ENABLE_LIBC_WASI=1 
-DWASM_ENABLE_LIB_PTHREAD=1 
-DWASM_ENABLE_LIB_PTHREAD_SEMAPHORE=1 
-DWASM_ENABLE_MINI_LOADER=0 
-DWASM_ENABLE_MODULE_INST_CONTEXT=1 
-DWASM_ENABLE_MULTI_MODULE=1 
-DWASM_ENABLE_SHARED_MEMORY=1 
-DWASM_ENABLE_THREAD_MGR=1 
-DWASM_GLOBAL_HEAP_SIZE=32768 
-D_LINUX_ 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/aot 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/libc-builtin 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/include 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/debug-engine 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/include 
-I/home/peter/git_workspace/eval_wasm/lib/FreeRTOS 
-I/home/peter/git_workspace/eval_wasm/lib/FreeRTOS/Source/include 
-I/home/peter/git_workspace/eval_wasm/lib/FreeRTOS/Source/portable/ThirdParty/GCC/Posix 
-I/home/peter/git_workspace/eval_wasm/lib/FreeRTOS/Source/portable/ThirdParty/GCC/Posix/utils 
-I/home/peter/git_workspace/eval_wasm/lib/FreeRTOS/cmsis 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/init 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/contrib/ports/unix 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/contrib/ports/unix/port/include 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/src/netif/ppp 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/src/include 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/src/include/netif 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/src/include/lwip 
-I/home/peter/git_workspace/eval_wasm/lib/lwip/src/include/compat 
-I/home/peter/git_workspace/eval_wasm/lib/thirdparty/littlefs/port 
-I/home/peter/git_workspace/eval_wasm/lib/thirdparty/littlefs/bd 
-I/home/peter/git_workspace/eval_wasm/lib/thirdparty/littlefs 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/platform/freertos 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/shared/platform/include 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/platform/freertos/fs 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/platform/freertos/fs/include 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/shared/platform/common/libc-util 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/shared/mem-alloc 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/shared/utils 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasi-apps/../wasm-micro-runtime/core/iwasm/libraries/lib-socket/inc 
-I/home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-socket/inc  
-pthread -O3 -g   -m32 -o CMakeFiles/test-apps.wasm.dir/test_thread.c.o   -c /home/peter/git_workspace/eval_wasm/lib/wasm/wasi-apps/test_thread.c

[tkernelcn]

add linking part

[ 87%] Linking C executable ../../../bin/test-apps.wasm
cd /home/peter/git_workspace/qcx216_simulator/build/lib/wasm/wasi-apps 
&& /usr/bin/cmake -E cmake_link_script CMakeFiles/test-apps.wasm.dir/link.txt --verbose=1
/opt/wasi-sdk/bin/clang --sysroot=/opt/wasi-sdk/share/wasi-sysroot -pthread -O3 -g  
-Wl,--no-entry,--strip-all, -Wl,--export=__heap_base,--export=__data_end 
-Wl,--export=malloc -Wl,--export=free -Wl,--export=main -Wl,--export=__main_argc_argv 
-Wl,--allow-undefined,--no-check-features -Wl,--max-memory=262144 -z stack-size=65536 
CMakeFiles/test-apps.wasm.dir/test.c.o 
CMakeFiles/test-apps.wasm.dir/test_cond.c.o 
CMakeFiles/test-apps.wasm.dir/test_dir.c.o 
CMakeFiles/test-apps.wasm.dir/test_dns.c.o 
CMakeFiles/test-apps.wasm.dir/test_event.c.o 
CMakeFiles/test-apps.wasm.dir/test_file.c.o 
CMakeFiles/test-apps.wasm.dir/test_main.c.o 
CMakeFiles/test-apps.wasm.dir/test_mutex.c.o 
CMakeFiles/test-apps.wasm.dir/test_rwlock.c.o 
CMakeFiles/test-apps.wasm.dir/test_select.c.o 
CMakeFiles/test-apps.wasm.dir/test_sem.c.o 
CMakeFiles/test-apps.wasm.dir/test_socket.c.o 
CMakeFiles/test-apps.wasm.dir/test_thread.c.o  
-o ../../../bin/test-apps.wasm ../../libsocket_wasi_ext.a

[yamt]

try:

/opt/wasi-sdk/bin/clang -pthread --target=wasm32-wasi-threads -Wl,--import-memory -Wl,--export-memory -Wl,--max-memory=262144 -z stack-size=65536

[tkernelcn]

try:

/opt/wasi-sdk/bin/clang -pthread --target=wasm32-wasi-threads -Wl,--import-memory -Wl,--export-memory -Wl,--max-memory=262144 -z stack-size=65536

popup info when running:
WASM module load failed: no sub module reader to load wasi

[yamt]

try:

/opt/wasi-sdk/bin/clang -pthread --target=wasm32-wasi-threads -Wl,--import-memory -Wl,--export-memory -Wl,--max-memory=262144 -z stack-size=65536

popup info when running: WASM module load failed: no sub module reader to load wasi

ensure that your wamr build has:
-DWAMR_BUILD_LIBC_WASI=1 -DWAMR_BUILD_LIB_WASI_THREADS=1

[tkernelcn]

Thank you very much!
summary, missing -DWAMR_BUILD_LIB_WASI_THREADS=1 and --target=wasm32-wasi-threads
currently no warning: failed to link import function (env, pthread_attr_setstacksize)

but unfortunately when thread exit, crash: I will debug the new problem

Thread 9 "simulator" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf694eb40 (LWP 13544)]
0x5658890d in wasm_exec_env_get_cluster (exec_env=0x20000) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr/thread_manager.c:392
392	    return exec_env->cluster;
(gdb) bt
#0  0x5658890d in wasm_exec_env_get_cluster (exec_env=0x20000) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr/thread_manager.c:392
#1  0x565af918 in delete_thread_info_node (thread_info=0xf6c0fbf0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:432
#2  0x565b0168 in pthread_exit_wrapper (exec_env=0xf6c0fdf0, retval_offset=0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:790
#3  0x5655fcbd in skip_push_args ()
#4  0xf6c0fdf0 in ?? ()
#5  0x5655e257 in wasm_runtime_invoke_native (exec_env=0xf6c0fdf0, func_ptr=0x565b0113 <pthread_exit_wrapper>, func_type=0xf6c011d0, signature=0x566118ea "(i)", attachment=0x0, 
    argv=0xf6c1061c, argc=1, argv_ret=0xf694d2cc) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common/wasm_runtime_common.c:4303
#6  0x56565294 in wasm_interp_call_func_native (module_inst=0xf6c0cdc0, exec_env=0xf6c0fdf0, cur_func=0xf6c0d0a0, prev_frame=0xf6c10584)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:933
#7  0x5657b57a in wasm_interp_call_func_bytecode (module=0xf6c0cdc0, exec_env=0xf6c0fdf0, cur_func=0xf6c0d0a0, prev_frame=0xf6c10584)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:3902
#8  0x5657bedf in wasm_interp_call_wasm (module_inst=0xf6c0cdc0, exec_env=0xf6c0fdf0, function=0xf6c0fbf0, argc=2, argv=0xf694e2b0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:4295
#9  0x56562e39 in wasm_call_function (exec_env=0xf6c0fdf0, function=0xf6c0fbf0, argc=2, argv=0xf694e2b0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_runtime.c:2386
#10 0x5655c40e in wasm_runtime_call_wasm (exec_env=0xf6c0fdf0, function=0xf6c0fbf0, argc=2, argv=0xf694e2b0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common/wasm_runtime_common.c:2043
#11 0x565aea8a in thread_start (arg=0xf6c0fdf0) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-wasi-threads/lib_wasi_threads_wrapper.c:60
#12 0x56588c41 in thread_manager_start_routine (arg=0xf6c0fdf0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr/thread_manager.c:635
#13 0x56596b77 in os_thread_wrapper (arg=0xf6c12590) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/shared/platform/common/freertos/freertos_thread.c:206
#14 0x566074eb in prvWaitForStart (pvParams=0x566d5f18 <ucHeap+10160>) at /home/peter/git_workspace/eval_wasm/lib/FreeRTOS/Source/portable/ThirdParty/GCC/Posix/port.c:482
#15 0xf7f913bd in start_thread (arg=0xf694eb40) at pthread_create.c:463
#16 0xf7e9dc96 in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:108
(gdb) 

[tkernelcn]

would you please help to explain the internal meaning of -Wl,--import-memory -Wl,--export-memory, thanks.

[yamt]

Thank you very much! summary, missing -DWAMR_BUILD_LIB_WASI_THREADS=1 and --target=wasm32-wasi-threads currently no warning: failed to link import function (env, pthread_attr_setstacksize)

but unfortunately when thread exit, crash: I will debug the new problem

Thread 9 "simulator" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf694eb40 (LWP 13544)]
0x5658890d in wasm_exec_env_get_cluster (exec_env=0x20000) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr/thread_manager.c:392
392	    return exec_env->cluster;
(gdb) bt
#0  0x5658890d in wasm_exec_env_get_cluster (exec_env=0x20000) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr/thread_manager.c:392
#1  0x565af918 in delete_thread_info_node (thread_info=0xf6c0fbf0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:432
#2  0x565b0168 in pthread_exit_wrapper (exec_env=0xf6c0fdf0, retval_offset=0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:790
#3  0x5655fcbd in skip_push_args ()
#4  0xf6c0fdf0 in ?? ()
#5  0x5655e257 in wasm_runtime_invoke_native (exec_env=0xf6c0fdf0, func_ptr=0x565b0113 <pthread_exit_wrapper>, func_type=0xf6c011d0, signature=0x566118ea "(i)", attachment=0x0, 
    argv=0xf6c1061c, argc=1, argv_ret=0xf694d2cc) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common/wasm_runtime_common.c:4303
#6  0x56565294 in wasm_interp_call_func_native (module_inst=0xf6c0cdc0, exec_env=0xf6c0fdf0, cur_func=0xf6c0d0a0, prev_frame=0xf6c10584)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:933
#7  0x5657b57a in wasm_interp_call_func_bytecode (module=0xf6c0cdc0, exec_env=0xf6c0fdf0, cur_func=0xf6c0d0a0, prev_frame=0xf6c10584)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:3902
#8  0x5657bedf in wasm_interp_call_wasm (module_inst=0xf6c0cdc0, exec_env=0xf6c0fdf0, function=0xf6c0fbf0, argc=2, argv=0xf694e2b0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:4295
#9  0x56562e39 in wasm_call_function (exec_env=0xf6c0fdf0, function=0xf6c0fbf0, argc=2, argv=0xf694e2b0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_runtime.c:2386
#10 0x5655c40e in wasm_runtime_call_wasm (exec_env=0xf6c0fdf0, function=0xf6c0fbf0, argc=2, argv=0xf694e2b0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common/wasm_runtime_common.c:2043
#11 0x565aea8a in thread_start (arg=0xf6c0fdf0) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-wasi-threads/lib_wasi_threads_wrapper.c:60
#12 0x56588c41 in thread_manager_start_routine (arg=0xf6c0fdf0)
    at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/thread-mgr/thread_manager.c:635
#13 0x56596b77 in os_thread_wrapper (arg=0xf6c12590) at /home/peter/git_workspace/eval_wasm/lib/wasm/wasm-micro-runtime/core/shared/platform/common/freertos/freertos_thread.c:206
#14 0x566074eb in prvWaitForStart (pvParams=0x566d5f18 <ucHeap+10160>) at /home/peter/git_workspace/eval_wasm/lib/FreeRTOS/Source/portable/ThirdParty/GCC/Posix/port.c:482
#15 0xf7f913bd in start_thread (arg=0xf694eb40) at pthread_create.c:463
#16 0xf7e9dc96 in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:108
(gdb) 

wasi-threads version of pthread doesn't have pthread_exit.
it seems you are mixing two pthread implementations.

[yamt]

would you please help to explain the internal meaning of -Wl,--import-memory -Wl,--export-memory, thanks.

wasi requires apps to export memory.
wasi-threads requires it to import memory.

[tkernelcn]

//wasi-threads version of pthread doesn't have pthread_exit.

so how to exit it safely in this configuration, just return?

[tkernelcn]

I test just uncoment pthread_exit, the test pass
//pthread_exit(NULL);

@yamt Thanks for your help!

[yamt]

//wasi-threads version of pthread doesn't have pthread_exit.

so how to exit it safely in this configuration, just return?

depending on how your app uses pthread_exit, it might be fine to just return. yes.

otherwise, explain your use case on WebAssembly/wasi-threads#7.

[tkernelcn]

thanks for your sharing, close.