Issue loading AOT on iwasm built for x86 windows

[iKlask]

I've been having issues getting AOT code to run on a WAMR runtime built for Win32. I'm not sure if there's something I'm missing or if there is an actual issue with windows 32 bit iwasm/wamrc.

When I take an example .wasm binary and compile it with wamrc and run on my win32-built iwasm, I get this error:

AOT module load failed: resolve symbol _aot_stack_sizes failed

I compiled my wasm file for i386, since compiling for x86_64 throws an error in my 32-bit iwasm:

AOT module load failed: invalid target bit width, expected 32-bit but got 64-bit

I noticed _aot_stack_sizes does not exist on the x86_64 compilation, which works fine in iwasm built for x64 windows. The symbol aot_func_internal also seems to exist on the 32 bit compilation where it does not on my x64.

If I compile my wasm code with --bounds-checks=0 (default in x64) then everything seems to work, but bounds checks is enabled by default for x86 so why does this not work with this default setting on?

[wenyongh]

@iKlask The symbol name of windows 32 may start with '_', so "_aot_stack_sizes" should be same as "aot_stack_sizes" in windows/linux 64, since the relocation for the latter is redirected into the relocation for the name of ".aot_stack_sizes" (the section name), I doubt we should redirect "_aot_stack_sizes" also. Could you please try the patch below:

diff --git a/core/iwasm/compilation/aot_emit_aot_file.c b/core/iwasm/compilation/aot_emit_aot_file.c
index 758681d6..3bad41f3 100644
--- a/core/iwasm/compilation/aot_emit_aot_file.c
+++ b/core/iwasm/compilation/aot_emit_aot_file.c
@@ -3947,7 +3947,12 @@ aot_resolve_object_relocation_group(AOTObjectData *obj_data,
          * Note: aot_stack_sizes_section_name section only contains
          * stack_sizes table.
          */
-        if (!strcmp(relocation->symbol_name, aot_stack_sizes_name)) {
+        if (!strcmp(relocation->symbol_name, aot_stack_sizes_name)
+            /* in windows 32, the symbol name may start with '_' */
+            || (strlen(relocation->symbol_name) > 0
+                && relocation->symbol_name[0] == '_'
+                && !strcmp(relocation->symbol_name + 1,
+                           aot_stack_sizes_name))) {
             /* discard const */
             relocation->symbol_name = (char *)aot_stack_sizes_section_name;
         }

[iKlask]

since the relocation for the latter is redirected into the relocation for the name of ".aot_stack_sizes" (the section name), I doubt we should redirect "_aot_stack_sizes" also.

Sorry I'm a little confused on that. So "_aot_stack_sizes" is not related to ".aot_stack_sizes" or is it? I understand how win32 can add '_' to symbols, so should this be the same as the ".aot_stack_sizes" section? I noticed that during load relocations (in do_text_relocation()) there is a check to see if the symbol name is the same as the macro AOT_STACK_SIZES_SECTION_NAME which is defined in aot.h as:

#define AOT_STACK_SIZES_SECTION_NAME ".aot_stack_sizes"

it fails that check and eventually moves onto resolve_target_sym() which fails to find this as a symbol in the target_symbol_map (does not exist). Should it be passing that check or is something else the problem?

this is the cmake config file I use for building my iwasm.exe, could I be missing something or doing something wrong?

set (WAMR_BUILD_PLATFORM "windows")
set (WAMR_BUILD_TARGET X86_32)

set (WAMR_BUILD_INTERP 1)
set (WAMR_BUILD_FAST_INTERP 0)
set (WAMR_BUILD_AOT 1)
set (WAMR_BUILD_JIT 0)
set (WAMR_BUILD_FAST_JIT 0)

set (WAMR_BUILD_SIMD 1)
set (WAMR_BUILD_TAIL_CALL 1)

set (WAMR_CONFIGUABLE_BOUNDS_CHECKS 1)

set (CMAKE_C_STANDARD 11)
set (CMAKE_CXX_STANDARD 20)

[wenyongh]

Yes, they are the same, "aot_stack_sizes" is the name of an internal global array and the array is put into a data section named ".aot_stack_sizes", and the aot code accesses the array elements, here is a sample LLVM IR file generated by wamrc --format=llvmir-unopt --target=i386 --target-abi=msvc -o test.ll <wasm_file>:

; ModuleID = 'WASM Module'
source_filename = "WASM Module"
target datalayout = "e-m:x-p:32:32-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32-a:0:32-S32"
target triple = "i386-pc-windows-msvc"

@aot_stack_sizes = internal global [14 x i32] [i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1, i32 -1], section ".aot_stack_sizes"

@aot_stack_sizes_alias = alias [14 x i32], ptr @aot_stack_sizes

; Function Attrs: noinline
define i32 @"aot_func#0"(ptr %exec_env) #0 {

Note that the data section ".aot_stack_sizes" (but not "aot_stack_sizes" or "_aot_stack_sizes") will be created by aot loader, so the relocation to "aot_stack_sizes" or "_aot_stack_sizes" should be change to the relocation to that data section instead. Could you leave iwasm unchanged, apply the patch of aot_emit_aot_file.c, rebuild wamrc and generate the AOT file and test again?

[iKlask]

Ah I see now. Yes applying that patch for wamrc fixed the "aot_stack_sizes" issue, but now it moves onto the the other symbol which was added by --bounds-checks=1 and fails again. I assume there's something else that needs to be patched for the leading "_":

AOT module load failed: resolve symbol _aot_func_internal#0 failed

[wenyongh]

Yes, please try:

diff --git a/core/iwasm/aot/aot_loader.c b/core/iwasm/aot/aot_loader.c
index 759954ad..3e832c27 100644
--- a/core/iwasm/aot/aot_loader.c
+++ b/core/iwasm/aot/aot_loader.c
@@ -2917,6 +2917,17 @@ do_text_relocation(AOTModule *module, AOTRelocationGroup *group,
             }
             symbol_addr = module->func_ptrs[func_index];
         }
+        else if (!strncmp(symbol, "_" AOT_FUNC_INTERNAL_PREFIX,
+                          strlen("_" AOT_FUNC_INTERNAL_PREFIX))) {
+            p = symbol + strlen("_" AOT_FUNC_INTERNAL_PREFIX);
+            if (*p == '\0'
+                || (func_index = (uint32)atoi(p)) > module->func_count) {
+                set_error_buf_v(error_buf, error_buf_size, "invalid symbol %s",
+                                symbol);
+                goto check_symbol_fail;
+            }
+            symbol_addr = module->func_ptrs[func_index];
+        }
 #endif
         else if (is_text_section(symbol)) {
             symbol_addr = module->code;

[wenyongh]

Please see #3231.

[iKlask]

Thank you.

EDIT: I realize now no need to compile wamrc for 32 bit in my case. the fixes in aot_loader are intended for the 32bit runtime. With the fixes in #3231 my wasm binary can be compiled to 32bit aot and execute on a 32bit runtime.

However I'm having issues building the x86 version of wamrc. I have been building and using an x64 version of wamrc to compile 32bit AOT code, however one of the fixes you linked is wrapped around a macro that expects defined(BUILD_TARGET_X86_32).

When I try regenerating the cmake project for the wamr compiler with "-DWAMR_BUILD_TARGET=x86_32", it enables the correct flags but does not setup the visual studio project configuration to Win32.

When forcing win32 in visual studio I end up with other errors since dependency static libraries like vmlib, aotclib, uvwasi_a, and uv_a are all generated and built for x64.

I'm not sure if theres something simple im missing or if I need to be forcing the WAMR_BUILD_TARGET on every dependency somehow.

[wenyongh]

Thank you.

EDIT: I realize now no need to compile wamrc for 32 bit in my case. the fixes in aot_loader are intended for the 32bit runtime. With the fixes in #3231 my wasm binary can be compiled to 32bit aot and execute on a 32bit runtime.

However I'm having issues building the x86 version of wamrc. I have been building and using an x64 version of wamrc to compile 32bit AOT code, however one of the fixes you linked is wrapped around a macro that expects defined(BUILD_TARGET_X86_32).

When I try regenerating the cmake project for the wamr compiler with "-DWAMR_BUILD_TARGET=x86_32", it enables the correct flags but does not setup the visual studio project configuration to Win32.

When forcing win32 in visual studio I end up with other errors since dependency static libraries like vmlib, aotclib, uvwasi_a, and uv_a are all generated and built for x64.

I'm not sure if theres something simple im missing or if I need to be forcing the WAMR_BUILD_TARGET on every dependency somehow.

OK, so let's merge #3231 first. For building 32-bit wamr compiler on Windows x86_64, we have no experience either, per my understanding, the llvm should be built as 32-bit first, and setting -DWAMR_BUILD_TARGET=x86_32 for wamrc-compiler cmake project may be not enough, maybe you should also setting some other cmake flags like CMAKE_SYSTEM_PROCESSOR?