Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge branch main into dev/exce_handling #2600

Merged
merged 123 commits into from
Sep 27, 2023
Merged

Merge branch main into dev/exce_handling #2600

merged 123 commits into from
Sep 27, 2023

Conversation

wenyongh
Copy link
Contributor

No description provided.

yamt and others added 30 commits June 30, 2023 10:22
Calling `__wasi_sock_addr_resolve` syscall causes native stack overflow.
Given this is a standard function available in WAMR, we should have at least
the default stack size large enough to handle this case.

The socket tests were updated so they also run in separate thread, but
the simple retro program is:

```C
void *th(void *p)
{
    struct addrinfo *res;
    getaddrinfo("amazon.com", NULL, NULL, &res);
    return NULL;
}

int main(int argc, char **argv)
{
    pthread_t pt;
    pthread_create(&pt, NULL, th, NULL);
    pthread_join(pt, NULL);
    return 0;
}
```
Add shorthand "--xip" option for wamrc, which is equal to
"--enalbe-indirect-mode --disable-llvm-intrinsics"
Add "--enable-llvm-passes=<passes>" option to wamrc for customizing LLVM passes
Allow to use `cmake -DWAMR_CONFIGURABLE_BOUNDS_CHECKS=1` to
build iwasm, and then run `iwasm --disable-bounds-checks` to disable the
memory access boundary checks.

And add two APIs:
`wasm_runtime_set_bounds_checks` and `wasm_runtime_is_bounds_checks_enabled`
Because it involves relocations for the table. (.Lswitch.table.XXX)

Discussions: #2316
- Provide a Dockerfile.old to fix issue of ESP32 custom linker scripts not working
  properly with the newer version of Zephyr, as reported in #2263
- Provide a Dockerfile with newer Zephyr for other boards
- Update the corresponding document
It's rare to require app heap with wasi and sometimes harmful in some cases:
#2275
Dockerfile had command 'pip install --no-cache-doir' which caused
the image build to fail. Fix this to correctly say '--no-cache-dir'.
In the codebase, the struct and functions were written without "f" for dwarf.
esp32-s3's instruction memory and data memory can be accessed through mutual mirroring way,
so we define a new feature named as WASM_MEM_DUAL_BUS_MIRROR.
We have observed a significant performance degradation after merging
#1991
Instead of protecting suspend flags with a mutex, we implement the flags
as atomic variable and only use mutex when atomics are not available
on a given platform.
LVM JIT failed to lookup symbol "aot_stack_sizes" as it is an internal symbol,
change to lookup "aot_stack_sizes_alias" instead.

Reported in #2372.
…2374)

Devcontainer is too old to catch up the latest versions of dependencies. This PR updates
the libraries & tools that WASM environment requires for Docker image.

**Impediments:**
- LLVM 16 can not be retrieved directly by using shell command as provided before.
So we upgrade all tools manually as a workaround and just put LLVM issue URL as a
comment in the script. 

**Improvements:**
- Use `Debian 12 (Bookworm)` as a base image instead of `Ubuntu 20.04`.
- `GCC9` upgraded to `GCC12`.
- `LLVM14` upgraded to `LLVM16`.
- `Clang10` upgraded to `Clang10`.
- `Binaryen111` upgraded to `Binaryen114`.
- `WASI-19` upgraded to `WASI-20`
- `WABT-1.0.29` upgraded to `WABT-1.0.33`
- `Bazelisk-1.12.0` upgraded to `Bazelisk-1.17.0`
- `GithubCLI-2.20.2` upgraded to `GithubCLI-2.32.0`
- `NodeJS-19.x` upgraded to `NodeJS-20.x`
- `EMSDK-3.0.0` upgraded to `EMSDK-3.1.43`

**Notes:**
`Python 2.7` is removed due to no support anymore and not found in the repository.
Fix some check issues on table.init, table.fill and table.copy, and unify the check method
for all running modes.
Fix issue #2390 and #2096.
The old method may not work for some cases. This PR iterates over all instructions
in the function, looking for memcpy, memmove and memset instructions, putting
them into a set, and finally expands them into a loop one by one.

And move this LLVM Pass after building the pipe line of pass builder to ensure that
the memcpy/memmove/memset instrinsics are generated before applying the pass.
cngzhnp and others added 29 commits September 7, 2023 14:29
Introduce module instance context APIs which can set one or more contexts created
by the embedder for a wasm module instance:
```C
    wasm_runtime_create_context_key
    wasm_runtime_destroy_context_key
    wasm_runtime_set_context
    wasm_runtime_set_context_spread
    wasm_runtime_get_context
```

And make libc-wasi use it and set wasi context as the first context bound to the wasm
module instance.

Also add samples.

Refer to #2460.
This fixes a few test cases in wasi-threads testsuite like wasi_threads_return_main_block.
And also move the special handling for "wasi proc exit" to a more appropriate place.
Preserve errno because this function is often used like
the following. The caller wants to report the error from the main
operation (`lseek` in this example), not from fd_object_release.

```
    off_t ret = lseek(fd_number(fo), offset, nwhence);
    fd_object_release(fo);
    if (ret < 0)
        return convert_errno(errno);
```
Add API wasm_runtime_terminate to terminate a module instance
by setting "terminated by user" exception to the module instance.

And update the product-mini of posix platforms.

Note: this doesn't work for some situations like blocking system calls.
Instead of showing the total case amount under *spec/test/core*, show
the accurate number of how many test cases were executed.
Remove thread local attribute of prev_sig_act_SIGSEGV/SIGBUS to allow using
custom signal handler from non-main thread since in a thread spawned by
embedder, embedder may be unable to call wasm_runtime_init_thread_env to
initialize them.

And fix the handling of prev_sig_act when its sa_handler is SIG_DFL, SIG_IGN,
or a user customized handler.
Send a signal whose handler is no-op to a blocking thread to wake up
the blocking syscall with either EINTR equivalent or partial success.

Unlike the approach taken in the `dev/interrupt_block_insn` branch (that is,
signal + longjmp similarly to `OS_ENABLE_HW_BOUND_CHECK`), this PR
does not use longjmp because:
* longjmp from signal handler doesn't work on nuttx
  refer to apache/nuttx#10326
* the singal+longjmp approach may be too difficult for average programmers
  who might implement host functions to deal with

See also #1910
struct dfs_fd has been deprecated
This improves test consistency between typical local environments and
github runners.

This is necessary for some of latest wasi-threads tests.

cf. yamt/toywasm@570e670
The WASI docs allow for fewer rights to be applied to an fd than requested but
not more. This behavior is also asserted in the rust WASI tests, so it's necessary
for those to pass as well.
…2583)

Unaligned store v128 value to the AOT function argument of the pointer for
the extra return value may cause segmentation fault.

Fix the issue reported in #2556.
…2582)

Apply "provide stdin by ourselves" to the aot case and check the correct exit code.
Return a WASI error code (rather than a host POSIX one). In addition,
there is no need to return an error in the case that the provided buffer
is too large.
Adapt API usage to new interfaces where applicable, including LLVM function
usage, obsoleted llvm::Optional type and removal of unavailable headers.

Know issues:
- AOT static PGO isn't enabled
- LLVM JIT may run failed due to llvm_orc_registerEHFrameSectionWrapper
  isn't linked into iwasm
Explicitly declare readdir_r in gpc_code_gen_util.c.
On posix-like platforms, the rest of wasi-threads tests
should pass after the recent changes including the following PRs:
#2516, #2524, #2529, #2571, #2576 and #2582.
There doesn't appear to be a clear reason not to support this behavior.
It seems it was disallowed previously as a precaution. See
bytecodealliance/wasmtime@67e2e57
for more context.
`wasm_loader_push_pop_frame_offset` may pop n operands by using
`loader_ctx->stack_cell_num` to check whether the operand can be
popped or not. While `loader_ctx->stack_cell_num` is updated in the
later `wasm_loader_push_pop_frame_ref`, the check may fail if the stack
is in polymorphic state and lead to `ctx->frame_offset` underflow.

Fix issue #2577 and #2586.
The CI might use clang-17 to build iwasm for Android platform and it may
report compilation error:
https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/6308980430/job/17128073777

/home/runner/work/wasm-micro-runtime/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/blocking_op.c:45:19: error: call to undeclared function 'preadv'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    ssize_t ret = preadv(fd, iov, iovcnt, offset);
                  ^

Explicitly declare preadv and pwritev in android platform header file to resolve it.
Suppress a sanitizer integer overflow error in signed integer add operation
in Fast JIT.
@wenyongh wenyongh merged commit 95b19a5 into dev/exce_handling Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.