WASI support?

[GregoryConrad]

Hi! I was looking to see if redb could compile to wasm32-wasi and came across a handful of issues (with mmap and file locking):

unresolved import `crate::tree_store::page_store::file_lock::LockedFile`
failed to resolve: use of undeclared type `MmapInner`
cannot find type `MmapInner` in this scope

I was wondering if you would be willing to support WASI as a compilation target in redb. I might be able to PR to fix the above compilation issues and test the following functionality:

1. mmap (which it looks like you may have grabbed from the unmaintained memmap crate?) can probably be fixed by just copying the unix implementation (WASI has a libc that should be the same as unix)
2. file locking may be exposable from unix like mmap; but I am not sure. Some file locking APIs are not available in WASI libc

[cberner]

If it can be added as cleanly as the Windows support, then ya I'm open to merging a PR for wasm support

[GregoryConrad]

Great! I'll do some experimenting later to see how plausible it is with the current file locking mechanism.

On a completely different note, does redb support "unlimited" key/value sizes, or is there some hard limit? I didn't see any mention in the README or design.md. I.e., what happens if a key exceeds the page size?

[GregoryConrad]

Alright, some initial findings:

It looks like WASI support will actually be possible sometime in near-ish™️ future, at least using nightly! The errors I'm encountering during compilation come mostly from libc, which are fixable on libc's end. This is because the default libc does not include bindings for mmap and file locking. mmap is already in wasi-libc, and file locking should be coming soon due to the recent spec update here: WebAssembly/wasi-filesystem#2

The only changes necessary should be adding a cargo feature to libc for mmap support and then waiting for file locking support to trickle down through wasi-libc.

[GregoryConrad]

See rust-lang/libc#3091

[GregoryConrad]

From my comment in #508:

Based on https://github.com/WebAssembly/wasi-libc/blob/main/libc-bottom-half/mman/mman.c, it looks as though redb compiled to WASI would only work in read-only, if at all. Closing this PR as I don't think it makes any sense at the moment.

[brianorwhatever]

wasm support even if read only would be amazing

[GregoryConrad]

Since the mmap backend is gone now, this issue can be revisited since that was the blocking issue before. I've done a little experimenting and this is looking somewhat promising, at least when using nightly.

[GregoryConrad]

For future implementation:

• Once wasi-libc gets flock and those changes propagate to Rust's libc: 0001-Example-commit-for-when-WASI-has-flock.patch
• Once wasi no longer requires nightly: 0001-Example-commit-for-when-WASI-no-longer-requires-nigh.patch

[dpc]

What would it take to support indexeddb as a backend for wasm32-unknown-unknown? Am I wrong thinking that indexeddb could serve as a block allocator for redb?