Add initial support for WebAssembly in user-defined functions (UDF) #9108
Conversation
|
Very exciting with not much code needed! Wow
…On Wed, Jul 28, 2021 at 7:22 AM Piotr Sarna ***@***.***> wrote:
This series adds very basic support for WebAssembly-based user-defined
functions. It's only published as draft due to multiple reasons:
- only a few basic types are implemented so far (string, int, bool)
- Fedora 33's (and probably 34's) v8-devel package contains a version
of the library which seemingly does not support creating a single-threaded
v8 platform, and instead creates a thread pool underneath, which is out of
the question for production
- occasionally the execution of wasm-based UDF's fails with Uncaught
RangeError: Maximum call stack size exceeded; I found
denoland/rusty_v8#486 <denoland/rusty_v8#486>
which may or may not be relevant, although it also failed in a single-shard
Scylla, so perhaps it's something else; I tried out multiple stack-size
values and too low values make this error come out every time, but even
very large values (64MiB and beyond) do not prevent the error, which makes
me think it's some kind of a race after all, especially that the functions
used in tests are not particularly large
- many more xfailing tests should be added to the cql-pytest-based
suite
- it's a low priority project, so its development pace is going to be
slow
This series comes with a basic set of tests which were used to designate a
minimal goal for this initial implementation
------------------------------
You can view, comment on, or merge this pull request online at:
#9108
Commit Summary
- cql3: generalize user-defined functions for more languages
- service: add v8 engine
- wasm: add initial WebAssembly runtime implementation
- treewide: add initial WebAssembly support to UDF
- cql-pytest: add wasm-based tests for user-defined functions
File Changes
- *M* configure.py
<https://github.com/scylladb/scylla/pull/9108/files#diff-4d5f3192809ec1b9add6b33007e0c50031ad9a0a2f3f55a481b506468824db2c>
(6)
- *M* cql3/functions/user_function.cc
<https://github.com/scylladb/scylla/pull/9108/files#diff-a62e487ba8c6138647787ee95c9804404e3bf848b817895c5bb8501d4e41ba74>
(15)
- *M* cql3/functions/user_function.hh
<https://github.com/scylladb/scylla/pull/9108/files#diff-5a0593faed1341ce16308a4b5293ef254fd34a37bfd7aa86a818adc327cfb811>
(32)
- *M* cql3/statements/create_function_statement.cc
<https://github.com/scylladb/scylla/pull/9108/files#diff-2fd9642298844a7bcf239fc0a7962b6f13d1918340a1a8cf74804f73c4e1209f>
(22)
- *M* db/schema_tables.cc
<https://github.com/scylladb/scylla/pull/9108/files#diff-2cc928e139ea64f5257e11df02cd2819944fc93210107633315ee2d0edd05c27>
(26)
- *M* install-dependencies.sh
<https://github.com/scylladb/scylla/pull/9108/files#diff-4852d2a5a482159af418383e38099c1321b69c6272981405ed32d629a19de73d>
(2)
- *M* main.cc
<https://github.com/scylladb/scylla/pull/9108/files#diff-bf1eacd22947b4daf9f4c2639427b8593d489f093eb1acfbba3e4cc1c9b0288b>
(13)
- *A* service/v8_platform.cc
<https://github.com/scylladb/scylla/pull/9108/files#diff-8bfc1053920f0e573f4e7ee61a91ca483c147c54df17de6981f2afa981b27a4e>
(39)
- *A* service/v8_platform.hh
<https://github.com/scylladb/scylla/pull/9108/files#diff-ccf66231493bc4d6ca0c52209e303ef57b4f938a2fbd6c7bd1ed7c218a23d25a>
(36)
- *A* test/cql-pytest/test_wasm.py
<https://github.com/scylladb/scylla/pull/9108/files#diff-ba1d88f7ba5b62defa4c35ac8202b8aca9937c2b6196d8099027dcb15ed4d533>
(78)
- *A* wasm.cc
<https://github.com/scylladb/scylla/pull/9108/files#diff-1bb7990fd2fa76e45520a9f308e8c2e932aeb48ee1be5e59d6fa99d6e030a7c9>
(338)
- *A* wasm.hh
<https://github.com/scylladb/scylla/pull/9108/files#diff-aa94895055b55b0e011c84b8ef67450e78152e7ea73d0beac3859d69a5cd00f6>
(69)
Patch Links:
- https://github.com/scylladb/scylla/pull/9108.patch
- https://github.com/scylladb/scylla/pull/9108.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#9108>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANHURIXTOJNX56GTPTILK3T2AHDXANCNFSM5BEPCQLA>
.
|
|
I think we should look for an alternative to v8, it's a monster. |
|
|
wasm.cc
Outdated
| v8::Local<v8::Value> result; | ||
| if (!function->Call(local_ctx, local_ctx->Global(), argv.size(), argv.data()).ToLocal(&result)) { | ||
| throw make_wasm_exception("Running wasm function failed", isolate, try_catch); | ||
| } |
There was a problem hiding this comment.
I see you thoughtfully return a future, but we need to make the run preemptible somehow. Lua supports this by counting bytecode instructions and calling a callback on overflow.
There was a problem hiding this comment.
I've been thinking about that, and I have a follow-up question - did this Lua trick prove useful in tests, or was it done as a matter of good practice? I'm asking because when it comes to UDF, simple transformations performed on a single row shouldn't usually be large enough to produce stalls - I'd imagine that it's usually refactoring a string or performing some simple math operations on the input, not actually computing things with an O(n^3) algorithm. The reason I ask is that Lua's trick also forced the call to be wrapped in a Seastar thread, which hurts its scalability - which can be alleviated a little e.g. by using a smaller stack, but it's still overhead. Perhaps the decision whether to mark the transformation preemptible could be left for users' consideration? I.e. by default the function call is not preemptible, which means it's either fast enough or the user doesn't care about occasional latency spikes (which would be common for analytical workloads, and I suspect UDF's are more useful for analytics than online queries), and the user can explicitly mark the function as heavy, which will make Scylla actually spawn a thread for it and preempt when needed.
Also, perhaps Seastar's cooperational model can be pushed down to UDF's, at least in case of javascript - it's probably possible to create a scylla_maybe_yield() function wrapper inside the v8 context (or any other engine), and calling this wrapper from inside a UDF would simply jump back to host and call thread::maybe_yield(). That would save us from counting bytecode and all. Finally, I suspect that these javascript engines won't become friends with our stall detector anyway, given that they often try just-in-time compilation and other tricks. But perhaps with enough configuration they can become more predictable.
|
Very nice! But I have a feeling we'll need to look for more runtime candidates. |
|
We can make progress with the api and the aggregation execution
on the Scylla side while we use v8 as a temporary solution that works.
This way we have a working solution in the interim (still in experimental
of course)
…On Wed, Jul 28, 2021 at 9:58 AM Avi Kivity ***@***.***> wrote:
Very nice! But I have a feeling we'll need to look for more runtime
candidates.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#9108 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANHURPQFAF6GO4VRHBGVVLT2AZKFANCNFSM5BEPCQLA>
.
|
Makes sense. Nothing would prevent people from passing TypeScript scripts or anything else that the underneath engine is capable of compiling, but it doesn't really hurt. Also, I'm not an expert in all these *script languages, probably one is a superset of the other and nobody distinguishes them anyway. |
A random list to consider based on 15 minute research: |
|
Also, semi-related: https://github.com/ChaiScript/ChaiScript . A different language, but written with C++ integration in mind. |
Exactly what we need |
|
Indeed, also, the webassembly text format looks workable for passing via CQL: https://github.com/bytecodealliance/wasmtime-cpp/blob/main/examples/fuel.wat |
|
I'm out of fuel for the 20% project this week, but I'll try to create a wasmtime-based proof of concept next week. |
|
I see wasmtime isn't packaged in Fedora, so we'll have to curl it in install-dependencies.sh like node_exporter. |
|
Yup, I already downloaded the binary package and tried it out, seems to work fine. The interface is much more friendly than v8, and the fuel thing solves the preemption issue. I'll try to push something workable for at least a couple of basic types early next week. |
|
edit: I couldn't resist: a very limited proof-of-concept based on wasmtime is posted instead of the original v8-based implementation. It's very far from being complete, but still shows much better potential than the original engine. |
|
Hm, unless I'm mistaken, it actually looks like the |
|
It looks like fuel is only partially supported in wasmtime-cpp bindings. Here's an excerpt from the original Rust docs (https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#method.out_of_fuel_trap): ... and it looks like only the trap-based mechanism, which, quote, |
|
Also, as for types other than int32s, int64s, floats and doubles - there used to be an implementation of "interface types", which allowed integrating strings and other custom types quite seamlessly, but they got deprecated and the new implementation is not shipped yet. Refs: That would make it a little more complicated to introduce support for all CQL types. I suppose we could define that all accepted wasm functions treat CQL types as a tuple of two ints - one of which indicates the address of the serialized value, and the other its length. Then, we just don't care about serialization/deserialization, because we would push the responsibility to the user-defined function body. Doesn't sound very convenient, but languages which compile to WebAssembly (e.g. Rust) usually have CQL serialization implemented for them already anyway. |
|
Well, we can implement the missing bindings, and start with the aborting variant. Assuming 1 insn == 1 ns, 1M fuel is 1ms, plenty for a simple UDF/UDA. |
|
And we can start with numeric types too. |
psarna
force-pushed
the
wasm_udf
branch
2 times, most recently
from
301d108 to
50a3ae4
Compare
|
v2:
Still missing:
To sum up, it's still a draft, because there's no automatic way of downloading a dependency and the compilation unconditionally assumes that libwasmtime.a should be linked to Scylla, but the series is much more ready to review than it was in v1. |
Please patch install-dependencies.sh with the download+install part. Send it as a separate patch and I will regenerate the toolchain when committing it. Note we generate the toolchain for s390x (though we don't use it), so it should be prepared to the binary not existing. |
wasmtime.hh
Outdated
| * | ||
| * This project is a C++ API for | ||
| * [Wasmtime](https://github.com/bytecodealliance/wasmtime). Support for the | ||
| * C++ API is exclusively built on the [C API of |
There was a problem hiding this comment.
Maybe better to do this as part of install-dependencies.sh and install it in /usr/local/include (I guess the library should be in /usr/local/lib64 too).
wasm.cc
Outdated
| @@ -0,0 +1,186 @@ | |||
| /* | |||
| * Copyright (C) 2019-present ScyllaDB | |||
There was a problem hiding this comment.
It's taken two years, but totally worth it.
wasm.cc
Outdated
| // or drop the global in favor of a sharded service | ||
| static wasmtime::Engine& wasm_engine() { | ||
| static thread_local wasmtime::Engine engine = wasmtime::Engine(make_config()); | ||
| return engine; |
There was a problem hiding this comment.
thread_local is not that bad (also not very good, but at least doesn't stand in the way of cluster-in-a-box).
There was a problem hiding this comment.
Still, better to have a sharded service so it can be configured with non-default config, if we ever need it.
There was a problem hiding this comment.
Yup, I decided to leave it like that and quote the thread_local usage from our schema_registry.cc as my main line of defense. This engine is such a thin layer that creating a sharded service out of it and propagating it all over the source code seems a little complicated without any immediate gain - especially that we compile in two separate layers - once when the function is created, and then when it arrives to another node during schema migration.
There are some configuration options which could be useful for exposure - e.g. max wasm stack size, but on the other hand they're fairly low level and we should probably come up with sensible defaults instead.
In this iteration I decided to accept this little debt, since if anybody needs custom configuration later, I'll be the one who implements the proper sharded service anyway, so at least I only indebt myself.
Another option: extend the syntax so you can write
I think NULL still make sense. If the aggregate gives up because the input is bad in some way other than being NULL, it can return NULL and will not be called again for this aggregation. |
It would still be possible though - by using a function that is Yeah, extending the syntax is another idea to consider. The end of the day is near, so soon I'll post a version that accepts null outputs only for |
|
Sure. |
|
v6:
For reference, here's the uglified version of the example fib program, which accepts nulls and is also capable of returning nulls - which involves allocating memory in order to serialize a single bigint and return a pointer to it: struct __attribute__((packed)) nullable_bigint {
int size;
long long v;
};
static long long swap_int64(long long val) {
val = ((val << 8) & 0xFF00FF00FF00FF00ULL ) | ((val >> 8) & 0x00FF00FF00FF00FFULL );
val = ((val << 16) & 0xFFFF0000FFFF0000ULL ) | ((val >> 16) & 0x0000FFFF0000FFFFULL );
return (val << 32) | ((val >> 32) & 0xFFFFFFFFULL);
}
long long fib_aux(long long n) {
if (n < 2) {
return n;
}
return fib_aux(n-1) + fib_aux(n-2);
}
struct nullable_bigint* fib(struct nullable_bigint* p) {
// Initialize memory for the return struct
struct nullable_bigint* ret = (struct nullable_bigint*)__builtin_wasm_memory_size(0);
__builtin_wasm_memory_grow(0, sizeof(struct nullable_bigint));
ret->size = sizeof(long long);
if (p->size == -1) {
ret->v = swap_int64(42);
} else {
ret->v = swap_int64(fib_aux(swap_int64(p->v)));
}
return ret;
} |
|
Of course, the uglification above should eventually be hidden behind a wrapper library that we provide. |
psarna
changed the title
|
v7:
I also hereby remove the |
|
To be xtra safe, let's change the language code from "wasm" to "x-wasm" so it's impossible for an old-abi function to be interpreted with the new abi, if/when we settle on that. |
In order to support more languages than just Lua in the future, Lua-specific configuration is now extracted to a separate structure.
Support for more languages is comming, so let's group them in a separate directory.
Courtesy of https://github.com/bytecodealliance/wasmtime-cpp . Taken as is, with a small licensing blurb added on top.
WASM engine stores the wasm runtime engine for user-defined functions.
WASM engine needs to be used from two separate contexts: - when a user-defined function is created via CQL - when a user-defined function is received during schema migration The common instance that these two have in common is the database object, so that's where the reference is stored.
|
v8:
|
…(UDF)' from Piotr Sarna
This series adds very basic support for WebAssembly-based user-defined functions.
This series comes with a basic set of tests which were used to designate a minimal goal for this initial implementation.
Example usage:
```cql
CREATE FUNCTION ks.fibonacci (str text)
RETURNS NULL ON NULL INPUT
RETURNS boolean
LANGUAGE xwasm
AS ' (module
(func $fibonacci (param $n i32) (result i32)
(if
(i32.lt_s (local.get $n) (i32.const 2))
(return (local.get $n))
)
(i32.add
(call $fibonacci (i32.sub (local.get $n) (i32.const 1)))
(call $fibonacci (i32.sub (local.get $n) (i32.const 2)))
)
)
(export "fibonacci" (func $fibonacci))
) '
```
Note that the language is currently called "xwasm" as in "experimental wasm", because its interface is still subject to change in the future.
Closes #9108
* github.com:scylladb/scylla:
docs: add a WebAssembly entry
cql-pytest: add wasm-based tests for user-defined functions
main: add wasm engine instantiation
treewide: add initial WebAssembly support to UDF
wasm: add initial WebAssembly runtime implementation
db: add wasm_engine pointer to database
lang: add wasm_engine service
import wasmtime.hh
lua: move to lang/ directory
cql3: generalize user-defined functions for more languages
|
Dequeued. Please retest, also make sure it works without the webassembly dependencies installed. |
The engine is based on wasmtime and is able to: - compile wasm text format to bytecode - run a given compiled function with custom arguments This implementation is missing crucial features, like running on any other types than 32-bit integers. It serves as a skeleton for future full implementation.
This commit adds a very basic support for user-defined functions coded in wasm. The support is very limited (only a few types work) and was not tested against reactor stalls and performance in general.
Once the engine is up, it can be used to execute user-defined functions.
A first set of wasm-based test cases is added. The tests include verifying that supported types work and that validation of the input wasm is performed.
The doc briefly describes the state of WASM support for user-defined functions.
|
v9:
|
|
Also - retested with and without libwasmtime |
|
Congrats! |
This series adds very basic support for WebAssembly-based user-defined functions.
This series comes with a basic set of tests which were used to designate a minimal goal for this initial implementation.
Example usage:
Note that the language is currently called "xwasm" as in "experimental wasm", because its interface is still subject to change in the future.