Implement wasi-keyvalue #2486

itowlson · 2024-05-03T01:16:54Z

An exploratory PR for exploring, apropos of #2447.

itowlson · 2024-05-03T02:53:30Z

I had to add the wasi-keyvalue import to the old world to get the runtime test to compile. I assume in practice that's not what we'd aim to do though, we'd instead create a new world version and expose it only through that?

itowlson · 2024-05-05T22:00:06Z

Do folks have any insights on how to move this forward? It seems to work but I am not quite sure what do about the worlds stuff:

Should this create a rev of the Spin world?
The runtime tests appear use an old world: is there plumbing to let them use a current world?

itowlson · 2024-06-10T21:49:19Z

@dicej @lann There is a request to move this forward for the impending release or the one after. Are you the right folks to review, or should I pester someone else?

dicej · 2024-06-10T21:59:34Z

Yup, I can take a look tomorrow morning.

dicej · 2024-06-10T22:00:24Z

@kate-goldenring might also be interested, given her recent work on the specification.

rylev · 2024-06-11T13:32:01Z

crates/key-value/src/lib.rs

+    async fn open(
+        &mut self,
+        identifier: String,
+    ) -> anyhow::Result<Result<Resource<wasi_keyvalue::store::Bucket>, wasi_keyvalue::store::Error>>


You can use the trappable_error_type option to make the signatures here a bit easier to work with. See the llm implementation for an example use.

Thanks @rylev. I gave this a try and am not sure how I feel about. I've left it in a separate commit and would value your thoughts...

rylev · 2024-06-11T13:34:11Z

crates/key-value/src/lib.rs

+    ) -> anyhow::Result<Result<wasi_keyvalue::store::KeyResponse, wasi_keyvalue::store::Error>>
+    {
+        if cursor.unwrap_or_default() != 0 {
+            anyhow::bail!("list_keys: cursor not supported");


Is the plan to eventually support the cursor?

"Plan" is a strong word but, yes, it would be desirable to do on back-ends that support it. For this initial implementation I stuck to using the existing KV implementation which unfortunately do not support cursors.

rylev · 2024-06-11T13:37:22Z

wit/world.wit

@@ -16,6 +16,7 @@ world http-trigger-rc20231018 {
 world platform {
  include wasi:cli/imports@0.2.0;
  import wasi:http/outgoing-handler@0.2.0;
+  import wasi:keyvalue/store@0.2.0-draft;


We definitely need to decide how we want to handle introducing new imports. This will break any Spin runtime that does not have wasi:keyvalue support.

It seems like we should be bumping the WIT package version here right? This is still a little scary as we haven't even done it with WASI 0.2.1 yet...

Yes, this was one of my questions: #2486 (comment). Do we need a separate conversation about how to do this?

Do we need a separate conversation about how to do this?

Probably? I'm not sure that anyone knows how best to handle this yet.

dicej

LGTM overall -- thanks for doing this!

I expect we'll need to implement cursors before we can really claim to support wasi:keyvalue/store, and that may be the most interesting part of this project given the variety of backends we now support (SQLite, Redis, and Azure).

dicej · 2024-06-11T14:03:48Z

wit/world.wit

@@ -30,6 +31,7 @@ world platform {
 world platform-rc20231018 {
  include wasi:cli/reactor@0.2.0-rc-2023-10-18;
  import wasi:http/outgoing-handler@0.2.0-rc-2023-10-18;
+  import wasi:keyvalue/store@0.2.0-draft;


I'd recommend not adding anything new to platform-rc20231018 -- it's only really here for backwards compatibility. In hindsight, I think we probably shouldn't have added mqtt either.

I agree! The trouble is that the tests all (currently) use this world, so if we want tests for wasi-keyvalue then we will need to rev the test world, and I'm not sure of the implications of that. #2486 (comment)

kate-goldenring

LGTM. @itowlson should we mark this as ready to review?

wit/deps/keyvalue-2024-05-03/world.wit

itowlson · 2024-06-13T20:27:57Z

@kate-goldenring I was planning to leave it as draft until we have a decision on WIT versioning. I can mark ready for review and label "do not merge" if that would be better?

kate-goldenring · 2024-06-13T21:17:37Z

I was planning to leave it as draft until we have a decision on WIT versioning. I can mark ready for review and label "do not merge" if that would be better?

@itowlson leaving as is is fine. Thanks for clarifying the decision criteria.

radu-matei · 2024-06-24T08:36:04Z

Just to validate an assumption I had reading this PR: this PR adds support for the WASI KV proposal — specifically, it allows guests to use the proposal to interact with stores — and stores are still managed through the manifest key_value_stores field. Is that accurate?

Thanks!

itowlson · 2024-06-24T21:59:52Z

@radu-matei Yes. The KV stores are the exact same KV stores as you would get if you used the Spin APIs, specified and configured in exactly the same way.

radu-matei · 2024-08-20T07:20:19Z

Is the intention to update this PR and have WASI KV be part of the Spin world?

I am not sure what the decision criteria for WIT versioning (which I understand is blocking this PR) is.

itowlson · 2024-08-20T19:54:19Z

@radu-matei That is the intention, but we don't yet know how to version the Spin world (at least I don't, and from previous comments this seems to be a general problem, unless the community has figured something out since June).

itowlson · 2024-10-16T03:30:10Z

Rebased onto world3 (#2887). @devigned if you are working on this then please note the proposed new world and new file layout. (The last commit is how wasi-keyvalue fits in - the earlier commits are tracking the world3 proposal.) Also please note that these are not yet final...!

Signed-off-by: itowlson <ivan.towlson@fermyon.com>

itowlson force-pushed the wasi-keyvalue branch from ed2cfa3 to fa81516 Compare May 5, 2024 22:39

itowlson linked an issue May 6, 2024 that may be closed by this pull request

Support wasi-keyvalue #2447

Open

itowlson requested review from dicej and lann May 9, 2024 00:17

itowlson requested a review from kate-goldenring June 10, 2024 22:21

rylev reviewed Jun 11, 2024

View reviewed changes

dicej reviewed Jun 11, 2024

View reviewed changes

kate-goldenring reviewed Jun 13, 2024

View reviewed changes

wit/deps/keyvalue-2024-05-03/world.wit Show resolved Hide resolved

itowlson force-pushed the wasi-keyvalue branch from fa81516 to 110e234 Compare June 17, 2024 23:41

itowlson mentioned this pull request Jul 9, 2024

Add native support for Golang, Python, and JavaScript Azure SDKs #2623

Open

itowlson force-pushed the wasi-keyvalue branch from 5d8d92c to 94a8ca4 Compare August 28, 2024 03:03

itowlson force-pushed the wasi-keyvalue branch from 94a8ca4 to cc5eac0 Compare September 27, 2024 01:18

itowlson mentioned this pull request Sep 27, 2024

Support wasi-keyvalue #2447

Open

itowlson force-pushed the wasi-keyvalue branch 3 times, most recently from 514500a to 14b0cc1 Compare October 16, 2024 03:25

itowlson changed the title ~~Exploration: wasi-keyvalue~~ Implement wasi-keyvalue Oct 16, 2024

Support wasi-keyvalue for key-value store access

6d594b9

Signed-off-by: itowlson <ivan.towlson@fermyon.com>

itowlson force-pushed the wasi-keyvalue branch from 14b0cc1 to 6d594b9 Compare October 16, 2024 20:09

devigned mentioned this pull request Oct 24, 2024

WASI Key Value 0.2.0-draft2 support #2895

Merged

vdice closed this in #2895 Nov 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement wasi-keyvalue #2486

Implement wasi-keyvalue #2486

itowlson commented May 3, 2024

itowlson commented May 3, 2024

itowlson commented May 5, 2024

itowlson commented Jun 10, 2024

dicej commented Jun 10, 2024

dicej commented Jun 10, 2024

rylev Jun 11, 2024

itowlson Jun 19, 2024

rylev Jun 11, 2024

itowlson Jun 11, 2024

rylev Jun 11, 2024

lann Jun 11, 2024

itowlson Jun 11, 2024

lann Jun 11, 2024

dicej left a comment

dicej Jun 11, 2024

itowlson Jun 11, 2024

kate-goldenring left a comment

itowlson commented Jun 13, 2024

kate-goldenring commented Jun 13, 2024

radu-matei commented Jun 24, 2024

itowlson commented Jun 24, 2024

radu-matei commented Aug 20, 2024 •

edited

Loading

itowlson commented Aug 20, 2024

itowlson commented Oct 16, 2024

Implement wasi-keyvalue #2486

Implement wasi-keyvalue #2486

Conversation

itowlson commented May 3, 2024

itowlson commented May 3, 2024

itowlson commented May 5, 2024

itowlson commented Jun 10, 2024

dicej commented Jun 10, 2024

dicej commented Jun 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dicej left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kate-goldenring left a comment

Choose a reason for hiding this comment

itowlson commented Jun 13, 2024

kate-goldenring commented Jun 13, 2024

radu-matei commented Jun 24, 2024

itowlson commented Jun 24, 2024

radu-matei commented Aug 20, 2024 • edited Loading

itowlson commented Aug 20, 2024

itowlson commented Oct 16, 2024

radu-matei commented Aug 20, 2024 •

edited

Loading