From e6fac5cf85adde905f7955ace7c34a4c7d5c018a Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Thu, 30 Nov 2023 15:42:18 -0800 Subject: [PATCH 01/14] feat(*): updated comments and changed signatures of get, get-many and get-keys This commit has changed many function signatures to account for the scenario where the key does not exist. Instead of returning an error for non-existance keys, the expected behvaior should be returning a none for the client to handle. The other changes are for providing more clarity to the behavior of the APIs ine `wasi:keyvalue` interfaces. It explained what it means to have batch operations without guarantees to atomicity and what are atomic operations like CAS. It also changed the portability criteria to follow the newest WASI proposal template, and added a new goal regarding keyvalue operation performance in this proposal. Signed-off-by: Jiaxiao Zhou (Mossaka) --- .github/workflows/main.yml | 4 +- README.md | 321 ++++++------------------------------- keyvalue-handle-watch.md | 114 ++++++++++--- keyvalue.md | 114 ++++++++++--- wit/atomic.wit | 19 ++- wit/batch.wit | 72 +++++++-- wit/error.wit | 12 +- wit/handle-watch.wit | 11 +- wit/readwrite.wit | 23 ++- wit/types.wit | 2 +- wit/world.wit | 14 ++ 11 files changed, 357 insertions(+), 349 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 6fb0ad1..72454f6 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -13,10 +13,10 @@ jobs: - uses: actions/checkout@v3 - name: ensure `./wit/deps` are in sync run: | - curl -Lo 'wit-deps' https://github.com/bytecodealliance/wit-deps/releases/download/v0.3.3/wit-deps-x86_64-unknown-linux-musl + curl -Lo 'wit-deps' https://github.com/bytecodealliance/wit-deps/releases/download/v0.3.5/wit-deps-x86_64-unknown-linux-musl chmod +x wit-deps ./wit-deps lock --check - uses: WebAssembly/wit-abi-up-to-date@v16 with: worlds: 'keyvalue keyvalue-handle-watch' - wit-bindgen: '0.13.0' \ No newline at end of file + wit-bindgen: '0.15.0' \ No newline at end of file diff --git a/README.md b/README.md index 69a0f25..febe46a 100644 --- a/README.md +++ b/README.md @@ -12,22 +12,19 @@ Phase 1 - David Justice - Jiaxiao Zhou -### Phase 4 Advancement Criteria +### Portability Criteria -At least two independent production implementations. - -At least two cloud provider implementations. - -Implementations available for at least Windows, Linux & MacOS. +`wasi:keyvalue` must have at least two complete independent implementations demonstrating +embeddability in a production key-value store context. The implementations must be able to run +on at least two different operating systems. A testsuite that passes on the platforms and implementations mentioned above. ## Table of Contents - - [WASI Key-Value Store](#wasi-key-value-store) - [Current Phase](#current-phase) - [Champions](#champions) - - [Phase 4 Advancement Criteria](#phase-4-advancement-criteria) + - [Portability Criteria](#portability-criteria) - [Table of Contents](#table-of-contents) - [Introduction](#introduction) - [Goals](#goals) @@ -47,300 +44,76 @@ A testsuite that passes on the platforms and implementations mentioned above. ### Introduction -WASI Key-Value Store is a WASI API primarily for accessing a key-value datastore. It has functions to: +This `wasi:keyvalue` proposal defines a collection of [interfaces] for +interacting with key-value stores. It additionally defines a [world], +`wasi:keyvalue/keyvalue`, that groups together common interfaces including -1. retrieve the value stored and associated with a key, -2. delete a key-value pair, and -3. create and update a key-value pair. +1. CRUD operations on single key-value pairs +2. Atomic operations on single key-value pairs +3. Batch operations on multiple key-value pairs The API is designed to hide data-plane differences between different key-value stores. The control-plane behavior/functionality (e.g., including cluster management, data consistency, replication, and sharding) are not specified, and are provider specific. +[Interfaces]: https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md#wit-interfaces +[World]: https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md#wit-worlds + ### Goals -The primary goal of this API is to allow users to use WASI programs to access key-value stores. It abstracts away the complexity of building stateful services that need to access key-value stores. In other words, it decouples the application code from the specific key-value store APIs. This allows WASI programs to be portable across different key-value store that supports this interface, be it on-prem, in the cloud, or in edge devices. +The primary goal of this API is to allow WASI programs to access a wide variety of key-value stores. It meant the `wasi:keyvalue` interfaces to be implementable by a wide variety of key-value stores, including but not limited to: in-memory key-value stores, on-disk key-value stores, document databases, relational databases, and either single-node or distributed key-value stores. + +The second goal of `wasi:keyvalue` interfaces is to abstract away the network stack, allowing applications that need to access key-value stores to not worry about what network protocol is used to access the key-value store. -The second goal of this API is to abstract away the network stack, allowing applications that need to access key-value stores to not worry about what network protocol is used to access the key-value store. +A third design goal of `wasi:keyvalue` interfaces is to allow performance optimizations without compromising the portability of the API. This includes minimizing the number of copies of data, minimizing the number of requests and round-trips, and support for asynchronous operations. ### Non-goals -- Cover all application requirements for a key-value store. Instead, it focuses on the most common use cases. It allows providers to extend the API to support more use cases. -- Transactional semantics. -- Data consistency. -- Data replication. -- Data sharding. -- Cluster management. -- Monitoring +- The `wasi:keyvalue` interfaces only focus on the most common use cases for key-value stores. It does not attempt to cover all possible use cases. +- The `wasi:keyvalue` interfaces do not guarantee data consistency. Data consistency models (eventual, strong, casual etc.) are diverse and are specific to each key-value store's architecture. The implication is that components importing `wasi:keyvalue` interfaces would need to be aware of the data consistency model of the key-value store they are using. +- The `wasi:keyvalue` interfaces do not handle data replication and sharding. These are control-plane behaviors that are specific to each key-value store's architecture. Replication needs and sharding management are left to the outside of the `wasi:keyvalue` interfaces. +- No cluster management is provided. Operational aspects like scaling, node management, and cluster health are to be managed separately from the wasi:keyvalue interface `wasi:keyvalue` interfaces. +- No built-in monitoring. Monitoring of key-value store performance and health should be done using external tools and not be expected as part of the `wasi:keyvalue` interfaces. ### API walk-through -#### Use case 1 - -Imagine you have an HTTP handler that needs to persist some data to a key-value store. The handler needs to be able to retrieve, delete, and update the data in the key-value store. The following Rust and Go code shows how you can use the WASI Key-Value Store API to in the handler. +The proposal can be understood by first reading the comments of `world.wit`, then `readwrite.wit`, `atomic.wit`, `batch.wit`, `caching.wit` and finally `types.wit` -```rust -wit_bindgen::generate!("outbound-keyvalue" in "wit/world.wit"); -use types::{ open_bucket, drop_bucket, payload_consume_sync }; -use readwrite::{ get, set, exists }; -{ - let my_bucket = open_bucket("my-bucket")?; - set(my_bucket, "my-key", "my-value")?; - if exists(my_bucket, "my-key")? { - let payload = get(my_bucket, "my-key")?; - let body: Vec = payload_consume_sync(payload)?; - let body = String::from_utf8(body)?; - println!("body: {}", body); - } - drop_bucket(my_bucket); -} -``` - -```go -bucket, err := types.OpenBucket("my-bucket") -if err != nil { - panic(err) -} -defer types.DropBucket(bucket) +[`world.wit`](./wit/world.wit) +[`readwrite.wit`](./wit/readwrite.wit) +[`atomic.wit`](./wit/atomic.wit) +[`batch.wit`](./wit/batch.wit) +[`caching.wit`](./wit/caching.wit) +[`types.wit`](./wit/types.wit) -err := readwrite.Set(bucket, "my-key", "my-value") -if err != nil { - panic(err) -} +### Working with the WIT -exists, err := readwrite.Exists(bucket, "my-key") -if exists { - payload, err := readwrite.Get(bucket, "my-key") - if err != nil { - panic(err) - } - body, err := types.PayloadConsumeSync(payload) - if err != nil { - panic(err) - } - fmt.Println("body: ", string(body)) -} +Bindings can be generated from the `wit` directory via: ``` - -#### Use case 2 - -If you want to watch for changes to a key-value store, you can write a wasm component that uses the inbound-keyvalue interface this API provides. The following Rust code shows how you can use the WASI Key-Value Store API to watch for changes to a key-value store. - -```rust -wit_bindgen::generate!("inbound-keyvalue" in "wit/world.wit"); -struct Handler; - -impl inbound_keyvalue::HandleWatch for Handler { - fn on_set(&mut self, bucket: Bucket, key: Key, value: Payload) { - let payload = get(my_bucket, "my-key").unwrap(); - let body: Vec = payload_consume_sync(payload).unwrap(); - let body = String::from_utf8(body).unwrap(); - println!("bucket: {}, key: {}, value: {}", bucket, key, body); - } - fn on_delete(&mut self, bucket: Bucket, key: Key) { - println!("bucket: {}, key: {}", bucket, key); - } -} - +wit-bindgen c wit/ --world proxy ``` - -### Detailed design discussion - -```go -/// A keyvalue interface that provides simple read and write operations. -interface readwrite { - /// A keyvalue interface that provides simple read and write operations. - use types.{bucket, error, incoming-value, key, outgoing-value} - - /// Get the value associated with the key in the bucket. It returns a incoming-value - /// that can be consumed to get the value. - /// - /// If the key does not exist in the bucket, it returns an error. - get: func(bucket: bucket, key: key) -> result - - /// Set the value associated with the key in the bucket. If the key already - /// exists in the bucket, it overwrites the value. - /// - /// If the key does not exist in the bucket, it creates a new key-value pair. - /// If any other error occurs, it returns an error. - set: func(bucket: bucket, key: key, outgoing-value: outgoing-value) -> result<_, error> - - /// Delete the key-value pair associated with the key in the bucket. - /// - /// If the key does not exist in the bucket, it returns an error. - delete: func(bucket: bucket, key: key) -> result<_, error> - - /// Check if the key exists in the bucket. - /// - /// If the key does not exist in the bucket, it returns an error. - exists: func(bucket: bucket, key: key) -> result -} -/// A keyvalue interface that provides atomic operations. -interface atomic { - /// A keyvalue interface that provides atomic operations. - use types.{bucket, error, key} - - /// Atomically increment the value associated with the key in the bucket by the - /// given delta. It returns the new value. - /// - /// If the key does not exist in the bucket, it creates a new key-value pair - /// with the value set to the given delta. - /// - /// If any other error occurs, it returns an error. - increment: func(bucket: bucket, key: key, delta: u64) -> result - - /// Atomically compare and swap the value associated with the key in the bucket. - /// It returns a boolean indicating if the swap was successful. - /// - /// If the key does not exist in the bucket, it returns an error. - compare-and-swap: func(bucket: bucket, key: key, old: u64, new: u64) -> result -} - -/// A keyvalue interface that provides batch operations. -interface batch { - /// A keyvalue interface that provides batch get operations. - use types.{bucket, error, key, keys, incoming-value, outgoing-value} - - /// Get the values associated with the keys in the bucket. It returns a list of - /// incoming-values that can be consumed to get the values. - /// - /// If any of the keys do not exist in the bucket, it returns an error. - get-many: func(bucket: bucket, keys: keys) -> result, error> - - /// Get all the keys in the bucket. It returns a list of keys. - get-keys: func(bucket: bucket) -> keys - - /// Set the values associated with the keys in the bucket. If the key already - /// exists in the bucket, it overwrites the value. - /// - /// If any of the keys do not exist in the bucket, it creates a new key-value pair. - /// If any other error occurs, it returns an error. - set-many: func(bucket: bucket, key-values: list>) -> result<_, error> - - /// Delete the key-value pairs associated with the keys in the bucket. - /// - /// If any of the keys do not exist in the bucket, it skips the key. - /// If any other error occurs, it returns an error. - delete-many: func(bucket: bucket, keys: keys) -> result<_, error> -} - -// A generic keyvalue interface for WASI. -interface types { - /// A bucket is a collection of key-value pairs. Each key-value pair is stored - /// as a entry in the bucket, and the bucket itself acts as a collection of all - /// these entries. - /// - /// It is worth noting that the exact terminology for bucket in key-value stores - /// can very depending on the specific implementation. For example, - /// 1. Amazon DynamoDB calls a collection of key-value pairs a table - /// 2. Redis has hashes, sets, and sorted sets as different types of collections - /// 3. Cassandra calls a collection of key-value pairs a column family - /// 4. MongoDB calls a collection of key-value pairs a collection - /// 5. Riak calls a collection of key-value pairs a bucket - /// 6. Memcached calls a collection of key-value pairs a slab - /// 7. Azure Cosmos DB calls a collection of key-value pairs a container - /// - /// In this interface, we use the term `bucket` to refer to a collection of key-value - // Soon: switch to `resource bucket { ... }` - type bucket = u32 - drop-bucket: func(bucket: bucket) - open-bucket: func(name: string) -> result - - /// A key is a unique identifier for a value in a bucket. The key is used to - /// retrieve the value from the bucket. - type key = string - - /// A list of keys - type keys = list - - use wasi:io/streams.{input-stream, output-stream} - use wasi-cloud-error.{ error } - /// A value is the data stored in a key-value pair. The value can be of any type - /// that can be represented in a byte array. It provides a way to write the value - /// to the output-stream defined in the `wasi-io` interface. - // Soon: switch to `resource value { ... }` - type outgoing-value = u32 - drop-outgoing-value: func(outgoing-value: outgoing-value) - new-outgoing-value: func() -> outgoing-value - outgoing-value-write-body: func(outgoing-value: outgoing-value) -> result - - /// A incoming-value is a wrapper around a value. It provides a way to read the value - /// from the input-stream defined in the `wasi-io` interface. - /// - /// The incoming-value provides two ways to consume the value: - /// 1. `incoming-value-consume-sync` consumes the value synchronously and returns the - /// value as a list of bytes. - /// 2. `incoming-value-consume-async` consumes the value asynchronously and returns the - /// value as an input-stream. - // Soon: switch to `resource incoming-value { ... }` - type incoming-value = u32 - type incoming-value-async-body = input-stream - type incoming-value-sync-body = list - drop-incoming-value: func(incoming-value: incoming-value) - incoming-value-consume-sync: func(incoming-value: incoming-value) -> result - incoming-value-consume-async: func(incoming-value: incoming-value) -> result - size: func(incoming-value: incoming-value) -> u64 -} +and can be validated and otherwise manipulated via: ``` - -- The `get-many` and `set-many` are atomic. -- The `increment` is atomic, in a way that it is a small transaction of get, increment, and set operations on the same key. - -The following interfaces are still under discussion: - -```go -interface transaction { - // transaction is an atomic operation that groups multiple operations together. - // If any operation fails, all operations in the transaction are rolled back. - use types.{ Error, payload, key } - - get-multi: func(keys: keys) -> result, Error> - - set-multi: func(key_values: list<(key, list)>) -> result<_, Error> -} - -interface ttl { - use types.{ Error, key } - - set-with-ttl: func(key: key, value: list, ttl: u64) -> result<_, Error> -} - -interface "wasi:kv/data/query" { - ... -} +wasm-tools component wit wit/ ... ``` -#### Tricky design choice 1 - -TODO - -#### Tricky design choice 2 - -TODO - -### Considered alternatives - -TODO - -#### Alternative 1 - -TODO -#### Alternative 2 - -TODO - -### Stakeholder Interest & Feedback - -TODO before entering Phase 3. +The `wit/deps` directory contains a live snapshot of the contents of several +other WASI proposals upon which this proposal depends. It is automatically +updated by running [`wit-deps update`](https://crates.io/crates/wit-deps-cli) +in the root directory, which fetches the live contents of the `main` branch of +each proposal. As things stabilize, `wit/deps.toml` will be updated to refer to +versioned releases. ### References & acknowledgements -Many thanks for valuable feedback and advice from: +This proposal was inspired by Dapr's [State API] -- [Person 1] -- [Person 2] -- [etc.] +[State API](https://docs.dapr.io/developing-applications/building-blocks/state-management/) ### Change log +- 2023-11-30: + - Changed the `get` and `get-many` and `get-keys` signatures + - Updated comments in all the interfaces. + - Renamed `wasi-cloud-error` to `wasi-keyvalue-error` - 2023-05-17: Updated batch example to use one interface instead of 2 - 2023-05-25: Change the WITs to the newest syntax. - 2023-02-13: The following changes were made to the API: diff --git a/keyvalue-handle-watch.md b/keyvalue-handle-watch.md index 6393a98..7c714b3 100644 --- a/keyvalue-handle-watch.md +++ b/keyvalue-handle-watch.md @@ -5,7 +5,7 @@
  • interface wasi:io/error@0.2.0-rc-2023-11-10
  • interface wasi:io/poll@0.2.0-rc-2023-11-10
  • interface wasi:io/streams@0.2.0-rc-2023-11-10
    • -
  • interface wasi:keyvalue/wasi-cloud-error
    • +
  • interface wasi:keyvalue/wasi-keyvalue-error
  • interface wasi:keyvalue/types
  • interface wasi:keyvalue/readwrite
  • interface wasi:keyvalue/atomic
    • @@ -419,12 +419,22 @@ is ready for reading, before performing the splice.

    -

    Import interface wasi:keyvalue/wasi-cloud-error

    +

    Import interface wasi:keyvalue/wasi-keyvalue-error

    Types

    resource error

    -

    An error resource type for keyvalue operations. -Currently, this provides only one function to return a string representation +

    An error resource type for keyvalue operations.

    +

    Common errors:

    +
      +
    • Connectivity errors (e.g. network errors): when the client cannot establish +a connection to the keyvalue service.
      • +
    • Autnetication and Authorization errors: when the client fails to authenticate +or does not have the required permissions to perform the operation.
      • +
    • Data errors: when the client sends incompatible or corrupted data.
      • +
    • Resource errors: when the system runs out of resources (e.g. memory).
      • +
    • Internal errors: unexpected errors on the server side.
      • +
    +

    Currently, this provides only one function to return a string representation of the error. In the future, this will be extended to provide more information about the error.

    Functions

    @@ -582,9 +592,11 @@ value as an input-stream. ----

    Functions

    get: func

    -

    Get the value associated with the key in the bucket. It returns a incoming-value -that can be consumed to get the value.

    -

    If the key does not exist in the bucket, it returns an error.

    +

    Get the value associated with the key in the bucket.

    +

    The value is returned as an option. If the key-value pair exists in the +bucket, it returns ok with the value. If the key does not exist in the +bucket, it returns ok with none.

    +

    If any other error occurs, it returns an error.

    Params
    • bucket: borrow<bucket>
      • @@ -592,13 +604,13 @@ that can be consumed to get the value.

    Return values

    set: func

    Set the value associated with the key in the bucket. If the key already exists in the bucket, it overwrites the value.

    -

    If the key does not exist in the bucket, it creates a new key-value pair. -If any other error occurs, it returns an error.

    +

    If the key does not exist in the bucket, it creates a new key-value pair.

    +

    If any other error occurs, it returns an error.

    Params
    • bucket: borrow<bucket>
      • @@ -611,7 +623,8 @@ If any other error occurs, it returns an error.

    delete: func

    Delete the key-value pair associated with the key in the bucket.

    -

    If the key does not exist in the bucket, it returns an error.

    +

    If the key does not exist in the bucket, it does nothing.

    +

    If any other error occurs, it returns an error.

    Params
    • bucket: borrow<bucket>
      • @@ -623,6 +636,9 @@ If any other error occurs, it returns an error.

    exists: func

    Check if the key exists in the bucket.

    +

    If the key exists in the bucket, it returns ok with true. If the key does +not exist in the bucket, it returns ok with false.

    +

    If any other error occurs, it returns an error.

    Params
    • bucket: borrow<bucket>
      • @@ -634,6 +650,10 @@ If any other error occurs, it returns an error.

    Import interface wasi:keyvalue/atomic

    A keyvalue interface that provides atomic operations.

    +

    Atomic operations are single, indivisible operations. When a fault causes +an atomic operation to fail, it will appear to the invoker of the atomic +operation that the action either completed successfully or did nothing +at all.

    Types

    type bucket

    @@ -649,7 +669,7 @@ If any other error occurs, it returns an error.

    Functions

    increment: func

    Atomically increment the value associated with the key in the bucket by the -given delta. It returns the new value.

    +given delta. It returns the new value. This is single, indivisible operation.

    If the key does not exist in the bucket, it creates a new key-value pair with the value set to the given delta.

    If any other error occurs, it returns an error.

    @@ -664,9 +684,13 @@ with the value set to the given delta.

  • result<u64, own<error>>

    • compare-and-swap: func

    -

    Atomically compare and swap the value associated with the key in the bucket. -It returns a boolean indicating if the swap was successful.

    -

    If the key does not exist in the bucket, it returns an error.

    +

    Compare-and-swap (CAS) atomically updates the value associated with the key +in the bucket if the value matches the old value. This operation returns +Ok(true) if the swap was successful, Ok(false) if the value did not match,

    +

    A successful CAS operation means the current value matched the old value +and was replaced with the new value.

    +

    If the key does not exist in the bucket, it returns Ok(false).

    +

    If any other error occurs, it returns an error.

    Params
    • bucket: borrow<bucket>
      • @@ -680,6 +704,22 @@ It returns a boolean indicating if the swap was successful.

    Import interface wasi:keyvalue/batch

    A keyvalue interface that provides batch operations.

    +

    A batch operation is an operation that operates on multiple keys at once.

    +

    Batch operations are useful for reducing network round-trip time. For example, +if you want to get the values associated with 100 keys, you can either do 100 get +operations or you can do 1 batch get operation. The batch operation is +faster because it only needs to make 1 network call instead of 100.

    +

    A batch operation does not guarantee atomicity, meaning that if the batch +operation fails, some of the keys may have been modified and some may not. +Transactional operations are being worked on and will be added in the future to +provide atomicity.

    +

    Data consistency in a key value store refers to the gaurantee that once a +write operation completes, all subsequent read operations will return the +value that was written.

    +

    The level of consistency in batch operations can vary depending on the +implementation. This interface does not guarantee strong consistency, meaning +that if a write operation completes, subsequent read operations may not return +the value that was written.

    Types

    type bucket

    @@ -704,8 +744,12 @@ It returns a boolean indicating if the swap was successful.

    Functions

    get-many: func

    Get the values associated with the keys in the bucket. It returns a list of -incoming-values that can be consumed to get the values.

    -

    If any of the keys do not exist in the bucket, it returns an error.

    +incoming-value that can be consumed to get the value associated with the key.

    +

    If any of the keys do not exist in the bucket, it returns an None value for +that key in the list.

    +

    Note that the key-value pairs are guaranteed to be returned in the same order

    +

    MAY show an out-of-date value if there are concurrent writes to the bucket.

    +

    If any other error occurs, it returns an error.

    Params
    • bucket: borrow<bucket>
      • @@ -713,23 +757,33 @@ incoming-values that can be consumed to get the values.

    Return values

    get-keys: func

    Get all the keys in the bucket. It returns a list of keys.

    +

    Note that the keys are not guaranteed to be returned in any particular order.

    +

    If the bucket is empty, it returns an empty list.

    +

    MAY show an out-of-date list of keys if there are concurrent writes to the bucket.

    +

    If any error occurs, it returns an error.

    Params
    Return values

    set-many: func

    Set the values associated with the keys in the bucket. If the key already exists in the bucket, it overwrites the value.

    -

    If any of the keys do not exist in the bucket, it creates a new key-value pair. -If any other error occurs, it returns an error.

    +

    Note that the key-value pairs are not guaranteed to be set in the order +they are provided.

    +

    If any of the keys do not exist in the bucket, it creates a new key-value pair.

    +

    If any other error occurs, it returns an error. When an error occurs, it +does not rollback the key-value pairs that were already set. Thus, this batch operation +does not guarantee atomicity, implying that some key-value pairs could be +set while others might fail.

    +

    Other concurrent operations may also be able to see the partial results.

    Params
    • bucket: borrow<bucket>
      • @@ -741,8 +795,14 @@ If any other error occurs, it returns an error.

    delete-many: func

    Delete the key-value pairs associated with the keys in the bucket.

    -

    If any of the keys do not exist in the bucket, it skips the key. -If any other error occurs, it returns an error.

    +

    Note that the key-value pairs are not guaranteed to be deleted in the order +they are provided.

    +

    If any of the keys do not exist in the bucket, it skips the key.

    +

    If any other error occurs, it returns an error. When an error occurs, it +does not rollback the key-value pairs that were already deleted. Thus, this batch operation +does not guarantee atomicity, implying that some key-value pairs could be +deleted while others might fail.

    +

    Other concurrent operations may also be able to see the partial results.

    Params
    • bucket: borrow<bucket>
      • @@ -767,8 +827,9 @@ If any other error occurs, it returns an error.

      ----

      Functions

      on-set: func

      -

      Handle the set event for the given bucket and key. -It returns a incoming-value that can be consumed to get the value.

      +

      Handle the set event for the given bucket and key. +It returns a incoming-value that represents the new value being set. +The new value can be consumed by the handler.

      Params

      on-delete: func

      -

      Handle the delete event for the given bucket and key.

      +

      Handle the delete event for the given bucket and key. +It returns a key that represents the key being deleted.

      Params
      • bucket: own<bucket>
        • diff --git a/keyvalue.md b/keyvalue.md index d6a8b06..b5344c6 100644 --- a/keyvalue.md +++ b/keyvalue.md @@ -1,11 +1,19 @@

        World keyvalue

        +

        The wasi:keyvalue/keyvalue world provides common APIs for interacting +with key-value stores. Components targeting this world will be able to +do

        +
          +
        1. CRUD (create, read, update, delete) operations on key-value stores.
          2. +
        2. Atomic increment and CAS (compare-and-swap) operations.
          3. +
        3. Batch operations that can reduce the number of round trips to the network.
          4. +
        • Imports:
          • interface wasi:io/error@0.2.0-rc-2023-11-10
          • interface wasi:io/poll@0.2.0-rc-2023-11-10
          • interface wasi:io/streams@0.2.0-rc-2023-11-10
            • -
          • interface wasi:keyvalue/wasi-cloud-error
            • +
          • interface wasi:keyvalue/wasi-keyvalue-error
          • interface wasi:keyvalue/types
          • interface wasi:keyvalue/readwrite
          • interface wasi:keyvalue/atomic
            • @@ -414,12 +422,22 @@ is ready for reading, before performing the splice.

            -

            Import interface wasi:keyvalue/wasi-cloud-error

            +

            Import interface wasi:keyvalue/wasi-keyvalue-error

            Types

            resource error

            -

            An error resource type for keyvalue operations. -Currently, this provides only one function to return a string representation +

            An error resource type for keyvalue operations.

            +

            Common errors:

            +
              +
            • Connectivity errors (e.g. network errors): when the client cannot establish +a connection to the keyvalue service.
              • +
            • Autnetication and Authorization errors: when the client fails to authenticate +or does not have the required permissions to perform the operation.
              • +
            • Data errors: when the client sends incompatible or corrupted data.
              • +
            • Resource errors: when the system runs out of resources (e.g. memory).
              • +
            • Internal errors: unexpected errors on the server side.
              • +
            +

            Currently, this provides only one function to return a string representation of the error. In the future, this will be extended to provide more information about the error.

            Functions

            @@ -577,9 +595,11 @@ value as an input-stream. ----

            Functions

            get: func

            -

            Get the value associated with the key in the bucket. It returns a incoming-value -that can be consumed to get the value.

            -

            If the key does not exist in the bucket, it returns an error.

            +

            Get the value associated with the key in the bucket.

            +

            The value is returned as an option. If the key-value pair exists in the +bucket, it returns ok with the value. If the key does not exist in the +bucket, it returns ok with none.

            +

            If any other error occurs, it returns an error.

            Params
            • bucket: borrow<bucket>
              • @@ -587,13 +607,13 @@ that can be consumed to get the value.

            Return values

            set: func

            Set the value associated with the key in the bucket. If the key already exists in the bucket, it overwrites the value.

            -

            If the key does not exist in the bucket, it creates a new key-value pair. -If any other error occurs, it returns an error.

            +

            If the key does not exist in the bucket, it creates a new key-value pair.

            +

            If any other error occurs, it returns an error.

            Params
            • bucket: borrow<bucket>
              • @@ -606,7 +626,8 @@ If any other error occurs, it returns an error.

            delete: func

            Delete the key-value pair associated with the key in the bucket.

            -

            If the key does not exist in the bucket, it returns an error.

            +

            If the key does not exist in the bucket, it does nothing.

            +

            If any other error occurs, it returns an error.

            Params
            • bucket: borrow<bucket>
              • @@ -618,6 +639,9 @@ If any other error occurs, it returns an error.

            exists: func

            Check if the key exists in the bucket.

            +

            If the key exists in the bucket, it returns ok with true. If the key does +not exist in the bucket, it returns ok with false.

            +

            If any other error occurs, it returns an error.

            Params
            • bucket: borrow<bucket>
              • @@ -629,6 +653,10 @@ If any other error occurs, it returns an error.

            Import interface wasi:keyvalue/atomic

            A keyvalue interface that provides atomic operations.

            +

            Atomic operations are single, indivisible operations. When a fault causes +an atomic operation to fail, it will appear to the invoker of the atomic +operation that the action either completed successfully or did nothing +at all.

            Types

            type bucket

            @@ -644,7 +672,7 @@ If any other error occurs, it returns an error.

            Functions

            increment: func

            Atomically increment the value associated with the key in the bucket by the -given delta. It returns the new value.

            +given delta. It returns the new value. This is single, indivisible operation.

            If the key does not exist in the bucket, it creates a new key-value pair with the value set to the given delta.

            If any other error occurs, it returns an error.

            @@ -659,9 +687,13 @@ with the value set to the given delta.

          • result<u64, own<error>>

          compare-and-swap: func

          -

          Atomically compare and swap the value associated with the key in the bucket. -It returns a boolean indicating if the swap was successful.

          -

          If the key does not exist in the bucket, it returns an error.

          +

          Compare-and-swap (CAS) atomically updates the value associated with the key +in the bucket if the value matches the old value. This operation returns +Ok(true) if the swap was successful, Ok(false) if the value did not match,

          +

          A successful CAS operation means the current value matched the old value +and was replaced with the new value.

          +

          If the key does not exist in the bucket, it returns Ok(false).

          +

          If any other error occurs, it returns an error.

          Params
          • bucket: borrow<bucket>
            • @@ -675,6 +707,22 @@ It returns a boolean indicating if the swap was successful.

          Import interface wasi:keyvalue/batch

          A keyvalue interface that provides batch operations.

          +

          A batch operation is an operation that operates on multiple keys at once.

          +

          Batch operations are useful for reducing network round-trip time. For example, +if you want to get the values associated with 100 keys, you can either do 100 get +operations or you can do 1 batch get operation. The batch operation is +faster because it only needs to make 1 network call instead of 100.

          +

          A batch operation does not guarantee atomicity, meaning that if the batch +operation fails, some of the keys may have been modified and some may not. +Transactional operations are being worked on and will be added in the future to +provide atomicity.

          +

          Data consistency in a key value store refers to the gaurantee that once a +write operation completes, all subsequent read operations will return the +value that was written.

          +

          The level of consistency in batch operations can vary depending on the +implementation. This interface does not guarantee strong consistency, meaning +that if a write operation completes, subsequent read operations may not return +the value that was written.

          Types

          type bucket

          @@ -699,8 +747,12 @@ It returns a boolean indicating if the swap was successful.

          Functions

          get-many: func

          Get the values associated with the keys in the bucket. It returns a list of -incoming-values that can be consumed to get the values.

          -

          If any of the keys do not exist in the bucket, it returns an error.

          +incoming-value that can be consumed to get the value associated with the key.

          +

          If any of the keys do not exist in the bucket, it returns an None value for +that key in the list.

          +

          Note that the key-value pairs are guaranteed to be returned in the same order

          +

          MAY show an out-of-date value if there are concurrent writes to the bucket.

          +

          If any other error occurs, it returns an error.

          Params
          • bucket: borrow<bucket>
            • @@ -708,23 +760,33 @@ incoming-values that can be consumed to get the values.

          Return values

          get-keys: func

          Get all the keys in the bucket. It returns a list of keys.

          +

          Note that the keys are not guaranteed to be returned in any particular order.

          +

          If the bucket is empty, it returns an empty list.

          +

          MAY show an out-of-date list of keys if there are concurrent writes to the bucket.

          +

          If any error occurs, it returns an error.

          Params
          Return values

          set-many: func

          Set the values associated with the keys in the bucket. If the key already exists in the bucket, it overwrites the value.

          -

          If any of the keys do not exist in the bucket, it creates a new key-value pair. -If any other error occurs, it returns an error.

          +

          Note that the key-value pairs are not guaranteed to be set in the order +they are provided.

          +

          If any of the keys do not exist in the bucket, it creates a new key-value pair.

          +

          If any other error occurs, it returns an error. When an error occurs, it +does not rollback the key-value pairs that were already set. Thus, this batch operation +does not guarantee atomicity, implying that some key-value pairs could be +set while others might fail.

          +

          Other concurrent operations may also be able to see the partial results.

          Params
          • bucket: borrow<bucket>
            • @@ -736,8 +798,14 @@ If any other error occurs, it returns an error.

          delete-many: func

          Delete the key-value pairs associated with the keys in the bucket.

          -

          If any of the keys do not exist in the bucket, it skips the key. -If any other error occurs, it returns an error.

          +

          Note that the key-value pairs are not guaranteed to be deleted in the order +they are provided.

          +

          If any of the keys do not exist in the bucket, it skips the key.

          +

          If any other error occurs, it returns an error. When an error occurs, it +does not rollback the key-value pairs that were already deleted. Thus, this batch operation +does not guarantee atomicity, implying that some key-value pairs could be +deleted while others might fail.

          +

          Other concurrent operations may also be able to see the partial results.

          Params
          • bucket: borrow<bucket>
            • diff --git a/wit/atomic.wit b/wit/atomic.wit index 1819219..9dfd304 100644 --- a/wit/atomic.wit +++ b/wit/atomic.wit @@ -1,10 +1,15 @@ /// A keyvalue interface that provides atomic operations. +/// +/// Atomic operations are single, indivisible operations. When a fault causes +/// an atomic operation to fail, it will appear to the invoker of the atomic +/// operation that the action either completed successfully or did nothing +/// at all. interface atomic { /// A keyvalue interface that provides atomic operations. use types.{bucket, error, key}; /// Atomically increment the value associated with the key in the bucket by the - /// given delta. It returns the new value. + /// given delta. It returns the new value. This is single, indivisible operation. /// /// If the key does not exist in the bucket, it creates a new key-value pair /// with the value set to the given delta. @@ -12,9 +17,15 @@ interface atomic { /// If any other error occurs, it returns an error. increment: func(bucket: borrow, key: key, delta: u64) -> result; - /// Atomically compare and swap the value associated with the key in the bucket. - /// It returns a boolean indicating if the swap was successful. + /// Compare-and-swap (CAS) atomically updates the value associated with the key + /// in the bucket if the value matches the old value. This operation returns + /// Ok(true) if the swap was successful, Ok(false) if the value did not match, + /// + /// A successful CAS operation means the current value matched the `old` value + /// and was replaced with the `new` value. /// - /// If the key does not exist in the bucket, it returns an error. + /// If the key does not exist in the bucket, it returns Ok(false). + /// + /// If any other error occurs, it returns an error. compare-and-swap: func(bucket: borrow, key: key, old: u64, new: u64) -> result; } \ No newline at end of file diff --git a/wit/batch.wit b/wit/batch.wit index 258009a..99d8fe8 100644 --- a/wit/batch.wit +++ b/wit/batch.wit @@ -1,27 +1,81 @@ /// A keyvalue interface that provides batch operations. +/// +/// A batch operation is an operation that operates on multiple keys at once. +/// +/// Batch operations are useful for reducing network round-trip time. For example, +/// if you want to get the values associated with 100 keys, you can either do 100 get +/// operations or you can do 1 batch get operation. The batch operation is +/// faster because it only needs to make 1 network call instead of 100. +/// +/// A batch operation does not guarantee atomicity, meaning that if the batch +/// operation fails, some of the keys may have been modified and some may not. +/// Transactional operations are being worked on and will be added in the future to +/// provide atomicity. +/// +/// Data consistency in a key value store refers to the gaurantee that once a +/// write operation completes, all subsequent read operations will return the +/// value that was written. +/// +/// The level of consistency in batch operations can vary depending on the +/// implementation. This interface does not guarantee strong consistency, meaning +/// that if a write operation completes, subsequent read operations may not return +/// the value that was written. interface batch { /// A keyvalue interface that provides batch get operations. use types.{bucket, error, key, keys, incoming-value, outgoing-value}; /// Get the values associated with the keys in the bucket. It returns a list of - /// incoming-values that can be consumed to get the values. + /// incoming-value that can be consumed to get the value associated with the key. /// - /// If any of the keys do not exist in the bucket, it returns an error. - get-many: func(bucket: borrow, keys: keys) -> result, error>; + /// If any of the keys do not exist in the bucket, it returns an None value for + /// that key in the list. + /// + /// Note that the key-value pairs are guaranteed to be returned in the same order + /// + /// MAY show an out-of-date value if there are concurrent writes to the bucket. + /// + /// If any other error occurs, it returns an error. + get-many: func(bucket: borrow, keys: keys) -> result>, error>; /// Get all the keys in the bucket. It returns a list of keys. - get-keys: func(bucket: borrow) -> keys; + /// + /// Note that the keys are not guaranteed to be returned in any particular order. + /// + /// If the bucket is empty, it returns an empty list. + /// + /// MAY show an out-of-date list of keys if there are concurrent writes to the bucket. + /// + /// If any error occurs, it returns an error. + get-keys: func(bucket: borrow) -> result; /// Set the values associated with the keys in the bucket. If the key already - /// exists in the bucket, it overwrites the value. + /// exists in the bucket, it overwrites the value. + /// + /// Note that the key-value pairs are not guaranteed to be set in the order + /// they are provided. /// /// If any of the keys do not exist in the bucket, it creates a new key-value pair. - /// If any other error occurs, it returns an error. + /// + /// If any other error occurs, it returns an error. When an error occurs, it + /// does not rollback the key-value pairs that were already set. Thus, this batch operation + /// does not guarantee atomicity, implying that some key-value pairs could be + /// set while others might fail. + /// + /// Other concurrent operations may also be able to see the partial results. set-many: func(bucket: borrow, key-values: list>>) -> result<_, error>; /// Delete the key-value pairs associated with the keys in the bucket. - /// + /// + /// Note that the key-value pairs are not guaranteed to be deleted in the order + /// they are provided. + /// /// If any of the keys do not exist in the bucket, it skips the key. - /// If any other error occurs, it returns an error. + /// + /// If any other error occurs, it returns an error. When an error occurs, it + /// does not rollback the key-value pairs that were already deleted. Thus, this batch operation + /// does not guarantee atomicity, implying that some key-value pairs could be + /// deleted while others might fail. + /// + /// Other concurrent operations may also be able to see the partial results. delete-many: func(bucket: borrow, keys: keys) -> result<_, error>; -} \ No newline at end of file +} diff --git a/wit/error.wit b/wit/error.wit index 79be85c..1caba7e 100644 --- a/wit/error.wit +++ b/wit/error.wit @@ -1,5 +1,15 @@ -interface wasi-cloud-error { +interface wasi-keyvalue-error { /// An error resource type for keyvalue operations. + /// + /// Common errors: + /// - Connectivity errors (e.g. network errors): when the client cannot establish + /// a connection to the keyvalue service. + /// - Autnetication and Authorization errors: when the client fails to authenticate + /// or does not have the required permissions to perform the operation. + /// - Data errors: when the client sends incompatible or corrupted data. + /// - Resource errors: when the system runs out of resources (e.g. memory). + /// - Internal errors: unexpected errors on the server side. + /// /// Currently, this provides only one function to return a string representation /// of the error. In the future, this will be extended to provide more information /// about the error. diff --git a/wit/handle-watch.wit b/wit/handle-watch.wit index 082a1a2..0ca7b37 100644 --- a/wit/handle-watch.wit +++ b/wit/handle-watch.wit @@ -1,12 +1,17 @@ /// A keyvalue interface that provides handle-watch operations. +/// +/// This interface is used to provide event-driven mechanisms to handle +/// keyvalue changes. interface handle-watch { /// A keyvalue interface that provides handle-watch operations. use types.{bucket, key, incoming-value}; - /// Handle the set event for the given bucket and key. - /// It returns a incoming-value that can be consumed to get the value. + /// Handle the `set` event for the given bucket and key. + /// It returns a `incoming-value` that represents the new value being set. + /// The new value can be consumed by the handler. on-set: func(bucket: bucket, key: key, incoming-value: borrow); - /// Handle the delete event for the given bucket and key. + /// Handle the `delete` event for the given bucket and key. + /// It returns a `key` that represents the key being deleted. on-delete: func(bucket: bucket, key: key); } \ No newline at end of file diff --git a/wit/readwrite.wit b/wit/readwrite.wit index 14b6f66..637cde9 100644 --- a/wit/readwrite.wit +++ b/wit/readwrite.wit @@ -3,24 +3,35 @@ interface readwrite { /// A keyvalue interface that provides simple read and write operations. use types.{bucket, error, incoming-value, key, outgoing-value}; - /// Get the value associated with the key in the bucket. It returns a incoming-value - /// that can be consumed to get the value. + /// Get the value associated with the key in the bucket. + /// + /// The value is returned as an option. If the key-value pair exists in the + /// bucket, it returns ok with the value. If the key does not exist in the + /// bucket, it returns ok with none. /// - /// If the key does not exist in the bucket, it returns an error. - get: func(bucket: borrow, key: key) -> result; + /// If any other error occurs, it returns an error. + get: func(bucket: borrow, key: key) -> result, error>; /// Set the value associated with the key in the bucket. If the key already /// exists in the bucket, it overwrites the value. /// /// If the key does not exist in the bucket, it creates a new key-value pair. + /// /// If any other error occurs, it returns an error. set: func(bucket: borrow, key: key, outgoing-value: borrow) -> result<_, error>; /// Delete the key-value pair associated with the key in the bucket. + /// + /// If the key does not exist in the bucket, it does nothing. /// - /// If the key does not exist in the bucket, it returns an error. + /// If any other error occurs, it returns an error. delete: func(bucket: borrow, key: key) -> result<_, error>; /// Check if the key exists in the bucket. + /// + /// If the key exists in the bucket, it returns ok with true. If the key does + /// not exist in the bucket, it returns ok with false. + /// + /// If any other error occurs, it returns an error. exists: func(bucket: borrow, key: key) -> result; -} \ No newline at end of file +} diff --git a/wit/types.wit b/wit/types.wit index a89d69a..105bdb2 100644 --- a/wit/types.wit +++ b/wit/types.wit @@ -27,7 +27,7 @@ interface types { type keys = list; use wasi:io/streams@0.2.0-rc-2023-11-10.{input-stream, output-stream}; - use wasi-cloud-error.{ error }; + use wasi-keyvalue-error.{ error }; /// A value is the data stored in a key-value pair. The value can be of any type /// that can be represented in a byte array. It provides a way to write the value /// to the output-stream defined in the `wasi-io` interface. diff --git a/wit/world.wit b/wit/world.wit index 5e0e025..d0d1d5e 100644 --- a/wit/world.wit +++ b/wit/world.wit @@ -1,8 +1,22 @@ package wasi:keyvalue; +/// The `wasi:keyvalue/keyvalue` world provides common APIs for interacting +/// with key-value stores. Components targeting this world will be able to +/// do +/// 1. CRUD (create, read, update, delete) operations on key-value stores. +/// 2. Atomic `increment` and CAS (compare-and-swap) operations. +/// 3. Batch operations that can reduce the number of round trips to the network. world keyvalue { + /// The `readwrite` capability allows the component to perform CRUD + /// operations on the key-value store. import readwrite; + + /// The `atomic` capability allows the component to perform atomic + /// `increment` and CAS (compare-and-swap) operations. import atomic; + + /// The `batch` capability allows the component to perform batch + /// operations that can reduce the number of round trips to the network. import batch; } From c224a28d55e35e68d5a7dbe84e18a19a93a02db8 Mon Sep 17 00:00:00 2001 From: Jiaxiao Zhou Date: Fri, 1 Dec 2023 11:15:25 -0800 Subject: [PATCH 02/14] Apply suggestions from code review Co-authored-by: David Justice --- keyvalue-handle-watch.md | 2 +- keyvalue.md | 2 +- wit/atomic.wit | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/keyvalue-handle-watch.md b/keyvalue-handle-watch.md index 7c714b3..6dc67f4 100644 --- a/keyvalue-handle-watch.md +++ b/keyvalue-handle-watch.md @@ -669,7 +669,7 @@ at all.

            Functions

            increment: func

            Atomically increment the value associated with the key in the bucket by the -given delta. It returns the new value. This is single, indivisible operation.

            +given delta. It returns the new value. This is a single, indivisible operation.

            If the key does not exist in the bucket, it creates a new key-value pair with the value set to the given delta.

            If any other error occurs, it returns an error.

            diff --git a/keyvalue.md b/keyvalue.md index b5344c6..6b58e3f 100644 --- a/keyvalue.md +++ b/keyvalue.md @@ -672,7 +672,7 @@ at all.

            Functions

            increment: func

            Atomically increment the value associated with the key in the bucket by the -given delta. It returns the new value. This is single, indivisible operation.

            +given delta. It returns the new value. This is a single, indivisible operation.

            If the key does not exist in the bucket, it creates a new key-value pair with the value set to the given delta.

            If any other error occurs, it returns an error.

            diff --git a/wit/atomic.wit b/wit/atomic.wit index 9dfd304..e6346a9 100644 --- a/wit/atomic.wit +++ b/wit/atomic.wit @@ -9,7 +9,7 @@ interface atomic { use types.{bucket, error, key}; /// Atomically increment the value associated with the key in the bucket by the - /// given delta. It returns the new value. This is single, indivisible operation. + /// given delta. It returns the new value. This is a single, indivisible operation. /// /// If the key does not exist in the bucket, it creates a new key-value pair /// with the value set to the given delta. From 7f80a9b551047677009b470e57a83777a62158d9 Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:17:35 -0800 Subject: [PATCH 03/14] minor change to use Ok(...) throughout the proposals Signed-off-by: Jiaxiao Zhou (Mossaka) --- wit/atomic.wit | 8 ++++---- wit/batch.wit | 10 +++++----- wit/readwrite.wit | 16 ++++++++-------- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/wit/atomic.wit b/wit/atomic.wit index e6346a9..2c97e72 100644 --- a/wit/atomic.wit +++ b/wit/atomic.wit @@ -14,18 +14,18 @@ interface atomic { /// If the key does not exist in the bucket, it creates a new key-value pair /// with the value set to the given delta. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. increment: func(bucket: borrow, key: key, delta: u64) -> result; /// Compare-and-swap (CAS) atomically updates the value associated with the key /// in the bucket if the value matches the old value. This operation returns - /// Ok(true) if the swap was successful, Ok(false) if the value did not match, + /// `Ok(true)` if the swap was successful, `Ok(false)` if the value did not match, /// /// A successful CAS operation means the current value matched the `old` value /// and was replaced with the `new` value. /// - /// If the key does not exist in the bucket, it returns Ok(false). + /// If the key does not exist in the bucket, it returns `Ok(false)`. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. compare-and-swap: func(bucket: borrow, key: key, old: u64, new: u64) -> result; } \ No newline at end of file diff --git a/wit/batch.wit b/wit/batch.wit index 99d8fe8..1884d97 100644 --- a/wit/batch.wit +++ b/wit/batch.wit @@ -27,14 +27,14 @@ interface batch { /// Get the values associated with the keys in the bucket. It returns a list of /// incoming-value that can be consumed to get the value associated with the key. /// - /// If any of the keys do not exist in the bucket, it returns an None value for + /// If any of the keys do not exist in the bucket, it returns a `none` value for /// that key in the list. /// /// Note that the key-value pairs are guaranteed to be returned in the same order /// /// MAY show an out-of-date value if there are concurrent writes to the bucket. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. get-many: func(bucket: borrow, keys: keys) -> result>, error>; /// Get all the keys in the bucket. It returns a list of keys. @@ -45,7 +45,7 @@ interface batch { /// /// MAY show an out-of-date list of keys if there are concurrent writes to the bucket. /// - /// If any error occurs, it returns an error. + /// If any error occurs, it returns an `Err(error)`. get-keys: func(bucket: borrow) -> result; /// Set the values associated with the keys in the bucket. If the key already @@ -56,7 +56,7 @@ interface batch { /// /// If any of the keys do not exist in the bucket, it creates a new key-value pair. /// - /// If any other error occurs, it returns an error. When an error occurs, it + /// If any other error occurs, it returns an `Err(error)`. When an error occurs, it /// does not rollback the key-value pairs that were already set. Thus, this batch operation /// does not guarantee atomicity, implying that some key-value pairs could be /// set while others might fail. @@ -71,7 +71,7 @@ interface batch { /// /// If any of the keys do not exist in the bucket, it skips the key. /// - /// If any other error occurs, it returns an error. When an error occurs, it + /// If any other error occurs, it returns an `Err(error)`. When an error occurs, it /// does not rollback the key-value pairs that were already deleted. Thus, this batch operation /// does not guarantee atomicity, implying that some key-value pairs could be /// deleted while others might fail. diff --git a/wit/readwrite.wit b/wit/readwrite.wit index 637cde9..a94f26d 100644 --- a/wit/readwrite.wit +++ b/wit/readwrite.wit @@ -6,10 +6,10 @@ interface readwrite { /// Get the value associated with the key in the bucket. /// /// The value is returned as an option. If the key-value pair exists in the - /// bucket, it returns ok with the value. If the key does not exist in the - /// bucket, it returns ok with none. + /// bucket, it returns `Ok(value)`. If the key does not exist in the + /// bucket, it returns `Ok(none)`. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. get: func(bucket: borrow, key: key) -> result, error>; /// Set the value associated with the key in the bucket. If the key already @@ -17,21 +17,21 @@ interface readwrite { /// /// If the key does not exist in the bucket, it creates a new key-value pair. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. set: func(bucket: borrow, key: key, outgoing-value: borrow) -> result<_, error>; /// Delete the key-value pair associated with the key in the bucket. /// /// If the key does not exist in the bucket, it does nothing. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. delete: func(bucket: borrow, key: key) -> result<_, error>; /// Check if the key exists in the bucket. /// - /// If the key exists in the bucket, it returns ok with true. If the key does - /// not exist in the bucket, it returns ok with false. + /// If the key exists in the bucket, it returns `Ok(true)`. If the key does + /// not exist in the bucket, it returns `Ok(false)`. /// - /// If any other error occurs, it returns an error. + /// If any other error occurs, it returns an `Err(error)`. exists: func(bucket: borrow, key: key) -> result; } From 124b3d03e9aa45ce482b8dd16ff67f2c973556fd Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:18:01 -0800 Subject: [PATCH 04/14] update CI to the newest Signed-off-by: Jiaxiao Zhou (Mossaka) --- .github/workflows/main.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 72454f6..96b6655 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -16,7 +16,7 @@ jobs: curl -Lo 'wit-deps' https://github.com/bytecodealliance/wit-deps/releases/download/v0.3.5/wit-deps-x86_64-unknown-linux-musl chmod +x wit-deps ./wit-deps lock --check - - uses: WebAssembly/wit-abi-up-to-date@v16 + - uses: WebAssembly/wit-abi-up-to-date@v17 with: worlds: 'keyvalue keyvalue-handle-watch' - wit-bindgen: '0.15.0' \ No newline at end of file + wit-bindgen: '0.16.0' \ No newline at end of file From 7b7f1e333d4f83c1de08ce2caca4f6014c1c02ce Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:24:42 -0800 Subject: [PATCH 05/14] modified the portability criteria Signed-off-by: Jiaxiao Zhou (Mossaka) --- README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index febe46a..5c85bd9 100644 --- a/README.md +++ b/README.md @@ -14,9 +14,12 @@ Phase 1 ### Portability Criteria -`wasi:keyvalue` must have at least two complete independent implementations demonstrating -embeddability in a production key-value store context. The implementations must be able to run -on at least two different operating systems. +`wasi:keyvalue` must have at least two implementations in terms of relevant mainstream key-value stores in each of the following categories: + +1. Open-source key-value stores like Redis, Memcached, Etcd etc. +2. Proprietary key-value stores like AWS DynamoDB, Azure CosmosDB, Google Cloud Firestore etc. + +The implementations must be able to run on Linux, MacOS, and Windows. A testsuite that passes on the platforms and implementations mentioned above. From fab85adf65fbc9dbb23b7e7e1a15a93e4e02e5ee Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:37:46 -0800 Subject: [PATCH 06/14] update level of consistency for readwrite and batch interfaces Signed-off-by: Jiaxiao Zhou (Mossaka) --- wit/batch.wit | 6 +++--- wit/readwrite.wit | 19 +++++++++++++++++++ 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/wit/batch.wit b/wit/batch.wit index 1884d97..8f56bea 100644 --- a/wit/batch.wit +++ b/wit/batch.wit @@ -16,9 +16,9 @@ /// write operation completes, all subsequent read operations will return the /// value that was written. /// -/// The level of consistency in batch operations can vary depending on the -/// implementation. This interface does not guarantee strong consistency, meaning -/// that if a write operation completes, subsequent read operations may not return +/// The level of consistency in batch operations is **eventual consistency**, the same +/// with the readwrite interface. This interface does not guarantee strong consistency, +/// meaning that if a write operation completes, subsequent read operations may not return /// the value that was written. interface batch { /// A keyvalue interface that provides batch get operations. diff --git a/wit/readwrite.wit b/wit/readwrite.wit index a94f26d..fc525da 100644 --- a/wit/readwrite.wit +++ b/wit/readwrite.wit @@ -1,4 +1,23 @@ /// A keyvalue interface that provides simple read and write operations. +/// +/// A read/write operation is an operation that acts on a single key-value pair. +/// +/// The value in the key-value pair is defined as a `u8` byte array and the intention +/// is that it is the common denominator for all data types defined by different +/// key-value stores to handle data, ensuring compatibility between different +/// key-value stores. Note: the clients will be expecting serialization/deserialization overhead +/// to be handled by the key-value store. The value could be a serialized object from +/// JSON, HTML or vendor-specific data types like AWS S3 objects. +/// +/// Data consistency in a key value store refers to the gaurantee that once a +/// write operation completes, all subsequent read operations will return the +/// value that was written. +/// +/// The level of consistency in readwrite interfaces is **eventual consistency**, +/// which means that if a write operation completes successfully, all subsequent +/// read operations will eventually return the value that was written. In other words, +/// if we pause the updates to the system, the system eventually will return +/// the last updated value for read. interface readwrite { /// A keyvalue interface that provides simple read and write operations. use types.{bucket, error, incoming-value, key, outgoing-value}; From 464c41cbf9c3f1f234c5215ef4df65cceeb58dae Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:40:48 -0800 Subject: [PATCH 07/14] change get-keys to keys Signed-off-by: Jiaxiao Zhou (Mossaka) --- wit/atomic.wit | 2 +- wit/batch.wit | 8 ++++---- wit/error.wit | 2 +- wit/types.wit | 3 --- 4 files changed, 6 insertions(+), 9 deletions(-) diff --git a/wit/atomic.wit b/wit/atomic.wit index 2c97e72..1d32b7e 100644 --- a/wit/atomic.wit +++ b/wit/atomic.wit @@ -9,7 +9,7 @@ interface atomic { use types.{bucket, error, key}; /// Atomically increment the value associated with the key in the bucket by the - /// given delta. It returns the new value. This is a single, indivisible operation. + /// given delta. It returns the new value. /// /// If the key does not exist in the bucket, it creates a new key-value pair /// with the value set to the given delta. diff --git a/wit/batch.wit b/wit/batch.wit index 8f56bea..5108f1b 100644 --- a/wit/batch.wit +++ b/wit/batch.wit @@ -22,7 +22,7 @@ /// the value that was written. interface batch { /// A keyvalue interface that provides batch get operations. - use types.{bucket, error, key, keys, incoming-value, outgoing-value}; + use types.{bucket, error, key, incoming-value, outgoing-value}; /// Get the values associated with the keys in the bucket. It returns a list of /// incoming-value that can be consumed to get the value associated with the key. @@ -35,7 +35,7 @@ interface batch { /// MAY show an out-of-date value if there are concurrent writes to the bucket. /// /// If any other error occurs, it returns an `Err(error)`. - get-many: func(bucket: borrow, keys: keys) -> result>, error>; + get-many: func(bucket: borrow, keys: list) -> result>, error>; /// Get all the keys in the bucket. It returns a list of keys. /// @@ -46,7 +46,7 @@ interface batch { /// MAY show an out-of-date list of keys if there are concurrent writes to the bucket. /// /// If any error occurs, it returns an `Err(error)`. - get-keys: func(bucket: borrow) -> result; + keys: func(bucket: borrow) -> result, error>; /// Set the values associated with the keys in the bucket. If the key already /// exists in the bucket, it overwrites the value. @@ -77,5 +77,5 @@ interface batch { /// deleted while others might fail. /// /// Other concurrent operations may also be able to see the partial results. - delete-many: func(bucket: borrow, keys: keys) -> result<_, error>; + delete-many: func(bucket: borrow, keys: list) -> result<_, error>; } diff --git a/wit/error.wit b/wit/error.wit index 1caba7e..cd244f6 100644 --- a/wit/error.wit +++ b/wit/error.wit @@ -4,7 +4,7 @@ interface wasi-keyvalue-error { /// Common errors: /// - Connectivity errors (e.g. network errors): when the client cannot establish /// a connection to the keyvalue service. - /// - Autnetication and Authorization errors: when the client fails to authenticate + /// - Authentication and Authorization errors: when the client fails to authenticate /// or does not have the required permissions to perform the operation. /// - Data errors: when the client sends incompatible or corrupted data. /// - Resource errors: when the system runs out of resources (e.g. memory). diff --git a/wit/types.wit b/wit/types.wit index 105bdb2..de14cf8 100644 --- a/wit/types.wit +++ b/wit/types.wit @@ -23,9 +23,6 @@ interface types { /// retrieve the value from the bucket. type key = string; - /// A list of keys - type keys = list; - use wasi:io/streams@0.2.0-rc-2023-11-10.{input-stream, output-stream}; use wasi-keyvalue-error.{ error }; /// A value is the data stored in a key-value pair. The value can be of any type From bd0769fba773c62d48f4ef1235a59c58a91de5d0 Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:40:58 -0800 Subject: [PATCH 08/14] regenerate markdowns Signed-off-by: Jiaxiao Zhou (Mossaka) --- keyvalue-handle-watch.md | 75 ++++++++++++++++++++++------------------ keyvalue.md | 75 ++++++++++++++++++++++------------------ 2 files changed, 84 insertions(+), 66 deletions(-) diff --git a/keyvalue-handle-watch.md b/keyvalue-handle-watch.md index 6dc67f4..d23c5bb 100644 --- a/keyvalue-handle-watch.md +++ b/keyvalue-handle-watch.md @@ -428,7 +428,7 @@ is ready for reading, before performing the splice.

            • Connectivity errors (e.g. network errors): when the client cannot establish a connection to the keyvalue service.
              • -
            • Autnetication and Authorization errors: when the client fails to authenticate +
            • Authentication and Authorization errors: when the client fails to authenticate or does not have the required permissions to perform the operation.
            • Data errors: when the client sends incompatible or corrupted data.
            • Resource errors: when the system runs out of resources (e.g. memory).
              • @@ -479,9 +479,6 @@ can very depending on the specific implementation. For example,

              string

              A key is a unique identifier for a value in a bucket. The key is used to retrieve the value from the bucket. -

              type keys

              -

              keys

              -

              A list of keys

              resource outgoing-value

              A value is the data stored in a key-value pair. The value can be of any type that can be represented in a byte array. It provides a way to write the value @@ -572,6 +569,21 @@ value as an input-stream.

            Import interface wasi:keyvalue/readwrite

            A keyvalue interface that provides simple read and write operations.

            +

            A read/write operation is an operation that acts on a single key-value pair.

            +

            The value in the key-value pair is defined as a u8 byte array and the intention +is that it is the common denominator for all data types defined by different +key-value stores to handle data, ensuring compatibility between different +key-value stores. Note: the clients will be expecting serialization/deserialization overhead +to be handled by the key-value store. The value could be a serialized object from +JSON, HTML or vendor-specific data types like AWS S3 objects.

            +

            Data consistency in a key value store refers to the gaurantee that once a +write operation completes, all subsequent read operations will return the +value that was written.

            +

            The level of consistency in readwrite interfaces is eventual consistency, +which means that if a write operation completes successfully, all subsequent +read operations will eventually return the value that was written. In other words, +if we pause the updates to the system, the system eventually will return +the last updated value for read.

            Types

            type bucket

            @@ -594,9 +606,9 @@ value as an input-stream.

            get: func

            Get the value associated with the key in the bucket.

            The value is returned as an option. If the key-value pair exists in the -bucket, it returns ok with the value. If the key does not exist in the -bucket, it returns ok with none.

            -

            If any other error occurs, it returns an error.

            +bucket, it returns Ok(value). If the key does not exist in the +bucket, it returns Ok(none).

            +

            If any other error occurs, it returns an Err(error).

            Params
            • bucket: borrow<bucket>
              • @@ -610,7 +622,7 @@ bucket, it returns ok with none.

              Set the value associated with the key in the bucket. If the key already exists in the bucket, it overwrites the value.

              If the key does not exist in the bucket, it creates a new key-value pair.

              -

              If any other error occurs, it returns an error.

              +

              If any other error occurs, it returns an Err(error).

              Params
              • bucket: borrow<bucket>
                • @@ -624,7 +636,7 @@ exists in the bucket, it overwrites the value.

                delete: func

                Delete the key-value pair associated with the key in the bucket.

                If the key does not exist in the bucket, it does nothing.

                -

                If any other error occurs, it returns an error.

                +

                If any other error occurs, it returns an Err(error).

                Params
                • bucket: borrow<bucket>
                  • @@ -636,9 +648,9 @@ exists in the bucket, it overwrites the value.

                exists: func

                Check if the key exists in the bucket.

                -

                If the key exists in the bucket, it returns ok with true. If the key does -not exist in the bucket, it returns ok with false.

                -

                If any other error occurs, it returns an error.

                +

                If the key exists in the bucket, it returns Ok(true). If the key does +not exist in the bucket, it returns Ok(false).

                +

                If any other error occurs, it returns an Err(error).

                Params
                • bucket: borrow<bucket>
                  • @@ -669,10 +681,10 @@ at all.

                  Functions

                  increment: func

                  Atomically increment the value associated with the key in the bucket by the -given delta. It returns the new value. This is a single, indivisible operation.

                  +given delta. It returns the new value.

                  If the key does not exist in the bucket, it creates a new key-value pair with the value set to the given delta.

                  -

                  If any other error occurs, it returns an error.

                  +

                  If any other error occurs, it returns an Err(error).

                  Params
                  • bucket: borrow<bucket>
                    • @@ -686,11 +698,11 @@ with the value set to the given delta.

                    compare-and-swap: func

                    Compare-and-swap (CAS) atomically updates the value associated with the key in the bucket if the value matches the old value. This operation returns -Ok(true) if the swap was successful, Ok(false) if the value did not match,

                    +Ok(true) if the swap was successful, Ok(false) if the value did not match,

                    A successful CAS operation means the current value matched the old value and was replaced with the new value.

                    -

                    If the key does not exist in the bucket, it returns Ok(false).

                    -

                    If any other error occurs, it returns an error.

                    +

                    If the key does not exist in the bucket, it returns Ok(false).

                    +

                    If any other error occurs, it returns an Err(error).

                    Params
                    • bucket: borrow<bucket>
                      • @@ -716,9 +728,9 @@ provide atomicity.

                      Data consistency in a key value store refers to the gaurantee that once a write operation completes, all subsequent read operations will return the value that was written.

                      -

                      The level of consistency in batch operations can vary depending on the -implementation. This interface does not guarantee strong consistency, meaning -that if a write operation completes, subsequent read operations may not return +

                      The level of consistency in batch operations is eventual consistency, the same +with the readwrite interface. This interface does not guarantee strong consistency, +meaning that if a write operation completes, subsequent read operations may not return the value that was written.

                      Types

                      @@ -731,9 +743,6 @@ the value that was written.

                      #### `type key` [`key`](#key)

                      -#### `type keys` -[`keys`](#keys) -

                      #### `type incoming-value` [`incoming-value`](#incoming_value)

                      @@ -745,33 +754,33 @@ the value that was written.

                      get-many: func

                      Get the values associated with the keys in the bucket. It returns a list of incoming-value that can be consumed to get the value associated with the key.

                      -

                      If any of the keys do not exist in the bucket, it returns an None value for +

                      If any of the keys do not exist in the bucket, it returns a none value for that key in the list.

                      Note that the key-value pairs are guaranteed to be returned in the same order

                      MAY show an out-of-date value if there are concurrent writes to the bucket.

                      -

                      If any other error occurs, it returns an error.

                      +

                      If any other error occurs, it returns an Err(error).

                      Params
                      Return values
                      -

                      get-keys: func

                      +

                      keys: func

                      Get all the keys in the bucket. It returns a list of keys.

                      Note that the keys are not guaranteed to be returned in any particular order.

                      If the bucket is empty, it returns an empty list.

                      MAY show an out-of-date list of keys if there are concurrent writes to the bucket.

                      -

                      If any error occurs, it returns an error.

                      +

                      If any error occurs, it returns an Err(error).

                      Params
                      Return values

                      set-many: func

                      Set the values associated with the keys in the bucket. If the key already @@ -779,7 +788,7 @@ exists in the bucket, it overwrites the value.

                      Note that the key-value pairs are not guaranteed to be set in the order they are provided.

                      If any of the keys do not exist in the bucket, it creates a new key-value pair.

                      -

                      If any other error occurs, it returns an error. When an error occurs, it +

                      If any other error occurs, it returns an Err(error). When an error occurs, it does not rollback the key-value pairs that were already set. Thus, this batch operation does not guarantee atomicity, implying that some key-value pairs could be set while others might fail.

                      @@ -798,7 +807,7 @@ set while others might fail.

                      Note that the key-value pairs are not guaranteed to be deleted in the order they are provided.

                      If any of the keys do not exist in the bucket, it skips the key.

                      -

                      If any other error occurs, it returns an error. When an error occurs, it +

                      If any other error occurs, it returns an Err(error). When an error occurs, it does not rollback the key-value pairs that were already deleted. Thus, this batch operation does not guarantee atomicity, implying that some key-value pairs could be deleted while others might fail.

                      @@ -806,7 +815,7 @@ deleted while others might fail.

                      Params
                      Return values
                        diff --git a/keyvalue.md b/keyvalue.md index 6b58e3f..f183ff4 100644 --- a/keyvalue.md +++ b/keyvalue.md @@ -431,7 +431,7 @@ is ready for reading, before performing the splice.

                        • Connectivity errors (e.g. network errors): when the client cannot establish a connection to the keyvalue service.
                          • -
                        • Autnetication and Authorization errors: when the client fails to authenticate +
                        • Authentication and Authorization errors: when the client fails to authenticate or does not have the required permissions to perform the operation.
                        • Data errors: when the client sends incompatible or corrupted data.
                        • Resource errors: when the system runs out of resources (e.g. memory).
                          • @@ -482,9 +482,6 @@ can very depending on the specific implementation. For example,

                          string

                          A key is a unique identifier for a value in a bucket. The key is used to retrieve the value from the bucket. -

                          type keys

                          -

                          keys

                          -

                          A list of keys

                          resource outgoing-value

                          A value is the data stored in a key-value pair. The value can be of any type that can be represented in a byte array. It provides a way to write the value @@ -575,6 +572,21 @@ value as an input-stream.

                        Import interface wasi:keyvalue/readwrite

                        A keyvalue interface that provides simple read and write operations.

                        +

                        A read/write operation is an operation that acts on a single key-value pair.

                        +

                        The value in the key-value pair is defined as a u8 byte array and the intention +is that it is the common denominator for all data types defined by different +key-value stores to handle data, ensuring compatibility between different +key-value stores. Note: the clients will be expecting serialization/deserialization overhead +to be handled by the key-value store. The value could be a serialized object from +JSON, HTML or vendor-specific data types like AWS S3 objects.

                        +

                        Data consistency in a key value store refers to the gaurantee that once a +write operation completes, all subsequent read operations will return the +value that was written.

                        +

                        The level of consistency in readwrite interfaces is eventual consistency, +which means that if a write operation completes successfully, all subsequent +read operations will eventually return the value that was written. In other words, +if we pause the updates to the system, the system eventually will return +the last updated value for read.

                        Types

                        type bucket

                        @@ -597,9 +609,9 @@ value as an input-stream.

                        get: func

                        Get the value associated with the key in the bucket.

                        The value is returned as an option. If the key-value pair exists in the -bucket, it returns ok with the value. If the key does not exist in the -bucket, it returns ok with none.

                        -

                        If any other error occurs, it returns an error.

                        +bucket, it returns Ok(value). If the key does not exist in the +bucket, it returns Ok(none).

                        +

                        If any other error occurs, it returns an Err(error).

                        Params
                        • bucket: borrow<bucket>
                          • @@ -613,7 +625,7 @@ bucket, it returns ok with none.

                          Set the value associated with the key in the bucket. If the key already exists in the bucket, it overwrites the value.

                          If the key does not exist in the bucket, it creates a new key-value pair.

                          -

                          If any other error occurs, it returns an error.

                          +

                          If any other error occurs, it returns an Err(error).

                          Params
                          • bucket: borrow<bucket>
                            • @@ -627,7 +639,7 @@ exists in the bucket, it overwrites the value.

                            delete: func

                            Delete the key-value pair associated with the key in the bucket.

                            If the key does not exist in the bucket, it does nothing.

                            -

                            If any other error occurs, it returns an error.

                            +

                            If any other error occurs, it returns an Err(error).

                            Params
                            • bucket: borrow<bucket>
                              • @@ -639,9 +651,9 @@ exists in the bucket, it overwrites the value.

                            exists: func

                            Check if the key exists in the bucket.

                            -

                            If the key exists in the bucket, it returns ok with true. If the key does -not exist in the bucket, it returns ok with false.

                            -

                            If any other error occurs, it returns an error.

                            +

                            If the key exists in the bucket, it returns Ok(true). If the key does +not exist in the bucket, it returns Ok(false).

                            +

                            If any other error occurs, it returns an Err(error).

                            Params
                            • bucket: borrow<bucket>
                              • @@ -672,10 +684,10 @@ at all.

                              Functions

                              increment: func

                              Atomically increment the value associated with the key in the bucket by the -given delta. It returns the new value. This is a single, indivisible operation.

                              +given delta. It returns the new value.

                              If the key does not exist in the bucket, it creates a new key-value pair with the value set to the given delta.

                              -

                              If any other error occurs, it returns an error.

                              +

                              If any other error occurs, it returns an Err(error).

                              Params
                              • bucket: borrow<bucket>
                                • @@ -689,11 +701,11 @@ with the value set to the given delta.

                                compare-and-swap: func

                                Compare-and-swap (CAS) atomically updates the value associated with the key in the bucket if the value matches the old value. This operation returns -Ok(true) if the swap was successful, Ok(false) if the value did not match,

                                +Ok(true) if the swap was successful, Ok(false) if the value did not match,

                                A successful CAS operation means the current value matched the old value and was replaced with the new value.

                                -

                                If the key does not exist in the bucket, it returns Ok(false).

                                -

                                If any other error occurs, it returns an error.

                                +

                                If the key does not exist in the bucket, it returns Ok(false).

                                +

                                If any other error occurs, it returns an Err(error).

                                Params
                                • bucket: borrow<bucket>
                                  • @@ -719,9 +731,9 @@ provide atomicity.

                                  Data consistency in a key value store refers to the gaurantee that once a write operation completes, all subsequent read operations will return the value that was written.

                                  -

                                  The level of consistency in batch operations can vary depending on the -implementation. This interface does not guarantee strong consistency, meaning -that if a write operation completes, subsequent read operations may not return +

                                  The level of consistency in batch operations is eventual consistency, the same +with the readwrite interface. This interface does not guarantee strong consistency, +meaning that if a write operation completes, subsequent read operations may not return the value that was written.

                                  Types

                                  @@ -734,9 +746,6 @@ the value that was written.

                                  #### `type key` [`key`](#key)

                                  -#### `type keys` -[`keys`](#keys) -

                                  #### `type incoming-value` [`incoming-value`](#incoming_value)

                                  @@ -748,33 +757,33 @@ the value that was written.

                                  get-many: func

                                  Get the values associated with the keys in the bucket. It returns a list of incoming-value that can be consumed to get the value associated with the key.

                                  -

                                  If any of the keys do not exist in the bucket, it returns an None value for +

                                  If any of the keys do not exist in the bucket, it returns a none value for that key in the list.

                                  Note that the key-value pairs are guaranteed to be returned in the same order

                                  MAY show an out-of-date value if there are concurrent writes to the bucket.

                                  -

                                  If any other error occurs, it returns an error.

                                  +

                                  If any other error occurs, it returns an Err(error).

                                  Params
                                  Return values
                                  -

                                  get-keys: func

                                  +

                                  keys: func

                                  Get all the keys in the bucket. It returns a list of keys.

                                  Note that the keys are not guaranteed to be returned in any particular order.

                                  If the bucket is empty, it returns an empty list.

                                  MAY show an out-of-date list of keys if there are concurrent writes to the bucket.

                                  -

                                  If any error occurs, it returns an error.

                                  +

                                  If any error occurs, it returns an Err(error).

                                  Params
                                  Return values

                                  set-many: func

                                  Set the values associated with the keys in the bucket. If the key already @@ -782,7 +791,7 @@ exists in the bucket, it overwrites the value.

                                  Note that the key-value pairs are not guaranteed to be set in the order they are provided.

                                  If any of the keys do not exist in the bucket, it creates a new key-value pair.

                                  -

                                  If any other error occurs, it returns an error. When an error occurs, it +

                                  If any other error occurs, it returns an Err(error). When an error occurs, it does not rollback the key-value pairs that were already set. Thus, this batch operation does not guarantee atomicity, implying that some key-value pairs could be set while others might fail.

                                  @@ -801,7 +810,7 @@ set while others might fail.

                                  Note that the key-value pairs are not guaranteed to be deleted in the order they are provided.

                                  If any of the keys do not exist in the bucket, it skips the key.

                                  -

                                  If any other error occurs, it returns an error. When an error occurs, it +

                                  If any other error occurs, it returns an Err(error). When an error occurs, it does not rollback the key-value pairs that were already deleted. Thus, this batch operation does not guarantee atomicity, implying that some key-value pairs could be deleted while others might fail.

                                  @@ -809,7 +818,7 @@ deleted while others might fail.

                                  Params
                                  Return values
                                    From d77574eb0dffff391bc628ee07db2b54e5a476a3 Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:48:36 -0800 Subject: [PATCH 09/14] updated README Signed-off-by: Jiaxiao Zhou (Mossaka) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5c85bd9..539ddf4 100644 --- a/README.md +++ b/README.md @@ -114,7 +114,7 @@ This proposal was inspired by Dapr's [State API] ### Change log - 2023-11-30: - - Changed the `get` and `get-many` and `get-keys` signatures + - Changed the `get` and `get-many` and `keys` signatures - Updated comments in all the interfaces. - Renamed `wasi-cloud-error` to `wasi-keyvalue-error` - 2023-05-17: Updated batch example to use one interface instead of 2 From 6c1018270a085885c0b4c8fbb63a81ec55dd79e9 Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 19 Dec 2023 15:59:41 -0800 Subject: [PATCH 10/14] incoming value size returns a result Signed-off-by: Jiaxiao Zhou (Mossaka) --- README.md | 3 ++- keyvalue-handle-watch.md | 27 +++++++++++++++++++++------ keyvalue.md | 27 +++++++++++++++++++++------ wit/types.wit | 25 ++++++++++++++++++++----- 4 files changed, 64 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 539ddf4..6a3c74c 100644 --- a/README.md +++ b/README.md @@ -112,7 +112,8 @@ This proposal was inspired by Dapr's [State API] [State API](https://docs.dapr.io/developing-applications/building-blocks/state-management/) ### Change log - +- 2023-12-19: + - Changed the `size` to `incoming-value-size` and it's signature - 2023-11-30: - Changed the `get` and `get-many` and `keys` signatures - Updated comments in all the interfaces. diff --git a/keyvalue-handle-watch.md b/keyvalue-handle-watch.md index d23c5bb..b8a8663 100644 --- a/keyvalue-handle-watch.md +++ b/keyvalue-handle-watch.md @@ -491,13 +491,16 @@ to the output-stream defined in the wasi-io interface.

                                    #### `resource incoming-value`

                                    A incoming-value is a wrapper around a value. It provides a way to read the value -from the input-stream defined in the wasi-io interface.

                                    +from the input-stream defined in the wasi-io interface.

                                    The incoming-value provides two ways to consume the value:

                                    1. incoming-value-consume-sync consumes the value synchronously and returns the -value as a list of bytes.
                                      2. +value as a list<u8>.
                                    2. incoming-value-consume-async consumes the value asynchronously and returns the -value as an input-stream.
                                      3. +value as an input-stream. +In addition, it provides a incoming-value-size function to get the size of the value. +This is useful when the value is large and the caller wants to allocate a buffer of +the right size to consume the value.

                                    type incoming-value-async-body

                                    input-stream

                                    @@ -508,6 +511,8 @@ value as an input-stream. ----

                                    Functions

                                    [static]bucket.open-bucket: func

                                    +

                                    Opens a bucket with the given name.

                                    +

                                    If any error occurs, including if the bucket does not exist, it returns an Err(error).

                                    Params

                                    [method]outgoing-value.outgoing-value-write-body-async: func

                                    +

                                    Writes the value to the output-stream asynchronously. +If any other error occurs, it returns an Err(error).

                                    Params

                                    [method]outgoing-value.outgoing-value-write-body-sync: func

                                    +

                                    Writes the value to the output-stream synchronously. +If any other error occurs, it returns an Err(error).

                                    Params

                                    [method]incoming-value.incoming-value-consume-sync: func

                                    +

                                    Consumes the value synchronously and returns the value as a list of bytes. +If any other error occurs, it returns an Err(error).

                                    Params

                                    [method]incoming-value.incoming-value-consume-async: func

                                    +

                                    Consumes the value asynchronously and returns the value as an input-stream. +If any other error occurs, it returns an Err(error).

                                    Params
                                    • self: borrow<incoming-value>
                                      • @@ -558,14 +571,16 @@ value as an input-stream. -

                                      [method]incoming-value.size: func

                                      +

                                      [method]incoming-value.incoming-value-size: func

                                      +

                                      The size of the value in bytes. +If the size is unknown or unavailable, this function returns an Err(error).

                                      Params
                                      Return values
                                        -
                                      • u64
                                        • +
                                      • result<u64, own<error>>

                                      Import interface wasi:keyvalue/readwrite

                                      A keyvalue interface that provides simple read and write operations.

                                      diff --git a/keyvalue.md b/keyvalue.md index f183ff4..091268d 100644 --- a/keyvalue.md +++ b/keyvalue.md @@ -494,13 +494,16 @@ to the output-stream defined in the wasi-io interface.

                                      #### `resource incoming-value`

                                      A incoming-value is a wrapper around a value. It provides a way to read the value -from the input-stream defined in the wasi-io interface.

                                      +from the input-stream defined in the wasi-io interface.

                                      The incoming-value provides two ways to consume the value:

                                      1. incoming-value-consume-sync consumes the value synchronously and returns the -value as a list of bytes.
                                        2. +value as a list<u8>.
                                      2. incoming-value-consume-async consumes the value asynchronously and returns the -value as an input-stream.
                                        3. +value as an input-stream. +In addition, it provides a incoming-value-size function to get the size of the value. +This is useful when the value is large and the caller wants to allocate a buffer of +the right size to consume the value.

                                      type incoming-value-async-body

                                      input-stream

                                      @@ -511,6 +514,8 @@ value as an input-stream. ----

                                      Functions

                                      [static]bucket.open-bucket: func

                                      +

                                      Opens a bucket with the given name.

                                      +

                                      If any error occurs, including if the bucket does not exist, it returns an Err(error).

                                      Params

                                      [method]outgoing-value.outgoing-value-write-body-async: func

                                      +

                                      Writes the value to the output-stream asynchronously. +If any other error occurs, it returns an Err(error).

                                      Params

                                      [method]outgoing-value.outgoing-value-write-body-sync: func

                                      +

                                      Writes the value to the output-stream synchronously. +If any other error occurs, it returns an Err(error).

                                      Params

                                      [method]incoming-value.incoming-value-consume-sync: func

                                      +

                                      Consumes the value synchronously and returns the value as a list of bytes. +If any other error occurs, it returns an Err(error).

                                      Params

                                      [method]incoming-value.incoming-value-consume-async: func

                                      +

                                      Consumes the value asynchronously and returns the value as an input-stream. +If any other error occurs, it returns an Err(error).

                                      Params
                                      • self: borrow<incoming-value>
                                        • @@ -561,14 +574,16 @@ value as an input-stream. -

                                        [method]incoming-value.size: func

                                        +

                                        [method]incoming-value.incoming-value-size: func

                                        +

                                        The size of the value in bytes. +If the size is unknown or unavailable, this function returns an Err(error).

                                        Params
                                        Return values
                                          -
                                        • u64
                                          • +
                                        • result<u64, own<error>>

                                        Import interface wasi:keyvalue/readwrite

                                        A keyvalue interface that provides simple read and write operations.

                                        diff --git a/wit/types.wit b/wit/types.wit index de14cf8..5080380 100644 --- a/wit/types.wit +++ b/wit/types.wit @@ -17,6 +17,9 @@ interface types { /// In this interface, we use the term `bucket` to refer to a collection of key-value // Soon: switch to `resource bucket { ... }` resource bucket { + /// Opens a bucket with the given name. + /// + /// If any error occurs, including if the bucket does not exist, it returns an `Err(error)`. open-bucket: static func(name: string) -> result; } /// A key is a unique identifier for a value in a bucket. The key is used to @@ -31,26 +34,38 @@ interface types { // Soon: switch to `resource value { ... }` resource outgoing-value { new-outgoing-value: static func() -> outgoing-value; + /// Writes the value to the output-stream asynchronously. + /// If any other error occurs, it returns an `Err(error)`. outgoing-value-write-body-async: func() -> result; + /// Writes the value to the output-stream synchronously. + /// If any other error occurs, it returns an `Err(error)`. outgoing-value-write-body-sync: func(value: outgoing-value-body-sync) -> result<_, error>; } type outgoing-value-body-async = output-stream; type outgoing-value-body-sync = list; - /// A incoming-value is a wrapper around a value. It provides a way to read the value - /// from the input-stream defined in the `wasi-io` interface. + /// from the `input-stream` defined in the `wasi-io` interface. /// /// The incoming-value provides two ways to consume the value: /// 1. `incoming-value-consume-sync` consumes the value synchronously and returns the - /// value as a list of bytes. + /// value as a `list`. /// 2. `incoming-value-consume-async` consumes the value asynchronously and returns the - /// value as an input-stream. + /// value as an `input-stream`. + /// In addition, it provides a `incoming-value-size` function to get the size of the value. + /// This is useful when the value is large and the caller wants to allocate a buffer of + /// the right size to consume the value. // Soon: switch to `resource incoming-value { ... }` resource incoming-value { + /// Consumes the value synchronously and returns the value as a list of bytes. + /// If any other error occurs, it returns an `Err(error)`. incoming-value-consume-sync: func() -> result; + /// Consumes the value asynchronously and returns the value as an `input-stream`. + /// If any other error occurs, it returns an `Err(error)`. incoming-value-consume-async: func() -> result; - size: func() -> u64; + /// The size of the value in bytes. + /// If the size is unknown or unavailable, this function returns an `Err(error)`. + incoming-value-size: func() -> result; } type incoming-value-async-body = input-stream; type incoming-value-sync-body = list; From 080046eb1917084da5eec051f57cf9188ed3ac1c Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Fri, 5 Jan 2024 11:57:02 -0800 Subject: [PATCH 11/14] feat(world): rename keyvalue world to imports Signed-off-by: Jiaxiao Zhou (Mossaka) --- keyvalue.md => imports.md | 4 ++-- wit/world.wit | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) rename keyvalue.md => imports.md (99%) diff --git a/keyvalue.md b/imports.md similarity index 99% rename from keyvalue.md rename to imports.md index 091268d..a58498a 100644 --- a/keyvalue.md +++ b/imports.md @@ -1,5 +1,5 @@ -

                                        World keyvalue

                                        -

                                        The wasi:keyvalue/keyvalue world provides common APIs for interacting +

                                        World imports

                                        +

                                        The wasi:keyvalue/imports world provides common APIs for interacting with key-value stores. Components targeting this world will be able to do

                                          diff --git a/wit/world.wit b/wit/world.wit index d0d1d5e..37037db 100644 --- a/wit/world.wit +++ b/wit/world.wit @@ -1,12 +1,12 @@ package wasi:keyvalue; -/// The `wasi:keyvalue/keyvalue` world provides common APIs for interacting +/// The `wasi:keyvalue/imports` world provides common APIs for interacting /// with key-value stores. Components targeting this world will be able to /// do /// 1. CRUD (create, read, update, delete) operations on key-value stores. /// 2. Atomic `increment` and CAS (compare-and-swap) operations. /// 3. Batch operations that can reduce the number of round trips to the network. -world keyvalue { +world imports { /// The `readwrite` capability allows the component to perform CRUD /// operations on the key-value store. import readwrite; @@ -21,6 +21,6 @@ world keyvalue { } world keyvalue-handle-watch { - include keyvalue; + include imports; export handle-watch; } \ No newline at end of file From 4b8573218617d86be59dde884acb18ab7a57b1d6 Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Fri, 5 Jan 2024 13:13:19 -0800 Subject: [PATCH 12/14] ci: change keyvalue to imports Signed-off-by: Jiaxiao Zhou (Mossaka) --- .github/workflows/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 96b6655..dcf74d7 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -18,5 +18,5 @@ jobs: ./wit-deps lock --check - uses: WebAssembly/wit-abi-up-to-date@v17 with: - worlds: 'keyvalue keyvalue-handle-watch' + worlds: 'imports keyvalue-handle-watch' wit-bindgen: '0.16.0' \ No newline at end of file From d9f4c4c3aed51f11784551672e746a415fd33723 Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 16 Jan 2024 13:10:50 -0800 Subject: [PATCH 13/14] chore(*): change readwrite interface name to eventual Signed-off-by: Jiaxiao Zhou (Mossaka) --- README.md | 60 ++++++++++++++------------- imports.md | 14 +++---- keyvalue-handle-watch.md | 14 +++---- wit/{batch.wit => eventual-batch.wit} | 4 +- wit/{readwrite.wit => eventual.wit} | 6 +-- wit/world.wit | 12 +++--- 6 files changed, 57 insertions(+), 53 deletions(-) rename wit/{batch.wit => eventual-batch.wit} (97%) rename wit/{readwrite.wit => eventual.wit} (93%) diff --git a/README.md b/README.md index 6a3c74c..ef3e880 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ Phase 1 ### Portability Criteria -`wasi:keyvalue` must have at least two implementations in terms of relevant mainstream key-value stores in each of the following categories: +`wasi:keyvalue` must have at least two implementations in terms of relevant mainstream key-value stores in each of the following categories: 1. Open-source key-value stores like Redis, Memcached, Etcd etc. 2. Proprietary key-value stores like AWS DynamoDB, Azure CosmosDB, Google Cloud Firestore etc. @@ -24,26 +24,24 @@ The implementations must be able to run on Linux, MacOS, and Windows. A testsuite that passes on the platforms and implementations mentioned above. ## Table of Contents -- [WASI Key-Value Store](#wasi-key-value-store) - - [Current Phase](#current-phase) - - [Champions](#champions) - - [Portability Criteria](#portability-criteria) - - [Table of Contents](#table-of-contents) - - [Introduction](#introduction) - - [Goals](#goals) - - [Non-goals](#non-goals) - - [API walk-through](#api-walk-through) - - [Use case 1](#use-case-1) - - [Use case 2](#use-case-2) - - [Detailed design discussion](#detailed-design-discussion) - - [Tricky design choice 1](#tricky-design-choice-1) - - [Tricky design choice 2](#tricky-design-choice-2) - - [Considered alternatives](#considered-alternatives) - - [Alternative 1](#alternative-1) - - [Alternative 2](#alternative-2) - - [Stakeholder Interest \& Feedback](#stakeholder-interest--feedback) - - [References \& acknowledgements](#references--acknowledgements) - - [Change log](#change-log) + +- [WASI Key-Value Store](#wasi-key-value-store) - [Current Phase](#current-phase) - [Champions](#champions) - [Portability Criteria](#portability-criteria) + - [Table of Contents](#table-of-contents) + - [Introduction](#introduction) + - [Goals](#goals) + - [Non-goals](#non-goals) + - [API walk-through](#api-walk-through) + - [Use case 1](#use-case-1) + - [Use case 2](#use-case-2) + - [Detailed design discussion](#detailed-design-discussion) + - [Tricky design choice 1](#tricky-design-choice-1) + - [Tricky design choice 2](#tricky-design-choice-2) + - [Considered alternatives](#considered-alternatives) + - [Alternative 1](#alternative-1) + - [Alternative 2](#alternative-2) + - [Stakeholder Interest \& Feedback](#stakeholder-interest--feedback) + - [References \& acknowledgements](#references--acknowledgements) + - [Change log](#change-log) ### Introduction @@ -78,22 +76,25 @@ A third design goal of `wasi:keyvalue` interfaces is to allow performance optimi ### API walk-through -The proposal can be understood by first reading the comments of `world.wit`, then `readwrite.wit`, `atomic.wit`, `batch.wit`, `caching.wit` and finally `types.wit` +The proposal can be understood by first reading the comments of `world.wit`, then `eventual.wit`, `atomic.wit`, `eventual-batch.wit`, `caching.wit` and finally `types.wit` [`world.wit`](./wit/world.wit) -[`readwrite.wit`](./wit/readwrite.wit) +[`eventual.wit`](./wit/eventual.wit) [`atomic.wit`](./wit/atomic.wit) -[`batch.wit`](./wit/batch.wit) +[`eventual-batch.wit`](./wit/eventual-batch.wit) [`caching.wit`](./wit/caching.wit) [`types.wit`](./wit/types.wit) ### Working with the WIT Bindings can be generated from the `wit` directory via: + ``` wit-bindgen c wit/ --world proxy ``` + and can be validated and otherwise manipulated via: + ``` wasm-tools component wit wit/ ... ``` @@ -112,9 +113,12 @@ This proposal was inspired by Dapr's [State API] [State API](https://docs.dapr.io/developing-applications/building-blocks/state-management/) ### Change log + +- 2024-01-16: + - Changed the `readwrite` and `batch` interface names to `eventual` and `eventual-batch`. See more details [here](https://github.com/WebAssembly/wasi-keyvalue/pull/30#discussion_r1442282650). - 2023-12-19: - Changed the `size` to `incoming-value-size` and it's signature -- 2023-11-30: +- 2023-11-30: - Changed the `get` and `get-many` and `keys` signatures - Updated comments in all the interfaces. - Renamed `wasi-cloud-error` to `wasi-keyvalue-error` @@ -123,10 +127,10 @@ This proposal was inspired by Dapr's [State API] - 2023-02-13: The following changes were made to the API: - Added `bucket` type to the `types` interface. - Uses pseudo-stream and pseudo-resource types inspired from [wasi-io ](https://github.com/WebAssembly/wasi-io) - - Added *.wit files for the interfaces and worlds, which are verified by the [wasm-tools](https://github.com/bytecodealliance/wasm-tools) + - Added \*.wit files for the interfaces and worlds, which are verified by the [wasm-tools](https://github.com/bytecodealliance/wasm-tools) - Added a inbound-keyvalue interface that handles the `watch` operation. - 2023-01-28: The following changes were made to the API: - - Changed `stream` type to `list` type because `stream` is not supported by the current version of [*.wit](https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md). - - Removed the worlds section because the star import syntax is not supported by the current version of [*.wit](https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md) + - Changed `stream` type to `list` type because `stream` is not supported by the current version of [\*.wit](https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md). + - Removed the worlds section because the star import syntax is not supported by the current version of [\*.wit](https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md) - 2022-11-29: Renamed `bulk-get` to `get-multi` and `bulk-set` to `set-multi` to be consistent with the naming convention of the other interfaces. - 2022-10-31: Initial draft diff --git a/imports.md b/imports.md index a58498a..60ea483 100644 --- a/imports.md +++ b/imports.md @@ -15,9 +15,9 @@ do

                                        1. interface wasi:io/streams@0.2.0-rc-2023-11-10
                                        2. interface wasi:keyvalue/wasi-keyvalue-error
                                        3. interface wasi:keyvalue/types
                                          4. -
                                        4. interface wasi:keyvalue/readwrite
                                          5. +
                                        5. interface wasi:keyvalue/eventual
                                        6. interface wasi:keyvalue/atomic
                                          7. -
                                        7. interface wasi:keyvalue/batch
                                          8. +
                                        8. interface wasi:keyvalue/eventual-batch
                                    @@ -585,9 +585,9 @@ If the size is unknown or unavailable, this function returns an Err(error)
                                    • result<u64, own<error>>
                                    -

                                    Import interface wasi:keyvalue/readwrite

                                    -

                                    A keyvalue interface that provides simple read and write operations.

                                    -

                                    A read/write operation is an operation that acts on a single key-value pair.

                                    +

                                    Import interface wasi:keyvalue/eventual

                                    +

                                    A keyvalue interface that provides eventually consistent CRUD operations.

                                    +

                                    A CRUD operation is an operation that acts on a single key-value pair.

                                    The value in the key-value pair is defined as a u8 byte array and the intention is that it is the common denominator for all data types defined by different key-value stores to handle data, ensuring compatibility between different @@ -732,8 +732,8 @@ and was replaced with the new value.

                                    • result<bool, own<error>>
                                    -

                                    Import interface wasi:keyvalue/batch

                                    -

                                    A keyvalue interface that provides batch operations.

                                    +

                                    Import interface wasi:keyvalue/eventual-batch

                                    +

                                    A keyvalue interface that provides eventually consistent batch operations.

                                    A batch operation is an operation that operates on multiple keys at once.

                                    Batch operations are useful for reducing network round-trip time. For example, if you want to get the values associated with 100 keys, you can either do 100 get diff --git a/keyvalue-handle-watch.md b/keyvalue-handle-watch.md index b8a8663..01f07ec 100644 --- a/keyvalue-handle-watch.md +++ b/keyvalue-handle-watch.md @@ -7,9 +7,9 @@

                                  • interface wasi:io/streams@0.2.0-rc-2023-11-10
                                  • interface wasi:keyvalue/wasi-keyvalue-error
                                  • interface wasi:keyvalue/types
                                    • -
                                  • interface wasi:keyvalue/readwrite
                                    • +
                                  • interface wasi:keyvalue/eventual
                                  • interface wasi:keyvalue/atomic
                                    • -
                                  • interface wasi:keyvalue/batch
                                    • +
                                  • interface wasi:keyvalue/eventual-batch
                                • Exports: @@ -582,9 +582,9 @@ If the size is unknown or unavailable, this function returns an Err(error)
                                  • result<u64, own<error>>
                                  -

                                  Import interface wasi:keyvalue/readwrite

                                  -

                                  A keyvalue interface that provides simple read and write operations.

                                  -

                                  A read/write operation is an operation that acts on a single key-value pair.

                                  +

                                  Import interface wasi:keyvalue/eventual

                                  +

                                  A keyvalue interface that provides eventually consistent CRUD operations.

                                  +

                                  A CRUD operation is an operation that acts on a single key-value pair.

                                  The value in the key-value pair is defined as a u8 byte array and the intention is that it is the common denominator for all data types defined by different key-value stores to handle data, ensuring compatibility between different @@ -729,8 +729,8 @@ and was replaced with the new value.

                                  • result<bool, own<error>>
                                  -

                                  Import interface wasi:keyvalue/batch

                                  -

                                  A keyvalue interface that provides batch operations.

                                  +

                                  Import interface wasi:keyvalue/eventual-batch

                                  +

                                  A keyvalue interface that provides eventually consistent batch operations.

                                  A batch operation is an operation that operates on multiple keys at once.

                                  Batch operations are useful for reducing network round-trip time. For example, if you want to get the values associated with 100 keys, you can either do 100 get diff --git a/wit/batch.wit b/wit/eventual-batch.wit similarity index 97% rename from wit/batch.wit rename to wit/eventual-batch.wit index 5108f1b..9482d8b 100644 --- a/wit/batch.wit +++ b/wit/eventual-batch.wit @@ -1,4 +1,4 @@ -/// A keyvalue interface that provides batch operations. +/// A keyvalue interface that provides eventually consistent batch operations. /// /// A batch operation is an operation that operates on multiple keys at once. /// @@ -20,7 +20,7 @@ /// with the readwrite interface. This interface does not guarantee strong consistency, /// meaning that if a write operation completes, subsequent read operations may not return /// the value that was written. -interface batch { +interface eventual-batch { /// A keyvalue interface that provides batch get operations. use types.{bucket, error, key, incoming-value, outgoing-value}; diff --git a/wit/readwrite.wit b/wit/eventual.wit similarity index 93% rename from wit/readwrite.wit rename to wit/eventual.wit index fc525da..707c47b 100644 --- a/wit/readwrite.wit +++ b/wit/eventual.wit @@ -1,6 +1,6 @@ -/// A keyvalue interface that provides simple read and write operations. +/// A keyvalue interface that provides eventually consistent CRUD operations. /// -/// A read/write operation is an operation that acts on a single key-value pair. +/// A CRUD operation is an operation that acts on a single key-value pair. /// /// The value in the key-value pair is defined as a `u8` byte array and the intention /// is that it is the common denominator for all data types defined by different @@ -18,7 +18,7 @@ /// read operations will eventually return the value that was written. In other words, /// if we pause the updates to the system, the system eventually will return /// the last updated value for read. -interface readwrite { +interface eventual { /// A keyvalue interface that provides simple read and write operations. use types.{bucket, error, incoming-value, key, outgoing-value}; diff --git a/wit/world.wit b/wit/world.wit index 37037db..fc922b6 100644 --- a/wit/world.wit +++ b/wit/world.wit @@ -7,17 +7,17 @@ package wasi:keyvalue; /// 2. Atomic `increment` and CAS (compare-and-swap) operations. /// 3. Batch operations that can reduce the number of round trips to the network. world imports { - /// The `readwrite` capability allows the component to perform CRUD - /// operations on the key-value store. - import readwrite; + /// The `eventual` capability allows the component to perform + /// eventually consistent CRUD operations on the key-value store. + import eventual; /// The `atomic` capability allows the component to perform atomic /// `increment` and CAS (compare-and-swap) operations. import atomic; - /// The `batch` capability allows the component to perform batch - /// operations that can reduce the number of round trips to the network. - import batch; + /// The `eventual-batch` capability allows the component to perform eventually + /// consistent batch operations that can reduce the number of round trips to the network. + import eventual-batch; } world keyvalue-handle-watch { From 2cedc65d8b671ee5739f27c8e59b99719b147b4f Mon Sep 17 00:00:00 2001 From: "Jiaxiao Zhou (Mossaka)" Date: Tue, 16 Jan 2024 13:16:58 -0800 Subject: [PATCH 14/14] chore(*): update the markdown files Signed-off-by: Jiaxiao Zhou (Mossaka) --- imports.md | 13 +++++++++---- keyvalue-handle-watch.md | 13 +++++++++---- 2 files changed, 18 insertions(+), 8 deletions(-) diff --git a/imports.md b/imports.md index 60ea483..d8affb7 100644 --- a/imports.md +++ b/imports.md @@ -439,7 +439,8 @@ or does not have the required permissions to perform the operation.

                                Currently, this provides only one function to return a string representation of the error. In the future, this will be extended to provide more information -about the error.

                                +about the error. +Soon: switch to resource error { ... }

                                Functions

                                [method]error.trace: func

                                Params
                                @@ -451,6 +452,7 @@ about the error.
                              • string

                              Import interface wasi:keyvalue/types

                              +

                              A generic keyvalue interface for WASI.

                              Types

                              type input-stream

                              @@ -477,7 +479,8 @@ can very depending on the specific implementation. For example,

                            • Memcached calls a collection of key-value pairs a slab
                            • Azure Cosmos DB calls a collection of key-value pairs a container
                              • -

                              In this interface, we use the term bucket to refer to a collection of key-value

                              +

                              In this interface, we use the term bucket to refer to a collection of key-value +Soon: switch to resource bucket { ... }

                              type key

                              string

                              A key is a unique identifier for a value in a bucket. The key is used to @@ -485,7 +488,8 @@ retrieve the value from the bucket.

                              resource outgoing-value

                              A value is the data stored in a key-value pair. The value can be of any type that can be represented in a byte array. It provides a way to write the value -to the output-stream defined in the wasi-io interface.

                              +to the output-stream defined in the wasi-io interface. +Soon: switch to resource value { ... }

                              type outgoing-value-body-async

                              output-stream

                              @@ -503,7 +507,8 @@ value as a list<u8>. value as an input-stream. In addition, it provides a incoming-value-size function to get the size of the value. This is useful when the value is large and the caller wants to allocate a buffer of -the right size to consume the value. +the right size to consume the value. +Soon: switch to resource incoming-value { ... }

                              type incoming-value-async-body

                              input-stream

                              diff --git a/keyvalue-handle-watch.md b/keyvalue-handle-watch.md index 01f07ec..7832961 100644 --- a/keyvalue-handle-watch.md +++ b/keyvalue-handle-watch.md @@ -436,7 +436,8 @@ or does not have the required permissions to perform the operation.

                            Currently, this provides only one function to return a string representation of the error. In the future, this will be extended to provide more information -about the error.

                            +about the error. +Soon: switch to resource error { ... }

                            Functions

                            [method]error.trace: func

                            Params
                            @@ -448,6 +449,7 @@ about the error.
                          • string

                          Import interface wasi:keyvalue/types

                          +

                          A generic keyvalue interface for WASI.

                          Types

                          type input-stream

                          @@ -474,7 +476,8 @@ can very depending on the specific implementation. For example,

                        • Memcached calls a collection of key-value pairs a slab
                        • Azure Cosmos DB calls a collection of key-value pairs a container
                          • -

                          In this interface, we use the term bucket to refer to a collection of key-value

                          +

                          In this interface, we use the term bucket to refer to a collection of key-value +Soon: switch to resource bucket { ... }

                          type key

                          string

                          A key is a unique identifier for a value in a bucket. The key is used to @@ -482,7 +485,8 @@ retrieve the value from the bucket.

                          resource outgoing-value

                          A value is the data stored in a key-value pair. The value can be of any type that can be represented in a byte array. It provides a way to write the value -to the output-stream defined in the wasi-io interface.

                          +to the output-stream defined in the wasi-io interface. +Soon: switch to resource value { ... }

                          type outgoing-value-body-async

                          output-stream

                          @@ -500,7 +504,8 @@ value as a list<u8>. value as an input-stream. In addition, it provides a incoming-value-size function to get the size of the value. This is useful when the value is large and the caller wants to allocate a buffer of -the right size to consume the value. +the right size to consume the value. +Soon: switch to resource incoming-value { ... }

                          type incoming-value-async-body

                          input-stream