From d61be753f14c0aa6c4470037fa8da6d5a1e52957 Mon Sep 17 00:00:00 2001 From: Badrish Chandramouli Date: Wed, 27 Oct 2021 15:17:15 -0700 Subject: [PATCH 1/4] Create 51-remote-pubsub.md --- docs/_docs/51-remote-pubsub.md | 61 ++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 docs/_docs/51-remote-pubsub.md diff --git a/docs/_docs/51-remote-pubsub.md b/docs/_docs/51-remote-pubsub.md new file mode 100644 index 000000000..f85bac975 --- /dev/null +++ b/docs/_docs/51-remote-pubsub.md @@ -0,0 +1,61 @@ +--- +title: "Remote FASTER - Publish/Subscribe" +permalink: /docs/pubsub/ +excerpt: "Remote FASTER" +last_modified_at: 2021-10-21 +toc: false +classes: wide +--- + +FASTER now supports Publish/Subscribe with remote clients. Clients can now subscribe to keys or prefixes of keys and get all updates to the values of +the keys made by any other client. Following are the options for Publish/Subscribe: + +1. **Publish/Subscribe with KV**: +One or multiple clients can subscribe to the updates of a key or pattern stored in FASTER. Whenever there is a call for `Upsert()` or `RMW()` for the +subscribed key or pattern, all the subscribers for the key/pattern are notified of the change. + +2. **Publish/Subscribe without KV**: +One or multiple clients can subscribe to a key or pattern that is not stored in the FASTER key-value store. Whenever there is a call to `Publish()` +a key or pattern, all the subscribers for the key/pattern are notified of the change. + +The basic approach in order to use Publish/Subscribe (with and without KV) is as follows: + +## Creating Subscribe(KV)Broker + +You can create a Subscribe(KV)Broker with either fixed-size (blittable struct) Key and Value types or variable-sized (varlen) Key and Value types +similar to byte arrays. The Subscribe(KV)Broker must be created along with the `FASTERServer`, and passed to the provider. +`FasterLog` is used by the broker for storing the keys and values until they are forwarded to the subscribers. + +The method of creating a SubscribeBroker is as follows: + +```cs +var kvBroker = new SubscribeKVBroker>(new SpanByteKeySerializer(), null, true); +var broker = new SubscribeBroker>(new SpanByteKeySerializer(), null, true); +``` + +The first argument is a `IKeySerializer` used for serializing/deserializing keys for pattern-based subscriptions. Second argument is the location +of the log directory used for `FasterLog`. The last argument is a boolean, whether the `FasterLog` should start fresh, or should recover from the +previous state. + +## Subscribing to a Key / Pattern from clients: + +A `FASTERClient` can subscribe to a key or glob-pattern with the following command: + +```cs +clientSession.Subscribe(key); // Used for subscribing to a key that is not stored in FasterKV +clientSession.PSubscribe(pattern); // Used for subscribing to a glob-style pattern that is not stored in FasterKV +clientSession.SubscribeKV(key); // Used for subscribing to a key that is stored in FasterKV +clientSession.PSubscribe(pattern); // Used for subscribing to a glob-style pattern that is stored in FasterKV +``` + +The clientSession can be used to subscribe to multiple keys or patterns in the same session. Once a key or pattern is subscribed, +the clientSession cannot accept other commands (such as `Upsert()`, `RMW()`, etc) until all the keys or patterns are unsubscribed. + +## Publishing a key from a client: + +a `FASTERClient` can publish a key and value, for pushing the updated value for the key to all its subscribers either synchronously or asynchronously. +```cs +clientSession.Publish(key, value); // Used for publishing a key and value that is not stored in FasterKV, asynchronously +clientSession.PublishNow(key, value); // Used for publishing a key and value and is not stored in FasterKV, synchronously +``` +For the case of (P)SubscribeKV, the key and value is automatically pushed to the subscribers on `Upsert()` or `RMW()`. From bc365a2fdba1117241a5f47b7a50008498954494 Mon Sep 17 00:00:00 2001 From: Badrish Chandramouli Date: Wed, 27 Oct 2021 15:18:15 -0700 Subject: [PATCH 2/4] Update navigation.yml --- docs/_data/navigation.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index b106464f9..c65a3b284 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -44,6 +44,8 @@ docs: children: - title: "Basics" url: /docs/remote-basics/ + - title: "Pub-sub" + url: /docs/remote-pubsub/ - title: Developer Guide children: From 64ca2859d292ff23e5a7e58e72488d2ec3d31c65 Mon Sep 17 00:00:00 2001 From: Badrish Chandramouli Date: Wed, 27 Oct 2021 15:20:39 -0700 Subject: [PATCH 3/4] Update 51-remote-pubsub.md --- docs/_docs/51-remote-pubsub.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_docs/51-remote-pubsub.md b/docs/_docs/51-remote-pubsub.md index f85bac975..3dc1f3f96 100644 --- a/docs/_docs/51-remote-pubsub.md +++ b/docs/_docs/51-remote-pubsub.md @@ -1,7 +1,7 @@ --- title: "Remote FASTER - Publish/Subscribe" -permalink: /docs/pubsub/ -excerpt: "Remote FASTER" +permalink: /docs/remote-pubsub/ +excerpt: "Remote FASTER - Pub/Sub" last_modified_at: 2021-10-21 toc: false classes: wide From 8d422de75a5baca3d9d60a80d99cfd0f6b3bdf07 Mon Sep 17 00:00:00 2001 From: Badrish Chandramouli Date: Wed, 27 Oct 2021 15:25:07 -0700 Subject: [PATCH 4/4] Update SerializedFasterWrapper.cs --- cs/playground/AsyncStress/SerializedFasterWrapper.cs | 1 + 1 file changed, 1 insertion(+) diff --git a/cs/playground/AsyncStress/SerializedFasterWrapper.cs b/cs/playground/AsyncStress/SerializedFasterWrapper.cs index 6ed253110..f86c72d9a 100644 --- a/cs/playground/AsyncStress/SerializedFasterWrapper.cs +++ b/cs/playground/AsyncStress/SerializedFasterWrapper.cs @@ -95,6 +95,7 @@ internal async ValueTask UpdateAsync(TUpdater updater, K } } Interlocked.Add(ref pendingCount, await updater.CompleteAsync(await task.ConfigureAwait(false))); + _sessionPool.Return(session); } public void Update(TUpdater updater, Key key, Value value)