PubSub #1949
PubSub #1949
Conversation
erights
force-pushed
the
async-iter-kit
branch
from
209d3f3 to
9924fbb
Compare
|
Hi @kriskowal, @Chris-Hibbert , ready for review. Naming is hard. I renamed everything as @kriskowal and I discussed over zoom. Please read the README first to react to the new names. I'm not in love with them. Once the only remaining problem is settling the names, unless there's an obvious and agreed choice, I would prefer to land this PR first so that the name bikeshedding can then proceed at its own pace. ERTP and Zoe went through several major naming changes before settling down. |
|
I’ve taken the presumptuous liberty of renaming the PR “PubSub”, not so much because that’s clearly the right name, but because this is now in earnest review, and to avoid committing with a message with “WIP”. That would have left “async iterable kit”, which isn’t representative. |
|
@erights I like the direction this is going, but it's missing one use case that I still have that I discussed with @kriskowal but unfortunately not with you. I'd like an async iterator that produces these kinds of streams:
The idea is that a recent client state can be built from the latest I want this for long-lived produces so that they don't consume arbitrary memory, reducing their history only to the latest snap+deltas. Consumers can still be assured that they get a lossless value by requesting the latest snapshot and following the stream from there. This seems to be the more general underlying hybrid mechanism from which lossless:
and lossy:
can be built. Thoughts? (@warner @dckc thought you might like the isomorphism with recovering a vat's state from kernel snapshots + deltas.) |
There was a problem hiding this comment.
Please consider your response to my snapshots+deltas model #1949 (comment) before merging.
Co-authored-by: Michael FIG <mfig@agoric.com>
erights
force-pushed
the
async-iter-kit
branch
from
576b67c to
8b0cf33
Compare
There was a problem hiding this comment.
I think there’s a coherent story for @michaelfig’s idea should we add a solution for that case, commented inline.
I don’t suppose we could also have test cases where the producer and consumer are genuinely in separate vats, to give eventual send some inter-vat exercise?
|
@michaelfig , In response to your suggestion at #1949 (comment) , please take a look at https://github.com/Agoric/agoric-sdk/blob/async-iter-kit/packages/notifier/test/map-unum.js .
Though note the rest of the disclaimer at the top. |
| Although Bob's code is correct, it is sub-optimal. Its distributed systems | ||
| properties are not terrible, but Bob can do better using the | ||
| `getSharableSubsciptionInternals()` method provided by both NotifierKit and | ||
| SubscriptionKit. This enables consumer Bob to make a local AsyncIterable that | ||
| coordinates better with producer Paula's IterationObserver. Alice's code is | ||
| less forgiving. She's using JavaScript's `for-await-of` loop which requires a | ||
| local AsyncIterable. It cannot handle a remote reference to an AsyncIterable | ||
| at | ||
| Paula's site. Alice has no choice but to make an AsyncIterable at her site. | ||
| Using `getSharableSubsciptionInternals()` is the best way for her to do so. She | ||
| can replace her last line, the call to `consume(subscription)` with |
There was a problem hiding this comment.
I am wondering whether there might be a way to hide this concern from users. This is a lot to chew.
These docs might benefit from being overall driven by a sequence of motivating use cases with concrete domains, like the current time, progress, ETA, absolute values vs deltas.
There was a problem hiding this comment.
I am wondering whether there might be a way to hide this concern from users. This is a lot to chew.
There is --- building more direct support for Unum replication into marshal and the comm systems built on it. However, it requires some kind of name-based code registry be privileged. This also comes up for the ERTP MathHelpers. We don't yet have a worked out design for reconciling such a name-based registry with permissionless-ness, so we have postponed the issue. But we know we'll need this.
There was a problem hiding this comment.
These docs might benefit from being overall driven by a sequence of motivating use cases with concrete domains, like the current time, progress, ETA, absolute values vs deltas.
They would indeed, but please not in this PR. For now I'm trying only for precision and a certain basic level of readability. Does it seem adequate to merge in its current state?
There was a problem hiding this comment.
I'm loving map-unum.js! An independently-discovered by @erights and @kriskowal elegant composition of the tools at hand to solve the case I had in mind.
Hence, this PR LGTM.
Thanks for the kind words! It was an interesting puzzle. Very gratifying. |
|
This looks awesome! We will need some way of getting this documentation out to the users so they know what is available and how to use it. @erights could you put a link in the documentation repo or coordinate with Tom to do so? |
|
Thanks! Already sent @tyg email |
Fixes #1345
Supports a lossless stream of values through the standard JavaScript async iterable API, in a remotable manner.