Observable API

[domfarolino]

こんにちは TAG-さん!

I'm requesting a TAG review of Observables.

This proposal adds an .on() method to EventTarget that becomes a better addEventListener(); specifically it returns a new Observable that adds a new event listener to the target when its subscribe() method is called. The Observable calls the subscriber's next() handler with each event.

Observables turn event handling, filtering, and termination, into an explicit, declarative flow that's easier to understand and compose than today's imperative version, which often requires nested calls to addEventListener() and hard-to-follow callback chains.

• Explainer (minimally containing user needs and example code): https://github.com/WICG/observable
• User research: We don't have "user research" per se, but the closest thing we have is a sense of how much developers (the direct users of this API) want it:
  • https://twitter.com/domfarolino/status/1684921351004430336 got some terrific reception
  • and https://foolip.github.io/spec-reactions/ shows that the original API proposal was the most-reacted to standards issue on GitHub
• Security and Privacy self-review: https://github.com/WICG/observable/blob/master/security-privacy-questionnaire.md
• GitHub repo (if you prefer feedback filed there): https://github.com/WICG/observable
• Primary contacts (and their relationship to the specification):
  • @domfarolino (Google), @benlesh (RxJS maintainer) — both are co-authors of the proposal
• Organization/project driving the design: Both Google (@domfarolino) and open source maintainers personal time
• External status/issue trackers for this feature: https://chromestatus.com/feature/5154593776599040

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• The group where the incubation/design work on this is being done (or is intended to be done in the future): WICG
• The group where standardization of this work is intended to be done ("unknown" if not known): WHATWG
• Existing major pieces of multi-stakeholder review or discussion of this design:
• Major unresolved issues with or opposition to this design: There are just a few big design questions remaining, see Why do some operators return Promises? WICG/observable#20, Is there any concern about the difference in constructor semantics from Promise? WICG/observable#22, Observables can be made async iterable by using a buffer WICG/observable#33
• This work is being funded by: Google I guess? Anyone contributing...

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify @domfarolino and @benlesh

[artalar]

Hi! One more opinion: https://dev.to/artalar/my-criticism-about-the-new-observables-api-37d5

People in my bubble have had a bad experience with observables (especially with merging a few streams) and are scared of the API and the complexity that it could bring to the platform.

[benlesh]

• It would introduce another way to accomplish the same thing, which would confuse beginners and make the platform more complex. We already have callbacks, promises, async/await, events, streams, and generators.

Observables are another way to accomplish a lot more than what current methods allow. So "sort of". Callbacks exist in everything else you listed. promises and async await are firmly different concepts and ECMAScript features, not related the DDOM. Events aren't really a construct, but rather something that can be handled by the other things you listed. Generators are wildly different than Observables, they're pull-based where observables are push-based. Async Generators are pull-then-push, and slightly more complicated than observables. Especially when you start talking about coroutines.

• There will always be a shortage of operators. Definitely check out rxjs.dev/api.

IMO, as the lead maintainer of the library, RxJS has too many operators, and we've been deprecating and removing them over the years.

• We are uncertain about how it should work. The popular observables library RxJS is constantly improving and changing. Its internal logic and codebase are not simple. So, why do we assume that the current proposed API will be sufficient for us in the future?

I'm very sorry, this part is just plainly false. We haven't added any new operators in years. We've removed a few operators, and a lot of the work that is going into RxJS over the last few years is aimed at simplifying the codebase. In general, outside of "pipeable" operators, RxJS has barely changed in the 10+ years of its existence.

The only thing that's really been evolving over this time period is the growing user base of the library. Despite not being funded by any major tech company. Despite not having a dedicated staff, and only being maintained by volunteers. Despite not having an advertising budget, people are choosing RxJS to help them manage coordinating events. And it's not just RxJS. If you checkout the README for the proposal, there are several wildly used projects employing an observable type in their codebase.

[LeaVerou]

Hi everyone, apologies for the delay @domfarolino.

We discussed this in a breakout today, though we ran out of time before reaching a conclusion. The following are some personal thoughts and comments.

First, the author pain points are clear, so motivation is pretty obvious.

My main concern is the standardization venue. It seems very clear that ES will eventually need an Observable primitive and we would not want the web platform to have another conflicting primitive. Also, when it comes to pure JS language features, TC39 is the clear fit for this. The fact that AbortSignal/AbortController were not standardized in TC39 is unfortunate, and shows in their ergonomics — I think it's important to avoid this pattern. You mention that Observable does to Events what Promises did to callbacks, but one of the reasons Promises were so successful is exactly that they were specced as a language feature.

Looking at tc39/proposal-observable#201 (comment) it appears that the primary issue stems exactly from AbortSignal / AbortController. FWIW although we generally recommend against monkey-patching, I think it would be the lesser of two evils to have the Observable API specced by TC39, and only a signal parameter elsewhere, than to have the entire Observable API specced by WHATWG or another browser-centric venue.

Another concern is that it seems that this still depends on the EventTarget infrastructure, and basically provides a wrapper over it with better DX. But if we go through the effort of creating a completely new API, it seems like it should not come with EventTarget baggage, an API that was designed for the DOM and later retrofitted to other objects. E.g. as a library author, I find it unfortunate that for my objects to support events that my users can listen to I need to make them inherit from EventTarget (and since we don't have multiple inheritance, this precludes them inheriting from anywhere else).

Stepping even further back, we have way too many different pub/sub patterns on the Web platform and DX suffers as a result. To an author it is unclear why we have an Events system and e.g. MutationObserver, ResizeObserver, etc. I understand the reasons, no need to explain them to me, I’m just saying most authors don't and from a UI perspective, it's an implementation detail that has leaked out into the UI because the DOM Events infrastructure did not support this use case well. For a new pub/sub pattern to win them all, we really need something that would be able to support all of these cases, and be used as the underlying model for all of them (i.e. DOM Events can become a special case of Observable, same with MutationObservers etc).

Lastly, as a more minor point than the architectural concerns above: is there 1-1 parity with what addEventListener can do? E.g. by looking at the API I could not figure out how to do an once event listener.

Also props for a nice explainer, and especially for having code examples! I also loved that some code examples had a collapsed version with how this is done currently — wish all of them had this!

[LeaVerou]

Hi @domfarolino,

We looked at this today during a breakout. We do have consensus on most of the concerns I expressed above, especially around standards venue.

Furthermore, we had some questions about the API shape that we couldn’t figure out from the explainer, and we were hoping you might be able to clarify.

It seems (though we are not 100% certain) that this is a separate primitive than EventTarget and simply integrates with it. If that is correct, the current explainer is a little unclear on what the boundaries of each are. Does it use the same Event objects? Does it register event listeners behind the scenes? How does bubbling work? Does it support bubbling to parent objects when not using the EventTarget integration?

• All examples are about listening to predefined events on existing objects. How do authors create objects that can emit events? Today they need to extend EventTarget which is suboptimal. Does Observable enable a new pattern for this?
• Do you folks have any plans to integrate with any of the other pub/sub mechanisms on the web platform or JS runtimes? E.g. all the *Observer objects, or Node’s Event emitter?
• While the code examples are certainly succinct, we had trouble figuring out how many of them worked and extrapolate how to write code about use cases not listed in the explainer. We were especially unclear on the arguments of subscribe() (what does next do? How does it differ from callback?)

We think a reworking of the explainer would really help answer these types of questions. Some (non-exhaustive) suggestions:

• Start from use cases and user needs.
• Include some simple code examples before the more pragmatic ones, so the reader can understand how the primitives involved work
• Include code showing how authors can use this to enable their own objects to emit events.
• A clear separation of what is Observable and what is EventTarget.

---

@littledan @ljharb @ctcpip @robpalme We saw you folks upvoted my last comment. Has there been any recent discussion in TC39 around this? Any plans to discuss it in the upcoming Plenary?

[ljharb]

@LeaVerou I've been suggesting to @domfarolino ever since I became aware of the browser effort that TC39 would be a better venue, but there's not been much discussion in plenary about it, since imo unless Chrome is willing to wait for the language to add Observables, our hands are kind of tied.

[robpalme]

TC39 is available as a venue with no special strings attached.

@domfarolino I believe you already belong to an Ecma member company (Google) so it will be trivial for your lead delegate (Shu) to onboard you as a TC39 delegate. Discussion can happen at any time in Matrix or in adhoc meetings organised on the Reflector. And you would be welcome to schedule the item for formal discussion in the next meeting which is July.

[littledan]

One key reason why it would be difficult to do Observables in TC39 is its use of AbortSignal as an unsubscription mechanism--which is a good API pattern in alignment with the rest of the web platform. There are other features under discussion in TC39 which may benefit from this notion of cancellation, e.g,. Signals. I see multiple ways we can work this problem out layering-wise, given some more explicit collaboration between TC39 and WHATWG. I have recently raised this AbortSignal layering topic to TC39, and have started chatting with some people within WHATWG about it; I hope to follow up in the coming months with a more clear proposal (but if someone wants to take a stab at this sooner, feel free).

I'd suggest bringing Observables to discussion in TC39 simply for a review of its current shape, focusing mostly on what the feature is for and how it serves developers, rather than focusing on its venue. In particular, I want to make sure that there's a strong, shared understanding of the relationship between Observables and Signals (they are complementary in their use cases, and we should have a well-understood, shared story around them, even if they're standardized in different venues).

[domfarolino]

Response to the early review comment

For a new pub/sub pattern to win them all, we really need something that would be able to support all of these cases, and be used as the underlying model for all of them (i.e. DOM Events can become a special case of Observable, same with MutationObservers etc).

I think this is what Observables attempts to do! It is true the initial integration point we're working on is EventTarget#when()¹ that returns an Observable over event listeners. But Observables are not, as you suggest, "dependent" on EventTarget (in that they are "limited" by EventTarget's dispatching semantics). Observables are generic; they need not vend Event objects, they can vend any kind of value. Because of this, we've thought of, and been open to, future integration with other APIs on the platform of the pub-sub sort.

Please see WICG/observable#72 which mentions MutationObserver, ResizeObserver, and more as APIs with potential Observable-vending capabilities. I don't believe our proposal in any way precludes these kinds of APIs from converging on Observables.

Lastly, as a more minor point than the architectural concerns above: is there 1-1 parity with what addEventListener can do? E.g. by looking at the API I could not figure out how to do an once event listener.

Thanks for asking, this should be adequately described in WICG/observable#66 and WICG/observable#65. Basically, the parity exists in the operators and abort mechanism.

Also props for a nice explainer, and especially for having code examples! I also loved that some code examples had a collapsed version with how this is done currently — wish all of them had this!

Thank you very much! We tried to construct it in a way to where the most visually-useful information with a high bang-for-your-buck came first.

Response to newer review comment

It seems (though we are not 100% certain) that this is a separate primitive than EventTarget and simply integrates with it. If that is correct, the current explainer is a little unclear on what the boundaries of each are. Does it use the same Event objects? Does it register event listeners behind the scenes?

It is most definitely is a separate primitive from EventTarget. Every question here should be answered in https://github.com/WICG/observable?tab=readme-ov-file#the-observable-api. It shows a custom-made Observable that has nothing to do with EventTarget or Event objects, and highlights the supported conversion from arbitrary iterables/promises/async iterables ➡️ Observables. https://github.com/WICG/observable#:~:text=While%20custom%20Observables,Observer%20handler%20functions specifically mentions how event listeners are registered behind the scenes.

How does bubbling work? Does it support bubbling to parent objects when not using the EventTarget integration?

When used with EventTargets, Observables are wrappers around event listeners. So, bubbling works the same as it would with them, since that's all controlled by DOM event dispatching infrastructure that our spec does not touch. Bubble/capture isn't a concept for Observables that are not vended by EventTargets — it's only a concept for the EventTarget method that produces Observables, to tell the underlying event infra that the event listener backing the Observable is interested in one or the other.

• All examples are about listening to predefined events on existing objects. How do authors create objects that can emit events? Today they need to extend EventTarget which is suboptimal. Does Observable enable a new pattern for this?

Some of this is already covered in https://github.com/WICG/observable?tab=readme-ov-file#the-observable-api, but our proposal does not provide a new way to emit events from within an EventTarget. It enables a new way to handle those events, by giving EventTargets the ability to vend Observables representing event listeners. Changes to EventTarget construction are outside the scope of our proposal.

• Do you folks have any plans to integrate with any of the other pub/sub mechanisms on the web platform or JS runtimes? E.g. all the *Observer objects, or Node’s Event emitter?

This was mentioned above, but yes just for completeness, see WICG/observable#72. I think lots of APIs have a potential future vending Observable objects.

While the code examples are certainly succinct, we had trouble figuring out how many of them worked and extrapolate how to write code about use cases not listed in the explainer. We were especially unclear on the arguments of subscribe() (what does next do? How does it differ from callback?)

Hmm, I was hoping https://github.com/WICG/observable#:~:text=The%20creator%20of,data%20is%20finished covered this sufficiently. Are the API shape / semantics really that opaque though? If so I suppose we could explain each input in more detail, although I would hate to duplicate too much of the IDL in the spec.

Standards venue

I personally think this is the least consequential topic regarding our proposal, and its review, so I'd like to not spend much time on it. For one, nothing precludes us from moving things into, or using things from, TC39 in the future if both communities really see this as absolutely necessary, as @littledan mentioned. AbortSignal/AbortController were standardized in WHATWG DOM, and there is some discussion today about being able to use these cancelation primitives from within TC39 world. That should be a solvable, backwards-compatible problem.

I'm not saying we'd design Observables with that path in mind. I'm saying that Observables and cancelation primitives have not gotten good footing in TC39, so they're being pursued elsewhere, and simultaneously they're not precluded from a future in TC39 world forever (should that eventually become necessary).

@LeaVerou I've been suggesting to @domfarolino ever since I became aware of the browser effort that TC39 would be a better venue, but there's not been much discussion in plenary about it, since imo unless Chrome is willing to wait for the language to add Observables, our hands are kind of tied.

I certainly hope we wouldn't have to wait another 8 years!!

@domfarolino I believe you already belong to an Ecma member company (Google) so it will be trivial for your lead delegate (Shu) to onboard you as a TC39 delegate.

Thank you, I will say though, Shu is supportive of our current path so I don't think we really need to make any sudden moves here. We're planning on standardizing it through WHATWG DOM.

Footnotes

1. At least that's what I suspect the current on() method will turn into. ↩

[ljharb]

To clarify, Observables' friction in TC39 was lack of implementer interest, from browsers at the time too; since that's changed, I see no reason it would have any difficulty in TC39 anymore - all it needs is someone to bring it back, and a good faith understanding that it wouldn't be shipped on the web outside of the staging process.

Cancelation primitives was a bit more complex and seemed more due to individual delegate misunderstandings/miscommunications than issues with TC39 itself. Also, by AbortSignal etc being designed outside of TC39, it ended up precluding itself from easy incorporation into the language, and it would be really unfortunate to repeat that mistake with Observable.