Asynchronous unsubscribe method

[GregRos]

Feature Request

Brief

I want Subscriptions to support an async unsubscribe method, i.e. one that returns a Promise, and I want some operators behave differently to accomodate this.

The problem

I have a cold observable base that creates a new resource on subscription and disposes of it on unsubscribe.

let base = Observable.create(async sub => {
    let res = Resource.create();
    res.start().then(x => {
        sub.next(x);
    }, err => {
        sub.error(err);
    });
    
    return {
        async unsubscribe() {
            await res.dispose();
        }
    }
});

The thing is that res.dispose() is async because it sends a request to a server, which is required to properly dispose of the resource.

Then, I use the base observable to create a dependent observable:

let derived = base.flatMap(res => {
    let dres = DerivedResource.create(res);
    return Observable.fromPromise(dres.start()).finalize(async () => {
        await dres.dispose();
    });
})

In rxjs, if I close the subscription to derived, first dres.dispose() will be called and only then will base.dispose() will be called.

Although I have not seen this in the documentation, it is the intuitive behavior and it does behave like this in practice.

However, in this case, the disposal is asynchronous. Rxjs doesn't support async disposal, so the two calls can effectively happen in parallel, and base might be disposed before derived, which is invalid behavior (derived is still using base at this time, after all)

Describe the solution you'd like

I want rxjs to allow for async disposal in order to make sure this kind of disposal works properly. This means awaiting on some unsubscribe calls before launching some new ones, depending on the operator.

An operator such as flatMap can first dispose of the subscriptions to the inner observables simultaneously using Promise.all, and only then dispose of the source sequence.

I can provide more detailed behavior changes if this suggestion gains any support/feedback.

The unsubscribe function will return void | Promise<void>, which will also allow the caller to determine when the subscription has been truly disposed.

Describe alternatives you've considered

I can't think of any other good solutions to the problem. If you can, please share them.

Other situations where this is helpful

1. It will be possible to signal to the caller that an async failure happened while unsubscribing. Right now, it's hard to say what should be done with a promise rejection in the unsubscribe method.

2. As a corollary of (1), operators can use this knowledge to improve their behavior. For example, if disposing of a resource is async and takes time, a retry or similar operator should wait until disposal of the previous resource has finished before trying to create another one. This may be desirable if, for example, two resources of the same arguments cannot coexist.

[kwonoj]

I believe I've seen this discussion in somewhere, but my github search failed to find specific issues. 😭

[cartant]

@kwonoj I could remember saying something about it, too, but could not find it. Then I recalled that it might have been on Stack Overflow. And it is:

https://stackoverflow.com/q/52368064/6680611

With the same OP, too.

I think that adding support for unsubscribe to return a promise would be such a fundamental change that it's simply not going to happen. I also have a feeling that doing so would not be in line with the Observable Contract.

However, I do think that this could be done with a dedicated operator - or, perhaps, by allowing the callback passed to finalize to return a promise.

I'll have a think about it.

[kwonoj]

return a promise would be such a fundamental change that it's simply not going to happen

this part I mostly agree. Some way to allow teardown asynchronously would be useful in some special cases definitely though.

Has there been discussion around this on tc-39 or whatwg proposals? eager introduction to these kind change might going to create some diverges I presume.

[cartant]

Has there been discussion around this on tc-39 or whatwg proposals?

Not that I'm aware of.

eager introduction to these kind change might going to create some diverges I presume.

I don't think any changes to the fundamentals are needed. It should be possible to have an asynchronous teardown facilitated by a user-land operator. finalize could even be tweaked to do it, I think. (Although, I've not had a coffee yet, this morning, so I could be totally mistaken.)

[GregRos]

Has there been discussion around this on tc-39 or whatwg proposals? eager introduction to these kind change might going to create some diverges I presume.

I should note that some other libraries for functional reactive programming, such as most.js, do allow for async disposal.

However, I do think that this could be done with a dedicated operator - or, perhaps, by allowing the callback passed to finalize to return a promise.

How would it work? Would it solve the example I gave?

You could have an async finalize queue associated with a subscription in addition to parent subscriptions/etc. A special function or a different property might allow retrieving a promise for the queue using the Subscription object and then you could await it, e.g. finalized(subscription) or sub.finalized. It would be a lot more elegant to use the existing unsubscribe method, but if you don't want to modify it, this other solution might be possible.

It would basically create an additional stage in the lifecycle of a subscription, the finalization stage.

I don't know if it would be possible to conserve the queue when piping to operators that aren't aware of its existence.

@kwonoj I could remember saying something about it, too, but could not find it. Then I recalled that it might have been on Stack Overflow. And it is:

https://stackoverflow.com/q/52368064/6680611

With the same OP, too.

Since the answer to the question appeared to be "no," I decided to make it a feature request. I also decided to give the full example instead of the watered down version I gave in the SO question.

[cartant]

@GregRos There are two problems:

• waiting for async disposal; and
• the order of unsubscriptions for dependent resources.

The first is solvable, but I'm not sure about the second. The order in which unsubscriptions occur depends upon whether a source completes or errors or whether the unsubscription is initiated explicitly from the destination.

I'm increasingly of the opinion that if you have dependent resources, you should not be relying upon unsubscription ordering to manage them.

I'm still thinking about this. The deferFinalize operator with which I've been experimenting is here. However, I'm not sure it's something that should be encouraged.

[cartant]

Also, in v6, the your dispose functions will be called in a different order and the base dispose will be called before the derived dispose.

If you have resources that have to be torn down in a specific order, you should not rely upon the unsubscription order to do so.

[GregRos]

@cartant Is there any reason for that? It's very counter-intuitive. Teardown is supposed to happen in the reverse order of creation. So when using flatMap, you first:

1. Subscribe to base, and then
2. Subscribe to the return of flatMap, which has the finalize.

When you unsubscribe, you're supposed to:

1. Unsubscribe from the result of flatMap, and then
2. Unsubscribe from base.

Since async creation of a resource is possible, async disposal of a resource also makes sense. But isn't possible right now.

[cartant]

Teardown is supposed to happen in the reverse order of creation.

This is not true. There are other circumstances in which teardown won't happen in reverse order of creation:

• When a source observable completes or errors, it will be torn down before any flat-mapped inner observables are torn down.
• If the source observable emits values a, b and c, there will be three inner observables. And when they are torn down, it will not happen in reverse order of creation: the inner observable created from the a emission will be torn down first, etc.

Since async creation of a resource is possible, async disposal of a resource also makes sense.

Any asynchronous creation of a resource is contained within an observable's implementation. As shown in the linked operator, it's also possible for an observable's implementation to ensure any asynchronous teardown is performed before emitting a complete or error notifications and before unsubscribing from its source. So that's not the problem.

The problem is your relying upon unsubscription order and, AFAICT, that order is not guaranteed by the contract. It's not even guaranteed that observables will cease emitting notifications upon unsubscription:

When an observer issues an Unsubscribe notification to an Observable, the Observable will attempt to stop issuing notifications to the observer. It is not guaranteed, however, that the Observable will issue no notifications to the observer after an observer issues it an Unsubscribe notification.

[GregRos]

This is not true. There are other circumstances in which teardown won't happen in reverse order of creation:

When a source observable completes or errors, it will be torn down before any flat-mapped inner observables are torn down.
If the source observable emits values a, b and c, there will be three inner observables. And when they are torn down, it will not happen in reverse order of creation: the inner observable created from the a emission will be torn down first, etc.

I guess if the order in which subscriptions are closed is undefined, then my core assumption really is wrong. With this in mind, we should really put async disposal aside for now and just talk a bit more about the order of closing subscriptions.

The intuition when acquiring disposable resources, is that they will be disposed in the order opposite to how they were acquired. This is pretty much universal whenever there is a using or similar construct for acquiring a resource (C#, F#, C++ destructors, etc).

In the same way, the intuition is that when you close a subscription S to an observable O, where S maintains subscriptions to another set of observables, then all parent subscriptions of S should be closed opposite to the order in which they were acquired.

This doesn't concern, for example, one of the parent subscriptions completing or erroring and thus being torn down (which you mentioned in your example). Then the natural assumption is that the error or completion is propagated forward.

The problem is your relying upon unsubscription order and, AFAICT, that order is not guaranteed by the contract. It's not even guaranteed that observables will cease emitting notifications upon unsubscription:

The observable contract doesn't talk about how operators are implemented at all. It doesn't even talk about parent subscriptions. This is where rxjs can make its guarantees about the operators it defines.

Also, the order of disposal isn't related to whether an observable will emit messages once unsubscribed.

I can't see a technical reason for why the order would be undefined. It's always possible to make this guarantee (or some other guarantee if you like). Unlike the observable contract, which is designed to be as minimal as possible so that it would be easy to implement, rxjs is an implementation and the more (useful) guarantees it makes the better.

[cartant]

Op> This is pretty much universal whenever there is a using or similar construct for acquiring a resource (C#, F#, C++ destructors, etc).

My understanding is that using - at least in C# - is a stack-frame based construct. This is not at all like the flatMap situation. The source does not encapsulate or own the inner observables; its notifications have effected them, that's all.

It would be possible to have the subscriptions to the inner observables unsubscribed before the source observable upon explicit unsubscription. All that would be needed would be for the inner subscriptions to be added to the destination subscription's array in such a was that they are added ahead of the source subscription.

However, I'm not sure doing so is especially useful, as, IIRC, the source will be unsubscribed first if it completes or errors. And I'm deeply suspicious of any use case that precludes observables from emitting complete or error notifications because doing so would see subscriptions unsubscribed in an unwanted order.

I'm also not convinced that guarantees regarding unsubscription ordering can be made, as operators can control when subscription and unsubscription occurs. RxJS is able to guarantee that observables won't emit after a complete or error notification, but I don't see how unsubscription order could be anything more than a guideline.

[GregRos]

My understanding is that using - at least in C# - is a stack-frame based construct. This is not at all like the flatMap situation. The source does not encapsulate or own the inner observables; its notifications have effected them, that's all.

I don't really see how using is related to stack frames, except that both are examples of stacks in general. It's just a pattern.

I'm actually not just talking about the flatMap situation. I'm talking about the general situation of the order in which owned "parent" (i.e. dependency) subscriptions are closed in a "child" (i.e. dependent) subscription.

Anyway, while the source does not own inner observables, we're not talking about inner observables. We're specifically talking about subscriptions, which definitely are owned and encapsulated by child subscriptions (for our definition of child subscription), and (as is already the case), a child subscription is responsible for closing its parent subscriptions which it explicitly acquired.

It would be possible to have the subscriptions to the inner observables unsubscribed before the source observable upon explicit unsubscription. All that would be needed would be for the inner subscriptions to be added to the destination subscription's array in such a was that they are added ahead of the source subscription.

Yup, this behavior is exactly what I want. All I'm talking about is that, in the event a deterministic disposal is called for, owned subscriptions will be disposed in a deterministic order.

I'm also not convinced that guarantees regarding unsubscription ordering can be made, as operators can control when subscription and unsubscription occurs. RxJS is able to guarantee that observables won't emit after a complete or error notification, but I don't see how unsubscription order could be anything more than a guideline.

I'm not talking about some kind of construct in the code that will make sure operators behave correctly and crash them if they don't. An operator is free to open subscriptions and never close them at all, at least technically. But a well-behaved operator probably shouldn't do that.

What I'm hoping is for the documentation say that "the order of deterministic disposal is X," and for the operators provided in the library to work this way. That is, for the rxjs-specific operator contract to say this. If someone writes an operator that doesn't work that way (somewhere else), well, it just won't precisely fit the contract.

(Actually, come to think of it, I don't even remember anything in the documentation saying that dependency/parent subscriptions will be closed, or talking about when unsubscribe is actually called, but maybe I just couldn't find it).

[cartant]

Actually, come to think of it, I don't even remember anything in the documentation saying that dependency/parent subscriptions will be closed, or talking about when unsubscribe is actually called, but maybe I just couldn't find it.

I'll give your comment more thought tomorrow - I'm off to bed - but regarding the above, have a look at the current implementation of shareReplay - or search for the open issues that reference it. It ignores explicit unsubscription!

[GregRos]

I'll give your comment more thought tomorrow - I'm off to bed

That's alright. Thanks for your time - and goodnight! 😄

have a look at the current implementation of shareReplay - or search for the open issues that reference it. It ignores explicit unsubscription!

Oh. That's a bummer. But I see people do think it's an issue, so I guess there is some consensus that it should happen?

I think an equally big problem is there being no information about the behavior in the docs.

[DanielKucal]

Any news about it?

[cartant]

Any news about it?

I added an item to my to-do list to write up an explanation for how subscription and unsubscription works, but didn't get around to it. And here we are, a year later, and it's still on my list.

[backbone87]

according to http://reactivex.io/documentation/contract.html it would be valid to emit notifications to observers after unsubscribe. so when calling unsubscribe, the subscription could just block/ignore/not forward subsequent next calls from the observable (which is actually more like a generator, while the actual observable is the subscription?) and inform the observable (generator) to teardown and eventually emit a complete/error notification, that the subscriber will forward and closing itself afterwards.

the agency over "ordered" unsubscription would be put into the observable's (generator) control, since it may delay its final complete/error until its source/all its sources have issued their complete/error. it could also just escalate the teardown notification and complete immediately, letting the sources dangling while they teardown (errors during their teardown would be uncatchable?)

[TrejGun]

Hey guys! We need this functionality too, please add