forward extremities are leaked when their successors are persisted as outliers and later de-outliered

[richvdh]

Consider a sequence of events

A <- B <- C <- D

Starting with A, it is the only forward extremity in the DAG.

Then, suppose we receive D, and for whatever reason, the missing-events algorithm ends up retrieving B and C, and B is persisted as an outlier. We now have two forward extremities: A and D.

[B can be an outlier even though we have all its predecessors and successors: it means that we haven't resolved the state at that point in the graph.]

Later, we might de-outlier B due to a backfill request. When this happens, A remains a forward extremity.

[richvdh]

This seems to be a feature rather than a bug.

Then, suppose we receive D, and for whatever reason, the missing-events algorithm ends up retrieving B and C, and B is persisted as an outlier. We now have two forward extremities: A and D.

As part of this operation, we'll have asked the remote server what the state is at B, and our concept of the state at D is based entirely on what the remote server told us the state was at D. That's ok so far, because A and D are still forward-extremities, so our concept of the room state will be the state-resolution between A and D.

Later, we might de-outlier B due to a backfill request. When this happens, A remains a forward extremity.

... if this did not happen, D would be our only forward extremity, which means the current state of the room would be that at D, ie completely controlled by the remote server.

So currently, this is safe... but it seems to only be safe because of this "bug".

We may need to change the definition of "forward extremity" from "an event with no known successors" to "an event with no known (successors which we didn't backfill)".