Remove publicroomsapi

[kegsay]

Try to update all the places but man are there a lot - I likely missed some. I've also swapped to current-state-server where appropriate (e.g in docker which I forgot to add stuff previously).

There's a few major gotchas:

• ExternalPublicRoomsProvider is not implemented yet.
• The storage interface which was being abused in the libp2p demo no longer exists, so that code has been removed.

Questions:

• What kind of API shape do we want for "hey I want to inject some additional rooms into your published rooms response?"
• dendritejs just asked for HSes which then had fan-out/fan-in semantics when hitting those servers, with aggregations.
• libp2p has both a pubsub and a DHT form to extract additional rooms, though only pubsub was hooked up to main.go.

I propose we instead have an interface along the lines of:

type ExtraPublicRoomsProvider interface {
    Rooms() []gomatrixserverlib.PublicRoom
}

which is called every time on receipt of a /publicRooms request. We move p2p specific stuff (pubsub/fan-out) to their respective /cmd files and implement the interface instead of using the storage DB or ExternalPublicRoomsProvider in its current form.

[neilalexander]

I'm happy with the idea of implementing either an interface like ExtraPublicRoomsProvider, or a function argument which the public rooms API can call.

I'm wondering though how we should handle paginating etc with extra rooms provided?

[kegsay]

Pagination currently is best-effort in that the underlying rooms can shuffle about between calls. We also have the problem of how to sort the published rooms. How about on receiving a /publicRooms request:

• pull out rooms from DB and cache in memory the PublicRooms.
• call the function (which may take a while or not, depends on the impl) and cache those PublicRooms.
• sort and return a slice into that array.
• expire the list on receipt of a new /publicRooms request with no since parameter.

We need to expire aggressively because sytest expects the list to be up-to-date.

[neilalexander]

That could work, yep.

[kegsay]

Okay will do that next.