Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch events from multiple homeservers #5

Open
MadLittleMods opened this issue Feb 24, 2022 · 0 comments
Open

Fetch events from multiple homeservers #5

MadLittleMods opened this issue Feb 24, 2022 · 0 comments
Labels
A-archive-room-view The view to look at a room day by day in the archive A-backend Related to the backend Node.js server pieces T-Enhancement New feature or request

Comments

@MadLittleMods
Copy link
Contributor

Problem

When using the MSC2716 /batch_send endpoint, it creates events all at the same depth and are only correctly sorted between because of the stream_ordering. This works great for the local homeserver as they are all in the correct order but can get sticky when federating to other homeservers because stream_ordering is just a local number and not federated like depth. If other homeservers always backfilled in order from newest -> oldest like scrollbar in a room, everything would be fine. But...

[...] it still doesn't guarantee the same stream_ordering (and more importantly the /messages order) on the other server. For example, if a room has a bunch of history imported and someone visits a permalink to a historical message back in time, their homeserver will skip over the historical messages in between and insert the permalink as the next message in the stream_order and totally throw off the sort.

  • This will be even more the case when we add the MSC3030 jump to date API endpoint so the static archives can navigate and jump to a certain date.
  • We're solving this in the future by switching to online topological ordering and chunking which by its nature will apply retroactively to fix any inconsistencies introduced by people permalinking

-- matrix-org/synapse#11114

This is especially important for the gitter.im homeserver because we will use MSC2716 to backfill the entire message history from Gitter to Matrix as part of the sunset plan. We want the messages to be in the correct order and available correctly in the archive. Pulling directly from the gitter.im homeserver will allow us to bide our time until we get to solving graph linearization the correct way via online topological ordering.

Potential solutions

Instead of pulling from a single config.matrixServerUrl with config.matrixAccessToken, we can have a map from homeserver to access token and if a given room alias matches a homeserver we have the access token for, we can pull from it directly. This saves time from not having to join over federation in those cases but also very important for the MSC2716 problem described above.

{
  "matrixServerSecretMap": {
    "matrix.org": "xxx",
    "gitter.im": "xxx"
  }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-archive-room-view The view to look at a room day by day in the archive A-backend Related to the backend Node.js server pieces T-Enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant