Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

lazy-load-members /sync blocks if an event arrives in a partial-state-room #13146

Closed
richvdh opened this issue Jun 30, 2022 · 3 comments · Fixed by #13477
Closed

lazy-load-members /sync blocks if an event arrives in a partial-state-room #13146

richvdh opened this issue Jun 30, 2022 · 3 comments · Fixed by #13477
Assignees
@squahtx
Labels
A-Federated-Join joins over federation generally suck A-Lazy-Loading Issues relating to `lazy_load_members` in `/sync` A-Sync defects related to /sync P3 (OBSOLETE: use S- labels.) Approved backlog: not yet scheduled, will accept patches T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Milestone
Q3 2022: Faster j...

Comments

@richvdh
Copy link
Member

richvdh commented Jun 30, 2022

lazy-load-members /sync should return new events immediately, without waiting for full state.
@richvdh richvdh added this to the Faster joins (further work) milestone Jun 30, 2022
@squahtx squahtx added A-Federated-Join joins over federation generally suck T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. A-Sync defects related to /sync A-Lazy-Loading Issues relating to `lazy_load_members` in `/sync` P3 (OBSOLETE: use S- labels.) Approved backlog: not yet scheduled, will accept patches T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. and removed T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. labels Jul 12, 2022
@squahtx squahtx self-assigned this Jul 26, 2022
@squahtx
Copy link
Contributor

squahtx commented Jul 27, 2022

We block waiting for full state of new events here:

2022-07-27 19:42:43,803 - synapse.storage.util.partial_state_events_tracker - 78 - INFO - GET-8 - Awaiting un-partial-stating of events ['$Z91x7vE6bB1EEdbdRR1gZOsYDk-I0tOayX11eEtKK6I']
Stack (most recent call last):
  [... lots of Twisted stack frames ...]
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1750, in gotResult
    current_context.run(_inlineCallbacks, r, gen, status)
  File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "/usr/local/lib/python3.9/site-packages/synapse/util/async_helpers.py", line 227, in _concurrently_execute_inner
    await maybe_awaitable(func(value))
  File "/usr/local/lib/python3.9/site-packages/synapse/handlers/sync.py", line 1562, in handle_room_entries
    await self._generate_room_entry(
  File "/usr/local/lib/python3.9/site-packages/synapse/handlers/sync.py", line 2072, in _generate_room_entry
    state = await self.compute_state_delta(
  File "/usr/local/lib/python3.9/site-packages/synapse/handlers/sync.py", line 1004, in compute_state_delta
    state_ids = await self._state_storage_controller.get_state_ids_for_event(
  File "/usr/local/lib/python3.9/site-packages/synapse/storage/controllers/state.py", line 307, in get_state_ids_for_event
    state_map = await self.get_state_ids_for_events(
  File "/usr/local/lib/python3.9/site-packages/synapse/storage/controllers/state.py", line 252, in get_state_ids_for_events
    event_to_groups = await self.get_state_group_for_events(
  File "/usr/local/lib/python3.9/site-packages/synapse/storage/controllers/state.py", line 343, in get_state_group_for_events
    await self._partial_state_events_tracker.await_full_state(event_ids)
  File "/usr/local/lib/python3.9/site-packages/synapse/storage/util/partial_state_events_tracker.py", line 78, in await_full_state
    logger.info(

@squahtx squahtx mentioned this issue Aug 10, 2022
Merged
@squahtx
Copy link
Contributor

squahtx commented Aug 10, 2022

We block waiting for full state of new events here:

This is only true for incremental lazy-loading /syncs. Initial and gappy /syncs also block inside compute_state_delta.

@squahtx
Copy link
Contributor

squahtx commented Aug 10, 2022 •
edited
Loading

Synapse:

Complement:

squahtx added a commit that referenced this issue Aug 18, 2022
@squahtx
Avoid blocking lazy-loading /syncs during partial joins (#13477)
84169a8 
Use a state filter or accept partial state in a few places where we
request state, to avoid blocking.

To make lazy-loading `/sync`s work, we need to provide the memberships
of event senders, which are not guaranteed to be in the room state.
Instead we dig through auth events for memberships to present to
clients. The auth events of an event are guaranteed to contain a
passable membership event, otherwise the event would have been rejected.

Note that this only covers the common code paths encountered during
testing. There has been no exhaustive checking of all sync code paths.

Fixes #13146.

Signed-off-by: Sean Quah <seanq@matrix.org>
@squahtx squahtx reopened this Aug 18, 2022
@squahtx squahtx closed this as completed Sep 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@squahtx squahtx

Labels
A-Federated-Join joins over federation generally suck A-Lazy-Loading Issues relating to `lazy_load_members` in `/sync` A-Sync defects related to /sync P3 (OBSOLETE: use S- labels.) Approved backlog: not yet scheduled, will accept patches T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Projects
None yet
Milestone
Q3 2022: Faster joins: fix major know...
Development

Successfully merging a pull request may close this issue.

Avoid blocking lazy-loading /syncs during partial joins matrix-org/synapse
2 participants
@richvdh @squahtx