Gitter sunsetting: A room with lots of hidden events is slow

[andybalaam]

Spawning from element-hq/element-meta#1339

Steps to reproduce

1. Running a local Synapse
2. Connect to the Synapse and create a room
3. Insert 10K hidden events e.g. using matrix-batch-send
4. Open the room

Outcome

What did you expect?

To be able to open the room and use it

What happened instead?

Opening the room is very slow, and when it it open, the app freezes every time I press a key in the composer.

When I try to send a message, it is very slow indeed.

Operating system

Ubuntu 22.04

Browser information

Chromium Version 109.0.5414.119 (Official Build) snap (64-bit)

URL for webapp

localhost

Application version

Element version: !!UNSET!! Olm version: 3.2.12

Homeserver

localhost

Will you send logs?

No

[andybalaam]

Chromium profile:

[andybalaam]

Profile-20230210T093649.zip

[andybalaam]

Related: element-hq/element-meta#1339

[andybalaam]

This unit test may be a good starting point for investigating this, although in exactly this form it doesn't trigger the problem:

    it("should handle large numbers of hidden events quickly", () => {
        const events = [];
        for (let i = 0; i < 10000; i++) {
            events.push(
                TestUtilsMatrix.mkEvent({
                    event: true,
                    type: "unknown.event.type",
                    content: { key: "value" },
                    room: "!room:id",
                    user: "@user:id",
                    ts: 1000000 + i,
                }),
            );
        }
        render(getComponent({ events }, { showHiddenEvents: false }));
    });

You may need to make it slightly more realistic to get into the code paths that are problematic.

[andybalaam]

I recommend looking into all the other places where we call shouldShowEvent and consider caching all of them.

[jgonggrijp]

How is this "minor" or "uncommon"? Gitter is completely unusable.

[Johennes]

I'm gonna close this now that matrix-org/matrix-react-sdk#10131 is merged. Tracking of the remaining performance problems happens in element-hq/element-meta#1339.

[andybalaam]

I've got one more draft PR open that completes the work in matrix-org/matrix-react-sdk#10131 , so briefly re-opening

[andybalaam]

With matrix-org/matrix-react-sdk#10353 I've repeated the testcase above, and while it's not pleasant, Element is usable, so it's a big improvement.

I'll close this when that is merged. Please log new issues to track other performance issues with large numbers of invitations or hidden events.

[MadLittleMods]

@jryans also improved the situation with matrix-org/matrix-js-sdk#3227 💪

[MadLittleMods]

@andybalaam I'm now realizing that I'm not sure this issue was optimizing for the right thing with Gitter since member events aren't considered "hidden". And the Gitter case was mostly about a big blocks of member events.

Perhaps these optimizations were of events in general in the timeline though 🤷

[andybalaam]

Yeah, I think you're right that my testcase is not directly relevant, but I do think that the optimisation should have helped with large numbers of events even if they weren't hidden.

Does the Gitter case seem better? Awesome news that @jryans fixed another part too!

[MadLittleMods]

@andybalaam Testing on develop.element.io with one of the Gitter rooms with the block of membership, it does seem very janky still. Go to this permalink and scroll up into the block of membership to experience the slow pagination and eventually slow Element.

This might be getting into DOM limits since 10k+ DOM elements of membership avatars isn't going to do well no matter what.

In terms of membership pagination, it's still just slow 20 at a time pagination of membership which takes forever to get through. But this will take some alternative solution mentioned in element-hq/element-meta#1339 to address this aspect.