Search_works_across_an_upgraded_room_and_its_predecessor is flaky

[DMRobertson]

https://github.com/matrix-org/synapse/actions/runs/3081667619/jobs/4982001564

Ported from sytest in #464

[H-Shay]

Should this be tracked here or in complement? I didn't see any notes about this in the minutes, was there a decision made about it?

[DMRobertson]

Minutes updated, but TL;DR

• if we suspect a Synapse bug/race, raise in Synapse
• if we think a test is just plain wrong, raise in complement

[DMRobertson]

https://github.com/matrix-org/synapse/actions/runs/3110974464/jobs/5042735159#step:4:5929

[richvdh]

The problem here is that the complement test is assuming that the search results will be returned in a particular order, which I don't think is correct (and certainly the sytest that it was ported from did not make that assumption).

The problem is, I'm not really sure how best to express that in Complement.

[richvdh]

if we think a test is just plain wrong, raise in complement

Moving to the complement repo, then.

[richvdh]

https://github.com/matrix-org/synapse/actions/runs/3130081166/jobs/5079975834

[DMRobertson]

Another one here: https://github.com/matrix-org/synapse/actions/runs/3126059365/jobs/5071114994