Possible race condition on group creation during large bursts of events within a single group. #5085
Description
A user on sentry.io received a burst of ~7,500 events over a 15 minute period the other day. Each of these events were fingerprinted with the same fingerprint. Somehow, this resulted in 5 different groups being created at one point. Two of those groups were eventually merged into the other groups, leaving 3 groups that were created during the burst. Only one of them was finally associated with the GroupHash, and this group contained the majority of the events.
From what I can tell, several workers must have been attempting to create the group at the same time, and when the _ensure_hashes_merged method was called, it triggered merges into groups that themselves may have already been being merged (and subsequently deleted) so some of the merges triggered by the _ensure_hashes_merged method didn't complete successfully.
There's not enough logging to put together a comprehensive view of what merges were in flight during the time period and how far along they got, but this is what seems to have occurred given the final state of the records.