Methods for simulating multi-speaker mixtures

[desh2608]

Several new tasks such as multi-talker ASR, continuous speech separation, and speaker diarization are solved by training a neural network on simulated multi-speaker mixtures. Several new papers are looking at what are the best strategies to simulate such mixtures (e.g. by modeling turn-taking using HMMs, using statistics from real conversational data, and so on). Some of these methods from different groups are listed below:

1. https://github.com/fgnt/mms_msg (Paderborn)
2. https://github.com/BUTSpeechFIT/EEND_dataprep (BUT) --> implemented in [workflow] Multi-talker meeting simulation #929
3. https://github.com/jackdeadman/turn-taking (Sheffield)

It may be useful to implement one or more of these simulation methods in Lhotse, since such simulation constitutes an important part of the data pipeline for these tasks. I am envisioning a MixtureGenerator class that can be extended for the different techniques. It should have a fit() function to learn statistics from a provided CutSet, and a generate() function to generate the desired number of mixtures in the form of CutSets. This can either be done in eager mode or lazily at the time of data loading.

I don't have time to implement this right away, but I might take this up at some point in the future.

[desh2608]

Some more simulation methods (and papers):

• EEND (original from Hitachi): https://arxiv.org/abs/1909.06247 --> implemented in [workflow] Multi-talker meeting simulation #929
• EEND conversational (new from Hitachi): http://arxiv.org/abs/2204.11232