Initial AlignTTS implementation

[erogol]

📎 Paper abstract - https://arxiv.org/abs/2003.01950

Targeting at both high efficiency and performance, we propose AlignTTS to predict the mel-spectrum in parallel. AlignTTS is based on a Feed-Forward Transformer which generates mel-spectrum from a sequence of characters, and the duration of each character is determined by a duration predictor.Instead of adopting the attention mechanism in Transformer TTS to align text to mel-spectrum, the alignment loss is presented to consider all possible alignments in training by use of dynamic programming. Experiments on the LJSpeech dataset show that our model achieves not only state-of-the-art performance which outperforms Transformer TTS by 0.03 in mean option score (MOS), but also a high efficiency which is more than 50 times faster than real-time.

☘️ modifications in our implementation:

• Use encoder outputs to predict durations and use convolutional layers instead of transformers.
• Encoder and Decoder modules are interchangable. You can either use FFTransformers as in the paper or select alternative modules like wavenet decoder or convolutional encoder.
• We are able to train the whole model without phasing the training. But if you like you can also train with the phases as in the paper.