Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial AlignTTS implementation #398

Closed
wants to merge 27 commits into from
Closed

Initial AlignTTS implementation #398

wants to merge 27 commits into from

Conversation

erogol
Copy link
Member

@erogol erogol commented Mar 23, 2021

📎 Paper abstract - https://arxiv.org/abs/2003.01950

Targeting at both high efficiency and performance, we propose AlignTTS to predict the mel-spectrum in parallel. AlignTTS is based on a Feed-Forward Transformer which generates mel-spectrum from a sequence of characters, and the duration of each character is determined by a duration predictor.Instead of adopting the attention mechanism in Transformer TTS to align text to mel-spectrum, the alignment loss is presented to consider all possible alignments in training by use of dynamic programming. Experiments on the LJSpeech dataset show that our model achieves not only state-of-the-art performance which outperforms Transformer TTS by 0.03 in mean option score (MOS), but also a high efficiency which is more than 50 times faster than real-time.

☘️ modifications in our implementation:

  • Use encoder outputs to predict durations and use convolutional layers instead of transformers.
  • Encoder and Decoder modules are interchangable. You can either use FFTransformers as in the paper or select alternative modules like wavenet decoder or convolutional encoder.
  • We are able to train the whole model without phasing the training. But if you like you can also train with the phases as in the paper.

@erogol erogol changed the base branch from main to dev March 23, 2021 15:03
@erogol erogol mentioned this pull request Mar 23, 2021
9 tasks
@erogol erogol closed this Apr 2, 2021
@erogol erogol deleted the aligntts branch April 8, 2021 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant