Tacotron-Chinese

An implementation of Tacotron speech synthesis in TensorFlow.

Thanks for Keith It's code and Data Baker company's Corpus

I just made changes(Corpus https://www.data-baker.com/open_source.html) on his basis code https://github.com/keithito/tacotron

Background

In April 2017, Google published a paper, Tacotron: Towards End-to-End Speech Synthesis, where they present a neural text-to-speech model that learns to synthesize speech directly from (text, audio) pairs. However, they didn't release their source code or training data. This is an independent attempt to provide an open-source implementation of the model described in their paper.

The quality isn't as good as Google's demo yet, but hopefully it will get there someday :-). Pull requests are welcome!

Quick Start

Installing dependencies

1. Install Python 3.

2. Install the latest version of TensorFlow for your platform. For better performance, install with GPU support if it's available. This code works with TensorFlow 1.3 and later.

3. Install requirements:

   pip install -r requirements.txt
   

Using a pre-trained model(just English not Chinese)

1. Download and unpack a model:

   curl http://data.keithito.com/data/speech/tacotron-20180906.tar.gz | tar xzC /tmp
   

2. Run the demo server:

   python3 demo_server.py --checkpoint /tmp/tacotron-20180906/model.ckpt
   

3. Point your browser at localhost:9000

   • Type what you want to synthesize

Training

Note: you need at least 40GB of free disk space to train a model.

1. Download a speech dataset.

   The following are supported out of the box:

   • LJ Speech (Public Domain)
   • Blizzard 2012 (Creative Commons Attribution Share-Alike)
   • Bznsyp (Chinese Corpus)

   You can use other datasets if you convert them to the right format. See TRAINING_DATA.md for more info.

2. Unpack the dataset into ~/tacotron

   After unpacking, your tree should look like this for LJ Speech:

   tacotron
     |- LJSpeech-1.1
         |- metadata.csv
         |- wavs
   

   or like this for Blizzard 2012:

   tacotron
     |- Blizzard2012
         |- ATrampAbroad
         |   |- sentence_index.txt
         |   |- lab
         |   |- wav
         |- TheManThatCorruptedHadleyburg
             |- sentence_index.txt
             |- lab
             |- wav
   

   Chinese like Bznsyp:

      tacotron
     |- Bznsyp
         |- bz.txt(preprocess 000001-010000.txt)
         |- PhoneLabeling
         |   |- 000001.interval
         |- prosodyLabeling
             |- 000001-010000.txt
         |- Wave
             |- 000001.wav
   

3. Preprocess the data

   python3 preprocess.py --dataset bznsyp
   

   • Use --dataset blizzard for Blizzard data
   • Use --dataset ljspeech for LJ Speech

4. Train a model

   python3 train.py
   

   Tunable hyperparameters are found in hparams.py. You can adjust these at the command line using the --hparams flag, for example --hparams="batch_size=32,outputs_per_step=2". Hyperparameters should generally be set to the same values at both training and eval time. The default hyperparameters are recommended for LJ Speech and other English-language data. See TRAINING_DATA.md for other languages.

5. Monitor with Tensorboard (optional)

   tensorboard --logdir ~/tacotron/logs-tacotron
   

   The trainer dumps audio and alignments every 1000 steps. You can find these in ~/tacotron/logs-tacotron.

6. Synthesize from a checkpoint

   python3 demo_server.py --checkpoint ~/tacotron/logs-tacotron/model.ckpt-185000
   

   Replace "185000" with the checkpoint number that you want to use, then open a browser to localhost:9000 and type what you want to speak. Alternately, you can run eval.py at the command line:

   python3 eval.py --checkpoint ~/tacotron/logs-tacotron/model.ckpt-185000
   

   If you set the --hparams flag when training, set the same value here.

   or like:

   open terminal

   jupyter notebook
   

   open Synthesisend.ipynb

   modify some path and parameter