v0.8.0 (#1810)

* Fix checkpointing GAN models (#1641)

* checkpoint sae step crash fix

* checkpoint save step crash fix

* Update gan.py

updated requested changes

* crash fix

* Fix the --model_name and --vocoder_name arguments need a <model_type> element (#1469)

Co-authored-by: Eren Gölge <erogol@hotmail.com>

* Fix Publish CI (#1597)

* Try out manylinux

* temporary removal of useless pipeline

* remove check and use only manylinux

* Try --plat-name

* Add install requirements

* Add back other actions

* Add PR trigger

* Remove conditions

* Fix sythax

* Roll back some changes

* Add other python versions

* Add test pypi upload

* Add username

* Add back __token__ as username

* Modify name of entry to testpypi

* Set it to release only

* Fix version checking

* Fix tokenizer for punc only (#1717)

* Remove redundant config field

* Fix SSIM loss

* Separate loss tests

* Fix BCELoss adressing  #1192

* Make style

* Add durations as aux input for VITS (#1694)

* Add durations as aux input for VITS

* Make style

* Fix tts_tests

* Fix test_get_aux_input

* Make lint

* feat: updated recipes and lr fix (#1718)

- updated the recipes activating more losses for more stable training
- re-enabling guided attention loss
- fixed a bug about not the correct lr fetched for logging

* Implement VitsAudioConfig (#1556)

* Implement VitsAudioConfig

* Update VITS LJSpeech recipe

* Update VITS VCTK recipe

* Make style

* Add missing decorator

* Add missing param

* Make style

* Update recipes

* Fix test

* Bug fix

* Exclude tests folder

* Make linter

* Make style

* Fix device allocation

* Fix SSIM loss correction

* Fix aux tests (#1753)

* Set n_jobs to 1 for resample script

* Delete resample test

* Set n_jobs 1 in vad test

* delete vad test

* Revert "Delete resample test"

This reverts commit bb7c846.

* Remove tests with resample

* Fix for FloorDiv Function Warning (#1760)

* Fix for Floor Function Warning

Fix for Floor Function Warning

* Adding double quotes to fix formatting

Adding double quotes to fix formatting

* Update glow_tts.py

* Update glow_tts.py

* Fix type in download_vctk.sh (#1739)

typo in comment

* Update decoder.py (#1792)

Minor comment correction.

* Update requirements.txt (#1791)

Support for #1775

* Update README.md (#1776)

Fix typo in different and code sample

* Fix & update WaveRNN vocoder model (#1749)

* Fixes KeyError bug. Adding logging to dashboard.

* Make pep8 compliant

* Make style compliant

* Still fixing style

* Fix rand_segment edge case (input_len == seg_len - 1)

* Update requirements.txt; inflect==5.6 (#1809)

New inflect version (6.0) depends on pydantic which has some issues irrelevant to 🐸 TTS. #1808 
Force inflect==5.6 (pydantic free) install to solve dependency issue.

* Update README.md; download progress bar in CLI. (#1797)

* Update README.md

- minor PR
- added model_info usage guide based on #1623 in README.md .

* "added tqdm bar for model download"

* Update manage.py

* fixed style

* fixed style

* sort imports

* Update wavenet.py (#1796)

* Update wavenet.py

Current version does not use "in_channels" argument. 
In glowTTS, we use normalizing flows and so "input dim" == "ouput dim" (channels and length). So, the existing code just uses hidden_channel sized tensor as input to first layer as well as outputs hidden_channel sized tensor. 
However, since it is a generic implementation, I believe it is better to update it for a more general use.

* "in_channels -> hidden_channels"

* Adjust default to be able to process longer sentences (#1835)

Running `tts --text "$text" --out_path …` with a somewhat longer
sentences in the text will lead to warnings like “Decoder stopped with
max_decoder_steps 500” and the sentences just being cut off in the
resulting WAV file.

This happens quite frequently when feeding longer texts (e.g. a blog
post) to `tts`. It's particular frustrating since the error is not
always obvious in the output. You have to notice that there are missing
parts. This is something other users seem to have run into as well [1].

This patch simply increases the maximum number of steps allowed for the
tacotron decoder to fix this issue, resulting in a smoother default
behavior.

[1] mozilla/TTS#734

* Fix language flags generated by espeak-ng phonemizer (#1801)

* fix language flags generated by espeak-ng phonemizer

* Style

* Updated language flag regex to consider all language codes alike

* fix get_random_embeddings --> get_random_embedding (#1726)

* fix get_random_embeddings --> get_random_embedding

function typo leads to training crash, no such function

* fix typo

get_random_embedding

* Introduce numpy and torch transforms (#1705)

* Refactor audio processing functions

* Add tests for numpy transforms

* Fix imports

* Fix imports2

* Implement bucketed weighted sampling for VITS (#1871)

* Update capacitron_layers.py (#1664)

crashing because of dimension miss match   at line no. 57
[batch, 256] vs [batch , 1, 512]
enc_out = torch.cat([enc_out, speaker_embedding], dim=-1)

* updates to dataset analysis notebooks for compatibility with latest version of TTS (#1853)

* Fix BCE loss issue (#1872)

* Fix BCE loss issue

* Remove import

* Remove deprecated files (#1873)

- samplers.py is moved
- distribute.py is replaces by the 👟Trainer

* Handle when no batch sampler (#1882)

* Fix tune wavegrad (#1844)

* fix imports in tune_wavegrad

* load_config returns Coqpit object instead None

* set action (store true) for flag "--use_cuda"; start to tune if module is running as the main program

* fix var order in the result of batch collating

* make style

* make style with black and isort

* Bump up to v0.8.0

* Add new DE Thorsten models (#1898)

- Tacotron2-DDC
- HifiGAN vocoder

Co-authored-by: manmay nakhashi <manmay.nakhashi@gmail.com>
Co-authored-by: camillem <camillem@users.noreply.github.com>
Co-authored-by: WeberJulian <julian.weber@hotmail.fr>
Co-authored-by: a-froghyar <adamfroghyar@gmail.com>
Co-authored-by: ivan provalov <iprovalo@yahoo.com>
Co-authored-by: Tsai Meng-Ting <sarah13680@gmail.com>
Co-authored-by: p0p4k <rajiv.punmiya@gmail.com>
Co-authored-by: Yuri Pourre <yuripourre@users.noreply.github.com>
Co-authored-by: vanIvan <alfa1211@gmail.com>
Co-authored-by: Lars Kiesow <lkiesow@uos.de>
Co-authored-by: rbaraglia <baraglia.r@live.fr>
Co-authored-by: jchai.me <jreus@users.noreply.github.com>
Co-authored-by: Stanislav Kachnov <42406556+geth-network@users.noreply.github.com>

.github/workflows/pypi-release.yml

@@ -42,16 +42,18 @@ jobs:
  - uses: actions/setup-python@v2
  with:
  python-version: ${{ matrix.python-version }}
- - run: |
+ - name: Install pip requirements
+ run: |
  python -m pip install -U pip setuptools wheel build
- - run: |
- python -m build
- - run: |
- python -m pip install dist/*.whl
+ python -m pip install -r requirements.txt
+ - name: Setup and install manylinux1_x86_64 wheel
+ run: |
+ python setup.py bdist_wheel --plat-name=manylinux1_x86_64
+ python -m pip install dist/*-manylinux*.whl
  - uses: actions/upload-artifact@v2
  with:
  name: wheel-${{ matrix.python-version }}
- path: dist/*.whl
+ path: dist/*-manylinux*.whl
  publish-artifacts:
  runs-on: ubuntu-20.04
  needs: [build-sdist, build-wheels]

MANIFEST.in

@@ -11,4 +11,5 @@ recursive-include TTS *.md
 recursive-include TTS *.py
 recursive-include TTS *.pyx
 recursive-include images *.png
-
+recursive-exclude tests *
+prune tests*

README.md

@@ -130,7 +130,7 @@ pip install -e .[all,dev,notebooks] # Select the relevant extras
 If you are on Ubuntu (Debian), you can also run following commands for installation.
 
 ```bash
-$ make system-deps # intended to be used on Ubuntu (Debian). Let us know if you have a diffent OS.
+$ make system-deps # intended to be used on Ubuntu (Debian). Let us know if you have a different OS.
 $ make install
 ```
 
@@ -145,25 +145,61 @@ If you are on Windows, 👑@GuyPaddock wrote installation instructions [here](ht
  ```
  $ tts --list_models
  ```
-
+- Get model info (for both tts_models and vocoder_models):
+ - Query by type/name:
+ The model_info_by_name uses the name as it from the --list_models. 
+ ```
+ $ tts --model_info_by_name "<model_type>/<language>/<dataset>/<model_name>"
+ ```
+ For example:
+
+ ```
+ $ tts --model_info_by_name tts_models/tr/common-voice/glow-tts
+ ```
+ ```
+ $ tts --model_info_by_name vocoder_models/en/ljspeech/hifigan_v2
+ ```
+ - Query by type/idx:
+ The model_query_idx uses the corresponding idx from --list_models. 
+ ```
+ $ tts --model_info_by_idx "<model_type>/<model_query_idx>"
+ ```
+ For example:
+
+ ```
+ $ tts --model_info_by_idx tts_models/3 
+ ```
+  
 - Run TTS with default models:
 
  ```
- $ tts --text "Text for TTS"
+ $ tts --text "Text for TTS" --out_path output/path/speech.wav
  ```
 
 - Run a TTS model with its default vocoder model:
 
  ```
- $ tts --text "Text for TTS" --model_name "<language>/<dataset>/<model_name>
+ $ tts --text "Text for TTS" --model_name "<model_type>/<language>/<dataset>/<model_name>" --out_path output/path/speech.wav
+ ```
+ For example:
+
+ ```
+ $ tts --text "Text for TTS" --model_name "tts_models/en/ljspeech/glow-tts" --out_path output/path/speech.wav
  ```
 
 - Run with specific TTS and vocoder models from the list:
 
  ```
- $ tts --text "Text for TTS" --model_name "<language>/<dataset>/<model_name>" --vocoder_name "<language>/<dataset>/<model_name>" --output_path
+ $ tts --text "Text for TTS" --model_name "<model_type>/<language>/<dataset>/<model_name>" --vocoder_name "<model_type>/<language>/<dataset>/<model_name>" --out_path output/path/speech.wav
  ```
 
+ For example:
+
+ ```
+ $ tts --text "Text for TTS" --model_name "tts_models/en/ljspeech/glow-tts" --vocoder_name "vocoder_models/en/ljspeech/univnet" --out_path output/path/speech.wav
+ ```
+
+
 - Run your own TTS model (Using Griffin-Lim Vocoder):
 
  ```

TTS/.models.json

@@ -215,6 +215,14 @@
  "author": "@thorstenMueller",
  "license": "apache 2.0",
  "commit": "unknown"
+ },
+ "tacotron2-DDC": {
+ "github_rls_url": "https://coqui.gateway.scarf.sh/v0.8.0_models/tts_models--de--thorsten--tacotron2-DDC.zip",
+ "default_vocoder": "vocoder_models/de/thorsten/hifigan_v1",
+ "description": "Thorsten-Dec2021-22k-DDC",
+ "author": "@thorstenMueller",
+ "license": "apache 2.0",
+ "commit": "unknown"
  }
  }
  },
@@ -460,6 +468,13 @@
  "author": "@thorstenMueller",
  "license": "apache 2.0",
  "commit": "unknown"
+ },
+ "hifigan_v1": {
+ "github_rls_url": "https://coqui.gateway.scarf.sh/v0.8.0_models/vocoder_models--de--thorsten--hifigan_v1.zip",
+ "description": "HifiGAN vocoder model for Thorsten Neutral Dec2021 22k Samplerate Tacotron2 DDC model",
+ "author": "@thorstenMueller",
+ "license": "apache 2.0",
+ "commit": "unknown"
  }
  }
  },

TTS/VERSION

@@ -1 +1 @@
-0.7.1
+0.8.0

TTS/bin/synthesize.py

@@ -60,13 +60,13 @@ def main():
 - Run a TTS model with its default vocoder model:
 
  ```
- $ tts --text "Text for TTS" --model_name "<language>/<dataset>/<model_name>"
+ $ tts --text "Text for TTS" --model_name "<model_type>/<language>/<dataset>/<model_name>
  ```
 
 - Run with specific TTS and vocoder models from the list:
 
  ```
- $ tts --text "Text for TTS" --model_name "<language>/<dataset>/<model_name>" --vocoder_name "<language>/<dataset>/<model_name>" --output_path
+ $ tts --text "Text for TTS" --model_name "<model_type>/<language>/<dataset>/<model_name>" --vocoder_name "<model_type>/<language>/<dataset>/<model_name>" --output_path
  ```
 
 - Run your own TTS model (Using Griffin-Lim Vocoder):

TTS/bin/train_encoder.py

@@ -13,13 +13,13 @@
 
 from TTS.encoder.dataset import EncoderDataset
 from TTS.encoder.utils.generic_utils import save_best_model, save_checkpoint, setup_encoder_model
-from TTS.encoder.utils.samplers import PerfectBatchSampler
 from TTS.encoder.utils.training import init_training
 from TTS.encoder.utils.visual import plot_embeddings
 from TTS.tts.datasets import load_tts_samples
 from TTS.utils.audio import AudioProcessor
 from TTS.utils.generic_utils import count_parameters, remove_experiment_folder
 from TTS.utils.io import copy_model_files
+from TTS.utils.samplers import PerfectBatchSampler
 from TTS.utils.training import check_update
 
 torch.backends.cudnn.enabled = True

TTS/bin/tune_wavegrad.py

@@ -1,4 +1,4 @@
-"""Search a good noise schedule for WaveGrad for a given number of inferece iterations"""
+"""Search a good noise schedule for WaveGrad for a given number of inference iterations"""
 import argparse
 from itertools import product as cartesian_product
 
@@ -7,94 +7,97 @@
 from torch.utils.data import DataLoader
 from tqdm import tqdm
 
+from TTS.config import load_config
 from TTS.utils.audio import AudioProcessor
-from TTS.utils.io import load_config
 from TTS.vocoder.datasets.preprocess import load_wav_data
 from TTS.vocoder.datasets.wavegrad_dataset import WaveGradDataset
-from TTS.vocoder.utils.generic_utils import setup_generator
+from TTS.vocoder.models import setup_model
 
-parser = argparse.ArgumentParser()
-parser.add_argument("--model_path", type=str, help="Path to model checkpoint.")
-parser.add_argument("--config_path", type=str, help="Path to model config file.")
-parser.add_argument("--data_path", type=str, help="Path to data directory.")
-parser.add_argument("--output_path", type=str, help="path for output file including file name and extension.")
-parser.add_argument(
- "--num_iter", type=int, help="Number of model inference iterations that you like to optimize noise schedule for."
-)
-parser.add_argument("--use_cuda", type=bool, help="enable/disable CUDA.")
-parser.add_argument("--num_samples", type=int, default=1, help="Number of datasamples used for inference.")
-parser.add_argument(
- "--search_depth",
- type=int,
- default=3,
- help="Search granularity. Increasing this increases the run-time exponentially.",
-)
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--model_path", type=str, help="Path to model checkpoint.")
+ parser.add_argument("--config_path", type=str, help="Path to model config file.")
+ parser.add_argument("--data_path", type=str, help="Path to data directory.")
+ parser.add_argument("--output_path", type=str, help="path for output file including file name and extension.")
+ parser.add_argument(
+ "--num_iter",
+ type=int,
+ help="Number of model inference iterations that you like to optimize noise schedule for.",
+ )
+ parser.add_argument("--use_cuda", action="store_true", help="enable CUDA.")
+ parser.add_argument("--num_samples", type=int, default=1, help="Number of datasamples used for inference.")
+ parser.add_argument(
+ "--search_depth",
+ type=int,
+ default=3,
+ help="Search granularity. Increasing this increases the run-time exponentially.",
+ )
 
-# load config
-args = parser.parse_args()
-config = load_config(args.config_path)
+ # load config
+ args = parser.parse_args()
+ config = load_config(args.config_path)
 
-# setup audio processor
-ap = AudioProcessor(**config.audio)
+ # setup audio processor
+ ap = AudioProcessor(**config.audio)
 
-# load dataset
-_, train_data = load_wav_data(args.data_path, 0)
-train_data = train_data[: args.num_samples]
-dataset = WaveGradDataset(
- ap=ap,
- items=train_data,
- seq_len=-1,
- hop_len=ap.hop_length,
- pad_short=config.pad_short,
- conv_pad=config.conv_pad,
- is_training=True,
- return_segments=False,
- use_noise_augment=False,
- use_cache=False,
- verbose=True,
-)
-loader = DataLoader(
- dataset,
- batch_size=1,
- shuffle=False,
- collate_fn=dataset.collate_full_clips,
- drop_last=False,
- num_workers=config.num_loader_workers,
- pin_memory=False,
-)
+ # load dataset
+ _, train_data = load_wav_data(args.data_path, 0)
+ train_data = train_data[: args.num_samples]
+ dataset = WaveGradDataset(
+  ap=ap,
+  items=train_data,
+  seq_len=-1,
+  hop_len=ap.hop_length,
+  pad_short=config.pad_short,
+  conv_pad=config.conv_pad,
+  is_training=True,
+  return_segments=False,
+  use_noise_augment=False,
+  use_cache=False,
+  verbose=True,
+ )
+ loader = DataLoader(
+  dataset,
+  batch_size=1,
+  shuffle=False,
+  collate_fn=dataset.collate_full_clips,
+  drop_last=False,
+  num_workers=config.num_loader_workers,
+  pin_memory=False,
+ )
 
-# setup the model
-model = setup_generator(config)
-if args.use_cuda:
- model.cuda()
+ # setup the model
+ model = setup_model(config)
+ if args.use_cuda:
+  model.cuda()
 
-# setup optimization parameters
-base_values = sorted(10 * np.random.uniform(size=args.search_depth))
-print(base_values)
-exponents = 10 ** np.linspace(-6, -1, num=args.num_iter)
-best_error = float("inf")
-best_schedule = None
-total_search_iter = len(base_values) ** args.num_iter
-for base in tqdm(cartesian_product(base_values, repeat=args.num_iter), total=total_search_iter):
- beta = exponents * base
- model.compute_noise_level(beta)
- for data in loader:
- mel, audio = data
- y_hat = model.inference(mel.cuda() if args.use_cuda else mel)
+ # setup optimization parameters
+ base_values = sorted(10 * np.random.uniform(size=args.search_depth))
+ print(f" > base values: {base_values}")
+ exponents = 10 ** np.linspace(-6, -1, num=args.num_iter)
+ best_error = float("inf")
+ best_schedule = None # pylint: disable=C0103
+ total_search_iter = len(base_values) ** args.num_iter
+ for base in tqdm(cartesian_product(base_values, repeat=args.num_iter), total=total_search_iter):
+  beta = exponents * base
+  model.compute_noise_level(beta)
+  for data in loader:
+  mel, audio = data
+  y_hat = model.inference(mel.cuda() if args.use_cuda else mel)
 
- if args.use_cuda:
- y_hat = y_hat.cpu()
- y_hat = y_hat.numpy()
+  if args.use_cuda:
+  y_hat = y_hat.cpu()
+  y_hat = y_hat.numpy()
 
- mel_hat = []
- for i in range(y_hat.shape[0]):
- m = ap.melspectrogram(y_hat[i, 0])[:, :-1]
- mel_hat.append(torch.from_numpy(m))
+  mel_hat = []
+  for i in range(y_hat.shape[0]):
+  m = ap.melspectrogram(y_hat[i, 0])[:, :-1]
+  mel_hat.append(torch.from_numpy(m))
 
- mel_hat = torch.stack(mel_hat)
- mse = torch.sum((mel - mel_hat) ** 2).mean()
- if mse.item() < best_error:
- best_error = mse.item()
- best_schedule = {"beta": beta}
- print(f" > Found a better schedule. - MSE: {mse.item()}")
- np.save(args.output_path, best_schedule)
+  mel_hat = torch.stack(mel_hat)
+  mse = torch.sum((mel - mel_hat) ** 2).mean()
+  if mse.item() < best_error:
+  best_error = mse.item()
+  best_schedule = {"beta": beta}
+  print(f" > Found a better schedule. - MSE: {mse.item()}")
+  np.save(args.output_path, best_schedule)

TTS/config/__init__.py

@@ -62,7 +62,7 @@ def _process_model_name(config_dict: Dict) -> str:
  return model_name
 
 
-def load_config(config_path: str) -> None:
+def load_config(config_path: str) -> Coqpit:
  """Import `json` or `yaml` files as TTS configs. First, load the input file as a `dict` and check the model name
  to find the corresponding Config class. Then initialize the Config.
 

TTS/encoder/models/resnet.py

@@ -1,7 +1,7 @@
 import torch
 from torch import nn
 
-# from TTS.utils.audio import TorchSTFT
+# from TTS.utils.audio.torch_transforms import TorchSTFT
 from TTS.encoder.models.base_encoder import BaseEncoder
 
 

TTS/tts/configs/shared_configs.py

@@ -200,9 +200,6 @@ class BaseTTSConfig(BaseTrainingConfig):
  loss_masking (bool):
  enable / disable masking loss values against padded segments of samples in a batch.
 
- sort_by_audio_len (bool):
- If true, dataloder sorts the data by audio length else sorts by the input text length. Defaults to `False`.
-
  min_text_len (int):
  Minimum length of input text to be used. All shorter samples will be ignored. Defaults to 0.
 
@@ -303,7 +300,6 @@ class BaseTTSConfig(BaseTrainingConfig):
  batch_group_size: int = 0
  loss_masking: bool = None
  # dataloading
- sort_by_audio_len: bool = False
  min_audio_len: int = 1
  max_audio_len: int = float("inf")
  min_text_len: int = 1