Improving repeatability of voice cloning

[ghost]

How can I make the voice cloning results repeatable? For a given model with same input.wav + text to synthesize, is there a way to ensure that I get the same output every time?

I see this code in synthesizer/train.py:

# Start by setting a seed for repeatability
tf.compat.v1.set_random_seed(hparams.tacotron_random_seed)

I tried inserting this in demo_cli.py and still get different results each time. Even if the program is restarted in between cloning attempts.

[ghost]

After reviewing code from @plummet555 , I realize that we should also set random.seed() for the python built-in RNG and np.random.seed() for the numpy RNG. I will try this later and see if repeatability improves.

[ghost]

With the changes on the 384_repeatable_voice_cloning branch of my fork, I get fully repeatable voice cloning output using demo_toolbox.py . Tested on a computer without a GPU.

Test case

• Reference audio: https://google.github.io/tacotron/publications/speaker_adaptation/demos/groundtruth/p240_00000.wav
• Text: Take a look at these pages for crooked creek drive.
• Export vocoder output as .wav file

You should find that the exported .wav files are identical for subsequent synthesize and vocode attempts within a session, and even across new toolbox sessions.

Required changes

1. Reload synthesizer and vocoder models every time they are used
2. Set torch.manual_seed() every time the vocoder is used
3. Set tensorflow seed every time synthesizer is used
   • The other RNG seeds are not relevant.
   • Also see 384 repeatable voice cloning #432 (comment)

Other changes

I reverted these changes and results are still repeatable on my platform. Listing here in case it helps troubleshooting in the future.

4. Force tensorflow to use a single-threaded session
5. Use kernel initializer for tf.nn.rnn_cell.GRUCell() and tf.layers.dense()
   • This might affect repeatability of training, but without these, inference is still repeatable.

References

(2) https://pytorch.org/docs/stable/notes/randomness.html
(3, 4) https://stackoverflow.com/a/52897216
(5) https://stackoverflow.com/a/51558159

[ghost]

Closing now that #432 is merged.