To Much Noise on Mandarin

[Johnzxf]

I have synthesize on Mandarin.But it seems too much noise,listen to the audio below. How can I do to decrease the noise. Also, it seems synthesize femal audio is better than male.
BTW: the audio was synthesizer from texts.

链接：https://pan.baidu.com/s/1jXy9-6KnBKVciapcbGZmFw
提取码：knjb

[ghost]

Not enough info provided. I can't really help with this but if you answer the following questions it will improve your chances of getting a helpful response.

1. Did you use the code from this repo? If so what modifications did you make?
2. Which dataset did you train on?
3. What are your settings? Attach the synthesizer/hparams.py file
4. Which vocoder are you using?

Also you should make a zip file of your audio samples and attach them here, many of us here can't download from pan.baidu.com.

[Johnzxf]

Not enough info provided. I can't really help with this but if you answer the following questions it will improve your chances of getting a helpful response.

1. Did you use the code from this repo? If so what modifications did you make?
2. Which dataset did you train on?
3. What are your settings? Attach the synthesizer/hparams.py file
4. Which vocoder are you using?

Also you should make a zip file of your audio samples and attach them here, many of us here can't download from pan.baidu.com.

1: Yes, I use this repo. I did not change anything on the model. The most modifications is the preprocess of the dataset.
2: The dataset I have trained on is internal, and the audio is clean. The audio in my dataset is not as longer as Libirspeech.Most of the audio is between 3-5s.
3：Attach is setting.
4: The vocoder is WaveRNN.

During training synthesizer, I found the aligments has some gap,bteween encoder and decoder. Also see the attachment. Is it has inflence?

SV2TTS.zip
hparams.txt

[ghost]

@Johnzxf I'm sorry you didn't get a response. Did you figure out what was causing the noise?

[Johnzxf]

Thank you for your replay. I have not got the reason yet. I will check the vocoder first then the acoustic model. If I found the reason I will share with you. thank you------------------&nbsp;原始邮件&nbsp;------------------ 发件人:&nbsp;&quot;blue-fish&quot;<notifications@github.com&gt; 发送时间:&nbsp;2020年8月25日(星期二) 上午10:04 收件人:&nbsp;&quot;CorentinJ/Real-Time-Voice-Cloning&quot;<Real-Time-Voice-Cloning@noreply.github.com&gt;; 抄送:&nbsp;&quot;Johnzxf&quot;<1240721730@qq.com&gt;;&quot;Mention&quot;<mention@noreply.github.com&gt;; 主题:&nbsp;Re: [CorentinJ/Real-Time-Voice-Cloning] To Much Noise on Mandarin (#498)

[ghost]

Let's reopen and see if anyone knows why this is the case. Are you using RTVC (this repo) or zhrtvc?

[lawrence124]

@blue-fish
i have not tried the waveRNN vocoder on RTVC, but the pretrained waveRNN vocoder can't give me anything useful. The melgan multi speaker (or other melgan vocoder) yield far better result. I'm not too sure why though. (do we need to train vocoder for a specific language ??)

[ghost]

@lawrence124 A vocoder trained on enough speakers can generalize to unseen speakers and even other languages. mozilla/TTS#221 (comment)

The vocoders work within very narrow parameters and will fail if input does not meet the specification. To avoid incompatibility, the vocoder in this repo gets some of its hparams from the synthesizer. The relevant parameters are:

• sample_rate
• num_mels
• n_fft, hop_length, window_length
• fmin, fmax
• pre-emphasize

With default settings, I got very mediocre results on zhrtvc and I suspect that some of the vocoder parameters are not set correctly. Griffin-Lim performed the best. You should try this repo with the English models, the pretrained WaveRNN works quite well.

[ghost]

Closing this issue due to inactivity. Would like to know what is causing the noise if you find out.