Changing f0_method to improve results?

[JamesDylan]

Hi all,

New to this, but this has helped me learn so much!

I've been training a model from scratch, and so far its going pretty well. The timbre is really great, and the model is starting to capture the singers voice quite well. But there are two issues I'm struggling with, and hoping someone here can help:

1. The output is very pitchy. And doesn't match the input used to infer at all. To the point that tuning it makes it sound unnatural because its going up or down nearly an octave, so the voice sounds unnatural
2. There are often gaps where the output will miss parts of words

To improve pitch:
I'm currently on colab and using the f0_method "dio". Can I switch to using Crepe to improve the pitch? Will that add to the model or will I have to start again? I'm about 4.5k epochs through and its kinda slow so dont really want to go back to the start.
Or does anyone have any other ideas that could help?

To improve the missed words - is there a way to improve this tracking? Not sure what I'm meant to be trying to improve for this.

I'm training on vocal stems from a studio album. So the training source is great quality. And I have a lot of it.
I've attached some of the tensor graphs. As far as I can see some look like they're converging nicely and others I'm not sure if they're right or not.

Thanks in advance!

[Grownz]

1. AFAIK you'd have to start over if you want to switch from dio to crepe (or anything else).
2. What happened to your training that resulted in this strange hiatus just before 150k?
   

[JamesDylan]

The only thing i can think of is that I added more samples to the dataset. Mostly just out of curiosity to see how new information would impact things.

FYI I tried to just change hubert from DIO to CREPE, and continue training to see what happened.
It didnt even attempt to run any more batches. Just spun up the tensor dashboard then said it was complete.

I'll start again and run crepe and see if it gets better results faster. Sounds like DIO is good for spoken voice, but maybe not so good for singing...

[upColor]

I am surprised to see that you are able to change F0 prediction method prior to the training session with online Colab.
I could not change F0 prediction method in the local training - at least, I did not see any option to change that. And I don't even know what F0 method was used in the local training.

I can only change it on the GUI when doing inference. I have tried all F0 methods for my inference. And the best F0 method in my case is "parselmouth"! It sounds more natural than other methods.

And the worst F0 method in my case was "crepe" or "crepe-tiny", which causes the tone to go up and down in an unnatural way.

To improve missing words, the Silence threshold on the GUI should be set to -60.0, which makes the threshold the lowest. The default value for the Silence threshold is so high that missing words are inevitable.

[JamesDylan]

Interesting... I cant see anywhere in colab to change the silence threshold. But maybe once I have trained the model a bit I will pull the files down and try run it locally again.

Am currently training again with parselmouth so will report back tomorrow after a few hours training

[upColor]

I only know the value of Silence threshold on the GUI can be changed when doing inference. But I also want to change this value before training, so that is why I am asking the question on the following thread:
#713 (comment)

[wyhauyeung]

are you referring to pre-hubert stage? Isn't that you can switch to use CREPE ?
// To use CREPE, replace svc pre-hubert with svc pre-hubert -fm crepe //

[devcooch]

I had the same issues.

1. is solved by passing -na to svc infer. Can't say about dio vs crepe, using dio I am getting good results.
2. is solved by passing -db -35 to svc infer.

I am also newbie here, but managed to solve that.. now more interested to see how to remove back vocals from UVR-processed input.

[upColor]

Could you tell me what "-na" and "-db" mean please?

Is there any document where I can find all parameters associated with "svc infer" please?

Thank you!

[Grownz]

The -h parameter gives a list of all parameters (this should be true for all commands of this repo, i believe):

[upColor]

From the main page, I thought "-h" only would work for "svc -h". Now I learn that it also works for "svc infer -h". Thank you!

[Grownz]

No problemo :)

It should work for every svc xxxxxx -h.

[hataori-p]

now more interested to see how to remove back vocals from UVR-processed input

https://github.com/hataori-p/utils fmid.exe (windows cmd line only, see the video), but don't expect high quality and backing vocals shouldn't be in the center. Can also suppress stereo reverb.