Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XTTS v2.0 #3137

Merged
merged 31 commits into from
Nov 6, 2023
Merged

XTTS v2.0 #3137

merged 31 commits into from
Nov 6, 2023

Conversation

Edresson
Copy link
Contributor

@Edresson Edresson commented Nov 3, 2023

Updates with V2

  • HU and KO added
  • Updated voice cloning
  • Trained more so likely to be better across the board.

@erogol
Copy link
Member

erogol commented Nov 4, 2023

@Edresson Currently cloning does not work correctly. The voice is different than it's supposed to be.

Also, this model can take multiple references, we need a way to allow that.

@Edresson Edresson marked this pull request as draft November 4, 2023 14:23
@erogol
Copy link
Member

erogol commented Nov 4, 2023

I figured one issue. I was forgetting to pass mel stats.

Copy link
Contributor

@akx akx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would probably be a good idea to run the changed/new files here through make style to avoid those changes later on.

TTS/tts/layers/xtts/gpt.py Show resolved Hide resolved
TTS/tts/models/xtts.py Outdated Show resolved Hide resolved
Comment on lines +649 to +653
# print(" > Input text: ", text)
# print(" > Input text preprocessed: ",self.tokenizer.preprocess_text(text, language))
# print(" > Input tokens: ", text_tokens)
# print(" > Decoded text: ", self.tokenizer.decode(text_tokens[0].cpu().numpy()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead code?

recipes/ljspeech/xtts_v2/train_gpt_xtts.py Outdated Show resolved Hide resolved
Comment on lines +109 to +113
run_description="""
GPT XTTS training
""",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
run_description="""
GPT XTTS training
""",
run_description="GPT XTTS training",

tests/xtts_tests/test_xtts_v2-0_gpt_train.py Outdated Show resolved Hide resolved
tests/xtts_tests/test_xtts_v2-0_gpt_train.py Show resolved Hide resolved
tests/xtts_tests/test_xtts_v2-0_gpt_train.py Outdated Show resolved Hide resolved
tests/xtts_tests/test_xtts_v2-0_gpt_train.py Show resolved Hide resolved
Comment on lines +173 to +179
wav_chuncks = []
for i, chunk in enumerate(chunks):
if i == 0:
assert chunk.shape[-1] > 5000
wav_chuncks.append(chunk)
assert len(wav_chuncks) > 1
Copy link
Contributor

@akx akx Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo chuncks, but this simplifies to

Suggested change
wav_chuncks = []
for i, chunk in enumerate(chunks):
if i == 0:
assert chunk.shape[-1] > 5000
wav_chuncks.append(chunk)
assert len(wav_chuncks) > 1
wav_chunks = list(chunks) # consume generator
assert wav_chunks[0].shape[-1] > 5000
assert len(wav_chunks) > 1

@akx
Copy link
Contributor

akx commented Nov 6, 2023

@erogol please consider #3127 instead of make styleing everything in this PR. (Running make style made this a +1900 line PR.)

@erogol erogol marked this pull request as ready for review November 6, 2023 13:52
@erogol erogol merged commit e45227d into dev Nov 6, 2023
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants