Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a basic trainer and dataset #1

Merged
merged 17 commits into from
Jul 14, 2024
Merged

Conversation

manmay-nakhashi
Copy link
Contributor

  • huggingface dataset
  • basic trainer

@manmay-nakhashi
Copy link
Contributor Author

manmay-nakhashi commented Jul 12, 2024

to test or train just run
by default it's using MushanW/GLOBE dataset 23,519 speakers and covers 164 accents

python3 train_e2.py

Note: need to add a path to vocab.json

@lucidrains
Copy link
Owner

@manmay-nakhashi Manmay! i remember you now from the natural speech work we did together some time ago

thanks for the PR! I will check it out tomorrow morning 😄

@lucidrains
Copy link
Owner

@manmay-nakhashi hey, looks good! 😄 do you want to try pulling and integrating the text as well?

@manmay-nakhashi
Copy link
Contributor Author

Sure I'll do that.

pyproject.toml Outdated Show resolved Hide resolved
e2_trainer.py Outdated Show resolved Hide resolved
e2_trainer.py Outdated Show resolved Hide resolved
@manmay-nakhashi
Copy link
Contributor Author

Screenshot 2024-07-14 at 8 14 32 AM

@manmay-nakhashi
Copy link
Contributor Author

@lucidrains it's ready

@manmay-nakhashi
Copy link
Contributor Author

i'll write a inference script next so we can do some quick experiments.

@lucidrains
Copy link
Owner

lucidrains commented Jul 14, 2024

nice! it looks good, but in the paper, they didn't use a tokenizer and just went character level

i was thinking we could just use utf character ids? (could remove the tokenizer and vocab.json altogether) keep it simple

@manmay-nakhashi
Copy link
Contributor Author

@lucidrains changes are done

train_e2.py Outdated Show resolved Hide resolved
e2_tts_pytorch/utils/compute_mel.py Outdated Show resolved Hide resolved
e2_tts_pytorch/dataset/e2_dataset.py Outdated Show resolved Hide resolved
@manmay-nakhashi
Copy link
Contributor Author

manmay-nakhashi commented Jul 14, 2024

@lucidrains resolved all the suggestions

@lucidrains
Copy link
Owner

@manmay-nakhashi thank you Manmay!

@lucidrains lucidrains merged commit 07b8532 into lucidrains:main Jul 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants