[Not a BUG] ❤ Helpful Feedback based on latest version - huge improvement!

[AlonDan]

Hello,
This is my small contribution based on my personal experience since few versions ago to the current one., I don't see many people do that so I decided to take the time and share.

I believe that my feedback can help the developers and some future users as it's related to minimum requirements.

---

This feedback is based on my old PC and GPU (almost 10 years old)

• CPU: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz 3.50 GHz
• GPU: Nvidia GTX 980 4GB GDDR5
• RAM 32GB
• SSD: Kingstone SUV400S37 480GB

---

When I first started to try this FORK version I could only use:

• batch_size = 1 was the MAXIMUM I could use without getting OOM (Out Of Memory)
• It took roughly about 168 - 174 minutes to get to 100 epochs

Using the latest version: (as for 22-04-2023) with the EXACT same PC / hardware:

• batch_size = 3 is the MAXIMUM I can use without getting OOM (Out Of Memory)
• It takes roughly about 126 - 132 minutes to get to 100 epochs

---

Obviously it is VERY slow compare to cloud training or any local MODERN PC with latest generation GPU, but this is exactly why I'm sharing this, as my PC isn't even listed on the minimum recommended hardware but still working + show improvements.

Since I'm not a programmer, I don't know HOW the developers did this magic but I must mention that you can SURE mention that it works with old tech such as GTX 980 (not even TI but the classic model)

I hope that this contribution of mine helps a bit based on my personal feedback,
I can now train much faster without OOM on the same 10 years old PC.

Please keep up the good work, much love 💙

[ADR3-N]

I have a similar experience with a modestly better 1080 Nvidia card. A little less time, maybe 15 minutes shorter per hundred epochs.

The training is SLOW compared to colab or any modern PC, but the results speak for themselves. I only had to make a few quality of life improvements such as increasing learning rate to 0.005 or so, and I was able to not only change my voice but record professional sounding music. Big plus.

Thank you all.

[AlonDan]

I was curious about changing learning rate, because I was afraid the FASTER it is (closer to zero) the more risky it will "roughly LEARN" while training, because even on cloud it is recommended by default on a VERY slow learning number.

I'm not sure if closer to zero goes (faster) the risk is higher for less quality results or not.

I'm doing most of my tests on SPEAKING so many things are more noticeable compare to singing, especially in different languages.

💙 If the developers can explain if the Learning Rate speed affecting quality or anything else that will be nice to know, if not it will be very helpful for my next: speed-tests | quality ratio.

[ADR3-N]

I read somewhere, can't remember where, that 0.0005 was ideal for language learning networks. It may be best to set it at that to begin and decrease as the network ages. I'm going to try 0.00145 and see how it goes

I'm really new to this stuff though, so please take my word with a grain of salt or try it on a model you don't have much investment in! I don't even know how to export my model properly yet as you can see from the discussion I just posted (god help me)

But other than the new batch of data I dropped into the folder for processing resulting in a temporary dip in quality, my results have been great so far.