Running Synthalingua on Arch Linux with an AMD GPU

[Expletive]

I got this running on my system with pytorch-rocm 2.0.1. I'm on arch linux with an AMD GPU and I didn't have to change anything in the code to get it working (but I did change a single line to make it use the large-v2 whisper model purely because I already had it downloaded and didn't want to download another one.)

[cyberofficial]

Nice glad to hear :D
I'm still working on rocm optimizations, the standard package can be slimmed and stripped of many things just for the project alone rather having it load and use a bunch of resources that aren't needed. If you encounter issues, let me know and I ill try and help out and I can use direct calling methods rather relying on bridges and pipe lines that dont need to be called.

[Expletive]

I've only had it running for a little while so the only problem I've noticed so far is one I've also had with using whisper directly, in that if there's no speech but there is other sound, like background music, the AI can just "hallucinate" and make up complete nonsense. I don't know if that's even a fixable problem.

[cyberofficial]

There is a way, you could try to use https://github.com/werman/noise-suppression-for-voice
and
https://github.com/josh-richardson/cadmus

and rerout the audio

[Expletive]

Oh, wow. I already have easy effects installed, so it was as easy as putting a noise reduction filter on the python audio capture.

[Expletive]

At the suggestion of a member of the stream chat I was watching, I did a quick hack and tried
modifying transcribe_audio.py to use faster_whisper
I don't actually know how to make it load model from a local path, so it downloaded another 3GB model file.

Anyway, here it is running the large-v2 model, and it seems to perform better as expected.