HELPPPPP: How many Workers can i handle if iam having 48GB VRAM RTX A6000 (SINGLE GPU) and using 72B llama3.3 model

[SaiAkhil066]

How many Workers can i handle if iam having 48GB VRAM RTX A6000 (SINGLE GPU) and using 72B llama3.3 model and also tell that can i use my CPU bcs it is of having 32 threads, so as per that we can run 32 workers inferencing parallely if we are using verba project in LAN network, please HELP me with this... tell me clearly what to do and how to switch from GPU to CPU if I want to do

[thomashacker]

Hey, thanks for the issue! Can you share more information about what you're trying to achieve?

I think it would make sense to direct your question to the Ollama GitHub (https://github.com/ollama/ollama) since this will be the most computationally expensive part of using Verba.