feat: graceful failure without model's YAML config

[jamesbraza]

LocalAI version:

e029cc6

Environment, CPU architecture, OS, and Version:

Darwin N7L493PWK4 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul  5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000 arm64

Describe the bug

When you don't have a YAML config in models/, you can get an obscure and large error response, not directly related to the actual underlying issue.

To Reproduce

Using llama-2-13b-ensemble-v5.Q4_K_M.gguf downloaded using huggingface_hub.hf_hub_download:

> ls models
llama-2-13b-ensemble-v5.Q4_K_M.gguf
> curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "llama-2-13b-ensemble-v5.Q4_K_M",
     "messages": [{"role": "user", "content": "What is an alpaca?"}],
     "temperature": 0.1
   }' | jq .
...
{
  "error": {
    "code": 500,
    "message": "could not load model - all backends returned error: 25 errors occurred:\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unknown desc = failed loading model\n\t* could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n\t* could not load model: rpc error: code = Unknown desc = stat /models/llama-2-13b-ensemble-v5.Q4_K_M: no such file or directory\n\t* could not load model: rpc error: code = Unknown desc = stat /models/llama-2-13b-ensemble-v5.Q4_K_M: no such file or directory\n\t* could not load model: rpc error: code = Unknown desc = unsupported model type /models/llama-2-13b-ensemble-v5.Q4_K_M (should end with .onnx)\n\t* backend unsupported: /build/extra/grpc/huggingface/huggingface.py\n\t* backend unsupported: /build/extra/grpc/autogptq/autogptq.py\n\t* backend unsupported: /build/extra/grpc/bark/ttsbark.py\n\t* backend unsupported: /build/extra/grpc/diffusers/backend_diffusers.py\n\t* backend unsupported: /build/extra/grpc/exllama/exllama.py\n\t* backend unsupported: /build/extra/grpc/vall-e-x/ttsvalle.py\n\t* backend unsupported: /build/extra/grpc/vllm/backend_vllm.py\n\n",
    "type": ""
  }
}

Expected behavior

I expect a concise and simple error message stating the actual issue (which was not having a model YAML config made), not a ton of semi-related rpc error: code = Unknown desc.

Logs

Here are logs in the docker compose container (run without --detach):

localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38781: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:39685: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46393: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:37429: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:43713: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38897: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:34381: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:35537: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38983: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:36063: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:37487: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:36697: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:34451: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:35385: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:43293: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42347: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:36319: connect: connection refused"
localai-api-1  | rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42321: connect: connection refused"

Additional context

#1069 (comment)

[mudler]

this is up for grabs - and indeed there is a lot of room of optimizations into logging here

[mudler]

I don't think we are chasing any bug here, but rather asking for better output. treating it as a feature request