having this error

[hiqsociety]

having this error, how to resolve?

[Aisuko]

hi @hiqsociety. Thanks for your feedback. May you show us more information like which version of localAI you are using. And may you please replace the title to more related to the issue?

[malpou]

Hi @Aisuko

I'm encountering the same issue.
I'm running it in docker desktop on windows.

Not quite sure where to check the version, but did a git clone at ‎8. ‎september ‎2023, ‏‎13:29:07 (GMT+2)

I've used your Example: Use GPT4ALL-J model with guide.

This is the logs from the docker container:

2023-09-08 15:28:15 @@@@@
2023-09-08 15:28:15 Skipping rebuild
2023-09-08 15:28:15 @@@@@
2023-09-08 15:28:15 If you are experiencing issues with the pre-compiled builds, try setting REBUILD=true
2023-09-08 15:28:15 If you are still experiencing issues with the build, try setting CMAKE_ARGS and disable the instructions set as needed:
2023-09-08 15:28:15 CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF"
2023-09-08 15:28:15 see the documentation at: https://localai.io/basics/build/index.html
2023-09-08 15:28:15 Note: See also https://github.com/go-skynet/LocalAI/issues/288
2023-09-08 15:28:15 @@@@@
2023-09-08 15:28:15 CPU info:
2023-09-08 15:28:15 model name  : Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
2023-09-08 15:28:15 flags               : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi ept vpid ept_ad fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves flush_l1d arch_capabilities
2023-09-08 15:28:15 CPU:    AVX    found OK
2023-09-08 15:28:15 CPU:    AVX2   found OK
2023-09-08 15:28:15 CPU: no AVX512 found
2023-09-08 15:28:15 @@@@@
2023-09-08 15:28:15 
2023-09-08 15:28:15  ┌───────────────────────────────────────────────────┐ 
2023-09-08 15:28:15  │                   Fiber v2.48.0                   │ 
2023-09-08 15:28:15  │               http://127.0.0.1:8080               │ 
2023-09-08 15:28:15  │       (bound on host 0.0.0.0 and port 8080)       │ 
2023-09-08 15:28:15  │                                                   │ 
2023-09-08 15:28:15  │ Handlers ............ 58  Processes ........... 1 │ 
2023-09-08 15:28:15  │ Prefork ....... Disabled  PID ................ 14 │ 
2023-09-08 15:28:15  └───────────────────────────────────────────────────┘ 
2023-09-08 15:28:15 
2023-09-08 15:29:19 rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:37715: connect: connection refused"
2023-09-08 15:29:21 rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46759: connect: connection refused"
2023-09-08 15:29:26 rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:46733: connect: connection refused"

The connection error shows up after sending the curl http://localhost:8080/v1/models request

[y4my4my4m]

I'm getting the same issue. The lunademo works, but i get these responses almost all the time with other models (even low RAM ones)

[EchedelleLR]

I am also getting those errors.

The version from a month ago did not show that. Is like if it had some kind of RPC server running by default even when the line specific for GPRC was commented.

Maybe some code error?

[nicolasembleton]

I've been dabbling with this a little bit, and my still uneducated guess is that you haven't set the THREADS count or haven't specified your GPU count composition (SINGLE_ACTIVE_BACKEND=true if only one is available). Setting THREADS to the vCPU count was what I needed in my case. Intuitive, but the defaults aren't very verbose so it took a bit of trial & error to figure that out.

[pengtb]

As for me, change the port in .env/Dockerfile solve the problem. Still wondering why...