Skip to content

Commit a843223

Browse files
ngxsonhodlen
authored andcommitted
Server: format error to json (ggml-org#5961)
* server: format error to json * server: do not crash on grammar error * fix api key test case * revert limit max n_predict * small fix * correct coding style * update completion.js * launch_slot_with_task * update docs * update_slots * update webui * update readme
1 parent 0f5255a commit a843223

File tree

6 files changed

+430
-317
lines changed

6 files changed

+430
-317
lines changed

Diff for: examples/server/README.md

+44-2
Original file line numberDiff line numberDiff line change
@@ -556,9 +556,51 @@ Run with bash:
556556
bash chat.sh
557557
```
558558

559-
### API like OAI
559+
### OAI-like API
560560

561-
The HTTP server supports OAI-like API
561+
The HTTP server supports OAI-like API: https://github.com/openai/openai-openapi
562+
563+
### API errors
564+
565+
Server returns error in the same format as OAI: https://github.com/openai/openai-openapi
566+
567+
Example of an error:
568+
569+
```json
570+
{
571+
"error": {
572+
"code": 401,
573+
"message": "Invalid API Key",
574+
"type": "authentication_error"
575+
}
576+
}
577+
```
578+
579+
Apart from error types supported by OAI, we also have custom types that are specific to functionalities of llama.cpp:
580+
581+
**When /metrics or /slots endpoint is disabled**
582+
583+
```json
584+
{
585+
"error": {
586+
"code": 501,
587+
"message": "This server does not support metrics endpoint.",
588+
"type": "not_supported_error"
589+
}
590+
}
591+
```
592+
593+
**When the server receives invalid grammar via */completions endpoint**
594+
595+
```json
596+
{
597+
"error": {
598+
"code": 400,
599+
"message": "Failed to parse grammar",
600+
"type": "invalid_request_error"
601+
}
602+
}
603+
```
562604

563605
### Extending or building alternative Web Front End
564606

0 commit comments

Comments
 (0)