server / ranking : add sorting and management of top_n

[YannFollet]

server / ranking : add sorting and management of top_n

exemple output for top_n = 4 :

{
  "model": "M",
  "object": "list",
  "usage": {
    "prompt_tokens": 218,
    "total_tokens": 218
  },
  "results": [
    {
      "index": 6,
      "relevance_score": 0.7979143261909485
    },
    {
      "index": 0,
      "relevance_score": 0.3886369466781616
    },
    {
      "index": 2,
      "relevance_score": 0.2865810990333557
    },
    {
      "index": 4,
      "relevance_score": -0.5105927586555481
    }
  ]
}

or

[
  {
    "index": 6,
    "score": 0.7979143261909485
  },
  {
    "index": 0,
    "score": 0.3886369466781616
  },
  {
    "index": 2,
    "score": 0.2865810990333557
  },
  {
    "index": 4,
    "score": -0.5105927586555481
  }
]

[ggerganov]

I don't think the change is backwards compatible - would it work if the request does not include "top_n"?

[YannFollet]

no it won't work if no top_n, I can change it to make it back compatible

[ggerganov]

Yes, it should remain backward compatible.

[ngxson]

adding a test case is also required for this type of change

[YannFollet]

adding a test case is also required for this type of change

What king of test case? one bash script in comment to test many cases ?

[YannFollet]

here is one script to test the reranking

#!/bin/bash

# Set default URL if not provided
URL=${1:-http://127.0.0.1:8181/}

#replace texts by textsx to cancel tei_format
#replace top_n by top_nx to simulate no top_n

curl "$URL/v1/rerank" -H "Content-Type: application/json" \
 -d '{ "model": "M", "query": "What is the recipe to make bread ?",
 "return_text" : true,
 "texts" : true,
 "top_n": 6,
 "documents": [
 "voici la recette pour faire du pain, il faut de la farine de l eau et du levain et du sel",
 "it is a bear",
 "bread recipe : floor, water, yest, salt",
 "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.",
 "here is the ingedients to bake bread : 500g floor, 350g water, 120g fresh refresh yest, 15g salt",
 "recipe to make cookies : floor, eggs, water, chocolat",
 "here is the recipe to make bread : 500g floor, 350g water, 120g fresh refresh yest, 15g salt",
 "il fait tres beau aujourd hui",
 "je n ai pas faim, je ne veux pas manger",
 "je suis a paris"
 ] }' | jq

[ggerganov]

adding a test case is also required for this type of change

What king of test case? one bash script in comment to test many cases ?

You should add a server test: https://github.com/ggml-org/llama.cpp/tree/master/tools/server/tests