Skip to content

Commit

Permalink
Remove deleted endpoints, explain evaluation
Browse files Browse the repository at this point in the history
Removed the description of the summarisation endpoint which now lives in its own repository. Also added a little disclaimer to the evaluation section.
  • Loading branch information
redadmiral authored Oct 2, 2024
1 parent 81c769c commit b462da7
Showing 1 changed file with 5 additions and 21 deletions.
26 changes: 5 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,6 @@ You can now view the Swagger documentation of the API in your browser under `loc

## Endpoints

### Summarisation
The endpoint `completion` returns a shortened version of the input text you provide as source.

You can access it with the following curl statement or via the Swagger docs.

```shell
curl -X 'POST' \
'http://localhost:3000/completion?model=gpt-3.5-turbo&honest=true' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"source": "string"
}'
```

Since this API is also designed for demonstration purposes it is possible to enforce hallucinations by setting the
honest parameter to `false`. As models you can choose either `gpt-3.5-turbo` or `gpt-4-turbo`.


## Fact check

The endpoint `check` performs a check if a `sentence` is contained in the `source`.
Expand Down Expand Up @@ -65,7 +46,10 @@ This repository contains two scripts designed to evaluate and enhance the accura
The script `evaluation.py` aims to validate the effectiveness of the system by comparing its predictions with the
gold standard dataset, ultimately providing a measure of accuracy.

The script `predictor.py` focuses on processing the test data set using the provided API to create set to validate against.
The script `predictor.py` focuses on processing the test data set using the provided API to create set to validate against.

:warning: This test set is designed to detect wrong answers to questions if the source is given. We think the two tasks
are similar enough to use this on our domain as well.

### Available Test- and Training Data

Expand Down Expand Up @@ -95,4 +79,4 @@ JSON object with the structure as follows:
To run the evaluation simply run `python evaluate.py` after you've placed your results in the data folder.
The evaluation script calculates the accuracy - e.g. the percentage of correctly predicted samples.

The current
The current

0 comments on commit b462da7

Please sign in to comment.