How can I do summarization

[testplop]

I'm trying to make something like https://platform.openai.com/examples/default-notes-summary

But it fails, I tried with gpt4all, llama and alpaca 7B. Maybe I should ajust the prompt ?

[Topping1]

I tried a prompt like this:
summarize the text in brackets [text to summarize]
with some good and not so good results. Sometimes the model hallucinates parts of the summary. I think you can avoid this by setting the temperature low (like 0.2 for example). Also is useful to increase the context length to 2048.
I hope someone fine tunes Llama with summarization datasets already existing in Hugginface. It could be really useful.

[Topping1]

I'm trying to make something like https://platform.openai.com/examples/default-notes-summary

But it fails, I tried with gpt4all, llama and alpaca 7B. Maybe I should ajust the prompt ?

I found this post that has some examples:
https://libreddit.spike.codes/r/LocalLLaMA/comments/1260d5i/summarizing_short_stories_with_llama_13b/

[klosax]

Example of using Alpaca model to make a summary

Put the following Alpaca-prompts in a file named prompt.txt

### Instruction:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

Write a detailed summary of the meeting in the input.

### Input:

Tom: Profits up 50%
Jane: New servers are online
Kjel: Need more time to fix software
Jane: Happy to help
Parkman: Beta testing almost done

### Response:

Run the model with sampling parameters that minimize hallucination

./main -c 2048 --temp 0.0 --top_p 0.0 --top_k 1.0 -n -1 -f prompt.txt -m <model file>

Example output

The meeting started with Tom announcing that the company's profits have increased by 50%. Jane then informed the group that the new servers are online. Kjel then stated that he needs more time to fix the software. Jane responded that she is happy to help. Parkman then announced that the beta testing is almost done.

Model used: gpt4-x-alpaca-30b-ggml-q4_1.bin

[ivanstepanovftw]

According to Locally Typical Sampling scientific paper, summarization works best with beam search.