Support different prompting templates

[lambadalambda]

Thank you for Amica, I'm having a lot of fun with it!

I tried using the yi-34B models with llama.cpp with it, and generally it works, but the results aren't optimal. For example, the reply always ends with "<|im_end|>" which ends up visible in the conversation.

It would be great to have a way to not just change the system prompt, but to change the prompting template altogether. Here's ChatML, which Yi is trained to use:

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

[DeanMaxwell]

I would agree with a need for this functionality. On several models, Amica tends to trend toward ending all responses with "agent: {... something they *think the user would say}", generating a potential user response instead of waiting for the next response from the actual user. This then tends to bias future generation toward the Amica 'user' prompt tokens instead of the actual following user reply.

[lambadalambda]

src/utils/buildPrompt.ts --- 1/1 --- TypeScript
 6   for (let m of messages) {                                                                  6   for (let m of messages) {
 7     switch(m.role) {                                                                         7     switch(m.role) {
 8       case 'system':                                                                         8       case 'system':
 9         prompt += config("system_prompt")+"\n\n";                                            9         prompt += "<|im_start|>system\n" + config("system_prompt")+"<|im_end|>\n";
10         break;                                                                              10         break;
11       case 'user':                                                                          11       case 'user':
12         prompt += `User: ${m.content}\n`;                                                   12         prompt += `<|im_start|>user\n${m.content}<|im_end|>\n`;
13         break;                                                                              13         break;
14       case 'assistant':                                                                     14       case 'assistant':
15         prompt += `${config("name")}: ${m.content}\n`;                                      15         prompt += `<|im_start|>assistant\n${m.content}\n<|im_end|>\n`;
16         break;                                                                              16         break;
17     }                                                                                       17     }
18   }                                                                                         18   }
19   prompt += `${config('name')}:`;                                                           19   prompt += `<|im_start|>assistant\n`;
20   return prompt;                                                                            20   return prompt;
21 }                                                                                           21 }
22

here's a diff that switches to the chatml format, just for other users who might find this issue

[slowsynapse]

Arbius has a $150 AIUS bounty for this issue!

Brief: Fix/Complete Issue, suggest adding it as expandable under system prompt page.

Please read carefully:

To begin work on a bounty, reply by saying “I claim this bounty” - you will have 48 hours to submit your PR before someone else may attempt to claim this bounty.

To complete the bounty, within 48 hours of claiming, reply with a link to your PR referencing this issue and an Ethereum address. You must comply with reviewers comments and have the PR merged to receive the bounty reward. Please be sure to focus on quality submissions to minimize the amount of time reviewers must take.