diff --git a/docs/en/latest/ai-prompt-template.md b/docs/en/latest/ai-prompt-template.md new file mode 100644 index 000000000000..bc5628352b44 --- /dev/null +++ b/docs/en/latest/ai-prompt-template.md @@ -0,0 +1,102 @@ +--- +title: ai-prompt-template +keywords: + - Apache APISIX + - API Gateway + - Plugin + - ai-prompt-template +description: This document contains information about the Apache APISIX ai-prompt-template Plugin. +--- + + + +## Description + +The `ai-prompt-template` plugin simplifies access to AI providers and models by predefining the request format +using a template and allowing users to pass only the values for template variables. + +## Plugin Attributes + +| **Field** | **Type** | **Description** | **Required** | +| ------------------------------------- | -------- | --------------------------------------------------- | ------------ | +| `templates` | Array | An array of template objects | Yes | +| `templates.name` | String | Name of the template. | Yes | +| `templates.template.model` | String | Model of the AI Model. Example: gpt-4, gpt-3.5 | Yes | +| `templates.template.messages.role` | String | Role of the message (`system`, `user`, `assistant`) | Yes | +| `templates.template.messages.content` | String | Content of the message. | Yes | + +## Example usage + +Create a route with the `ai-prompt-template` plugin like so: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes/1" -X PUT \ + -H "X-API-KEY: ${ADMIN_API_KEY}" \ + -d '{ + "uri": "/v1/chat/completions", + "upstream": { + "type": "roundrobin", + "nodes": { + "api.openai.com:443": 1 + }, + "scheme": "https", + "pass_host": "node" + }, + "plugins": { + "ai-prompt-template": { + "templates": [ + { + "name": "level of detail", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "user", + "content": "Explain about {{ topic }} in {{ level }}." + } + ] + } + } + ] + } + } + }' +``` + +Now send a request: + +```shell +curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ + "template_name": "level of detail, + "topic": "psychology", + "level": "brief" +}' +``` + +Then the request body will be modified to something like this: + +```json +{ + "model": "some model", + "messages": [ + { "role": "user", "content": "Explain about psychology in brief." } + ] +} +```