Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add schema attribute definition for some log plug-ins #10738

Merged
merged 1 commit into from
Jan 3, 2024
Merged

feat: Add schema attribute definition for some log plug-ins #10738

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions apisix/plugins/file-logger.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ local schema = {
type = "string"
},
log_format = {type = "object"},
include_req_body = {type = "boolean", default = false},
include_req_body_expr = {
type = "array",
minItems = 1,
items = {
type = "array"
}
},
include_resp_body = {type = "boolean", default = false},
include_resp_body_expr = {
type = "array",
Expand Down
7 changes: 7 additions & 0 deletions apisix/plugins/http-logger.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,13 @@ local schema = {
timeout = {type = "integer", minimum = 1, default = 3},
log_format = {type = "object"},
include_req_body = {type = "boolean", default = false},
include_req_body_expr = {
type = "array",
minItems = 1,
items = {
type = "array"
}
},
include_resp_body = {type = "boolean", default = false},
include_resp_body_expr = {
type = "array",
Expand Down
7 changes: 7 additions & 0 deletions apisix/plugins/loggly.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,13 @@ local schema = {
description = "base severity log level",
},
include_req_body = {type = "boolean", default = false},
include_req_body_expr = {
type = "array",
minItems = 1,
items = {
type = "array"
}
},
include_resp_body = {type = "boolean", default = false},
include_resp_body_expr = {
type = "array",
Expand Down
14 changes: 14 additions & 0 deletions apisix/plugins/tencent-cloud-cls.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,21 @@ local schema = {
default = 1
},
include_req_body = { type = "boolean", default = false },
include_req_body_expr = {
type = "array",
minItems = 1,
items = {
type = "array"
}
},
include_resp_body = { type = "boolean", default = false },
include_resp_body_expr = {
type = "array",
minItems = 1,
items = {
type = "array"
}
},
global_tag = { type = "object" },
log_format = {type = "object"},
},
Expand Down
2 changes: 2 additions & 0 deletions docs/en/latest/plugins/file-logger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ The `file-logger` Plugin is used to push log streams to a specific location.
| ---- | ------ | -------- | ------------- |
| path | string | True | Log file path. |
| log_format | object | False | Log format declared as key value pairs in JSON format. Values only support strings. [APISIX](../apisix-variable.md) or [Nginx](http://nginx.org/en/docs/varindex.html) variables can be used by prefixing the string with `$`. |
| include_req_body | boolean | False | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations. |
| include_req_body_expr | array | False | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
| include_resp_body | boolean | False | When set to `true` includes the response body in the log file. |
| include_resp_body_expr | array | False | When the `include_resp_body` attribute is set to `true`, use this to filter based on [lua-resty-expr](https://github.com/api7/lua-resty-expr). If present, only logs the response into file if the expression evaluates to `true`. |
| match | array[] | False | Logs will be recorded when the rule matching is successful if the option is set. See [lua-resty-expr](https://github.com/api7/lua-resty-expr#operator-list) for a list of available expressions. |
Expand Down
1 change: 1 addition & 0 deletions docs/en/latest/plugins/http-logger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ This will allow the ability to send log data requests as JSON objects to monitor
| timeout | integer | False | 3 | [1,...] | Time to keep the connection alive for after sending a request. |
| log_format | object | False | | | Log format declared as key value pairs in JSON format. Values only support strings. [APISIX](../apisix-variable.md) or [Nginx](http://nginx.org/en/docs/varindex.html) variables can be used by prefixing the string with `$`. |
| include_req_body | boolean | False | false | [false, true] | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations. |
| include_req_body_expr | array | False | | | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
| include_resp_body | boolean | False | false | [false, true] | When set to `true` includes the response body in the log. |
| include_resp_body_expr | array | False | | | When the `include_resp_body` attribute is set to `true`, use this to filter based on [lua-resty-expr](https://github.com/api7/lua-resty-expr). If present, only logs the response if the expression evaluates to `true`. |
| concat_method | string | False | "json" | ["json", "new_line"] | Sets how to concatenate logs. When set to `json`, uses `json.encode` for all pending logs and when set to `new_line`, also uses `json.encode` but uses the newline (`\n`) to concatenate lines. |
Expand Down
21 changes: 11 additions & 10 deletions docs/en/latest/plugins/loggly.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,16 +37,17 @@ When the maximum batch size is exceeded, the data in the queue is pushed to Logg

## Attributes

| Name | Type | Required | Default | Description |
|------------------------|---------------|----------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| customer_token | string | True | | Unique identifier used when sending logs to Loggly to ensure that they are sent to the right organisation account. |
| severity | string (enum) | False | INFO | Syslog log event severity level. Choose between: `DEBUG`, `INFO`, `NOTICE`, `WARNING`, `ERR`, `CRIT`, `ALERT`, and `EMEGR`. |
| severity_map | object | False | nil | A way to map upstream HTTP response codes to Syslog severity. Key-value pairs where keys are the HTTP response codes and the values are the Syslog severity levels. For example `{"410": "CRIT"}`. |
| tags | array | False | | Metadata to be included with any event log to aid in segmentation and filtering. |
| log_format | object | False | {"host": "$host", "@timestamp": "$time_iso8601", "client_ip": "$remote_addr"} | Log format declared as key value pairs in JSON format. Values only support strings. [APISIX](../apisix-variable.md) or [Nginx](http://nginx.org/en/docs/varindex.html) variables can be used by prefixing the string with `$`. |
| include_req_body | boolean | False | false | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations. |
| include_resp_body | boolean | False | false | When set to `true` includes the response body in the log. |
| include_resp_body_expr | array | False | | When the `include_resp_body` attribute is set to `true`, use this to filter based on [lua-resty-expr](https://github.com/api7/lua-resty-expr). If present, only logs the response if the expression evaluates to `true`. |
| Name | Type | Required | Default | Description |
|------------------------|---------------|----------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| customer_token | string | True | | Unique identifier used when sending logs to Loggly to ensure that they are sent to the right organisation account. |
| severity | string (enum) | False | INFO | Syslog log event severity level. Choose between: `DEBUG`, `INFO`, `NOTICE`, `WARNING`, `ERR`, `CRIT`, `ALERT`, and `EMEGR`. |
| severity_map | object | False | nil | A way to map upstream HTTP response codes to Syslog severity. Key-value pairs where keys are the HTTP response codes and the values are the Syslog severity levels. For example `{"410": "CRIT"}`. |
| tags | array | False | | Metadata to be included with any event log to aid in segmentation and filtering. |
| log_format | object | False | {"host": "$host", "@timestamp": "$time_iso8601", "client_ip": "$remote_addr"} | Log format declared as key value pairs in JSON format. Values only support strings. [APISIX](../apisix-variable.md) or [Nginx](http://nginx.org/en/docs/varindex.html) variables can be used by prefixing the string with `$`. |
| include_req_body | boolean | False | false | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations. |
| include_req_body_expr | array | False | | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
| include_resp_body | boolean | False | false | When set to `true` includes the response body in the log. |
| include_resp_body_expr | array | False | | When the `include_resp_body` attribute is set to `true`, use this to filter based on [lua-resty-expr](https://github.com/api7/lua-resty-expr). If present, only logs the response if the expression evaluates to `true`. |

This Plugin supports using batch processors to aggregate and process entries (logs/data) in a batch. This avoids the need for frequently submitting the data. The batch processor submits data every `5` seconds or when the data in the queue reaches `1000`. See [Batch Processor](../batch-processor.md#configuration) for more information or setting your custom configuration.

Expand Down
6 changes: 4 additions & 2 deletions docs/en/latest/plugins/tencent-cloud-cls.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,18 @@ The `tencent-cloud-cls` Plugin uses [TencentCloud CLS](https://cloud.tencent.com
## Attributes

| Name | Type | Required | Default | Valid values | Description |
| ----------------- | ------- | -------- |---------| ------------- |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ----------------- | ------- |----------|---------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| cls_host | string | Yes | | | CLS API host，please refer [Uploading Structured Logs](https://www.tencentcloud.com/document/api/614/16873). |
| cls_topic | string | Yes | | | topic id of CLS. |
| secret_id | string | Yes | | | SecretId of your API key. |
| secret_key | string | Yes | | | SecretKey of your API key. |
| sample_ratio | number | No | 1 | [0.00001, 1] | How often to sample the requests. Setting to `1` will sample all requests. |
| include_req_body | boolean | No | false | [false, true] | When set to `true` includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to NGINX's limitations. |
| include_req_body_expr | array | No | | | Filter for when the `include_req_body` attribute is set to `true`. Request body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
| include_resp_body | boolean | No | false | [false, true] | When set to `true` includes the response body in the log. |
| include_resp_body_expr | array | No | | | Filter for when the `include_resp_body` attribute is set to `true`. Response body is only logged when the expression set here evaluates to `true`. See [lua-resty-expr](https://github.com/api7/lua-resty-expr) for more. |
| global_tag | object | No | | | kv pairs in JSON，send with each log. |
| log_format | object | No | | | Log format declared as key value pairs in JSON format. Values only support strings. [APISIX](../apisix-variable.md) or [Nginx](http://nginx.org/en/docs/varindex.html) variables can be used by prefixing the string with `$`. |
| log_format | object | No | | | Log format declared as key value pairs in JSON format. Values only support strings. [APISIX](../apisix-variable.md) or [Nginx](http://nginx.org/en/docs/varindex.html) variables can be used by prefixing the string with `$`. |

NOTE: `encrypt_fields = {"secret_key"}` is also defined in the schema, which means that the field will be stored encrypted in etcd. See [encrypted storage fields](../plugin-develop.md#encrypted-storage-fields).

Expand Down
12 changes: 7 additions & 5 deletions docs/zh/latest/plugins/file-logger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,13 @@ description: API 网关 Apache APISIX file-logger 插件可用于将日志数据
## 属性

| 名称 | 类型 | 必选项 | 描述 |
| ---------------- | ------- | ------ | ------------------------------------------------ |
| path | string | 是 | 自定义输出文件路径。例如：`logs/file.log`。 |
| log_format | object | 否 | 以 JSON 格式的键值对来声明日志格式。对于值部分，仅支持字符串。如果是以 `$` 开头，则表明是要获取 [APISIX 变量](../apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
| include_resp_body | boolean | 否 | 当设置为 `true` 时，生成的文件包含响应体。 |
| include_resp_body_expr | array | 否 | 当 `include_resp_body` 属性设置为 `true` 时，使用该属性并基于 [lua-resty-expr](https://github.com/api7/lua-resty-expr) 进行过滤。如果存在，则仅在表达式计算结果为 `true` 时记录响应。 |
| ---------------- | ------- |-----| ------------------------------------------------ |
| path | string | 是 | 自定义输出文件路径。例如：`logs/file.log`。 |
| log_format | object | 否 | 以 JSON 格式的键值对来声明日志格式。对于值部分，仅支持字符串。如果是以 `$` 开头，则表明是要获取 [APISIX 变量](../apisix-variable.md) 或 [NGINX 内置变量](http://nginx.org/en/docs/varindex.html)。 |
| include_req_body | boolean | 否 | 当设置为 `true` 时，日志中将包含请求体。如果请求体太大而无法在内存中保存，则由于 Nginx 的限制，无法记录请求体。|
| include_req_body_expr | array | 否 | 当 `include_req_body` 属性设置为 `true` 时的过滤器。只有当此处设置的表达式求值为 `true` 时，才会记录请求体。有关更多信息，请参阅 [lua-resty-expr](https://github.com/api7/lua-resty-expr) 。 |
| include_resp_body | boolean | 否 | 当设置为 `true` 时，生成的文件包含响应体。 |
| include_resp_body_expr | array | 否 | 当 `include_resp_body` 属性设置为 `true` 时，使用该属性并基于 [lua-resty-expr](https://github.com/api7/lua-resty-expr) 进行过滤。如果存在，则仅在表达式计算结果为 `true` 时记录响应。 |
| match | array[] | 否 | 当设置了这个选项后，只有匹配规则的日志才会被记录。`match` 是一个表达式列表，具体请参考 [lua-resty-expr](https://github.com/api7/lua-resty-expr#operator-list)。 |

## 插件元数据设置
Expand Down
Loading
Loading