Skip to content

Commit

Permalink
Change the way to configure "concat" filter for container logs (#117)
Browse files Browse the repository at this point in the history
- Add support to configure the "concat" filter section to be applied for all the container logs. For example, to parse multiline java stack trace from every container:
```
fluentd:
  config:
    logs:
      java-stacktrace:
        from:
          pod: "*"
        multiline:
          firstline: /\d{4}-\d{1,2}-\d{1,2}/
```
- Do not use the name of the source as fallback for `container` field. If `container` value is not set, apply configuration to all containers in the pod.
  • Loading branch information
dmitryax authored Apr 29, 2021
1 parent 9215dc3 commit 49fba52
Show file tree
Hide file tree
Showing 4 changed files with 11 additions and 3 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

## [0.24.11] - 2021-04-29

- Change the way to configure "concat" filter for container logs (#117)

## [0.24.10] - 2021-04-21

### Changed
Expand Down
2 changes: 1 addition & 1 deletion helm-charts/splunk-otel-collector/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
name: splunk-otel-collector
version: 0.24.10
version: 0.24.11
description: Splunk OpenTelemetry Connector for Kubernetes
icon: https://github.com/signalfx/splunk-otel-collector-chart/tree/main/splunk.png
type: application
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,8 @@ data:
# = filters for container logs =
{{- range $name, $logDef := .Values.fluentd.config.logs }}
{{- if and $logDef.from.pod $logDef.multiline }}
<filter tail.containers.var.log.containers.{{ $logDef.from.pod }}*{{ or $logDef.from.container $name }}*.log>
{{- $filenameGlob := regexReplaceAll "\\*+" (printf "%s*%s*" $logDef.from.pod ($logDef.from.container | default "")) "*" }}
<filter tail.containers.var.log.containers.{{ $filenameGlob }}.log>
@type concat
key log
timeout_label @SPLUNK
Expand Down
5 changes: 4 additions & 1 deletion helm-charts/splunk-otel-collector/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,10 @@ fluentd:
# path: /var/log/docker.log
# ```
#
# For `container` logs, pod name is required. You can also provide the container name, if it's not provided, the name of this source will be used as the container name:
# For `container` logs, `pod` field is required. It represents part of
# the pod name, can be name of a deployment or replica set. Use "*" to
# apply the configuration to all pods. Optional `container` value can be
# used to apply configuration to a particular container.
# ```
# kube-apiserver:
# from:
Expand Down

0 comments on commit 49fba52

Please sign in to comment.