Skip to content

Output supervisord log in the same format #14

Open
@vvbogdanov87

Description

@vvbogdanov87

Is it possible to have supervisord's log in the same format?
I gather and parse docker(kubernetes) logs using fluentd. This is my config:

<source>
  type tail
  path /var/log/containers/*.log
  pos_file /var/log/fluentd-containers.log.pos
  time_format %Y-%m-%dT%H:%M:%S.%NZ
  tag kubernetes.*
  format json
  read_from_head true
</source>

<filter kubernetes.**>
  @type parser
  format /^(?<name>[^ ]*) (?<stream>[^ ]*) [^ ]* (?<log>.*)$/
  key_name log
  reserve_data true
  inject_key_prefix kubernetes.app.
</filter>

As a result I see this in kibana:

"log": "mongos_jetlore stdout | 2017-07-07T12:46:54.424+0000 I SHARDING [Balancer] distributed lock with ts: 595f82be07b9697e6caf0af3' unlocked.\n",
    "stream": "stdout",
    "kubernetes.app.name": "mongos_jetlore",
    "kubernetes.app.stream": "stdout",
    "kubernetes.app.log": "2017-07-07T12:46:54.424+0000 I SHARDING [Balancer] distributed lock with ts: 595f82be07b9697e6caf0af3' unlocked.",
    "docker": {
      "container_id": "4fb5b6422d57385d1f9cf26ddec83b71f2fe4cc2452af2240af75938ace98158"
    },

But also I see entries:

"stream": "stdout",
    "kubernetes.app.name": "2017-07-07",
    "kubernetes.app.stream": "11:41:06,240",
    "kubernetes.app.log": "success: goofys entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)",

This happens because supervisor-stdout doesn't append supervisord's out. It appends log only for programs controlled by supervisord.
I tried to use this config:

[supervisord]
logfile=/var/log/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=true               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)
stdout_events_enabled = true
stderr_events_enabled = true

But this doesn't work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions