Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot parse empty syslog RFC5424 logs #2997

Closed
evandam opened this issue May 18, 2020 · 1 comment · Fixed by #3015
Closed

Cannot parse empty syslog RFC5424 logs #2997

evandam opened this issue May 18, 2020 · 1 comment · Fixed by #3015
Assignees
Labels
bug Something isn't working

Comments

@evandam
Copy link

evandam commented May 18, 2020

Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.

Describe the bug

Sending empty logs to a syslog source results in a warning from Fluentd failed to parse message data.

To Reproduce

Log an echo with no parameters to Syslog. I am using the following docker-compose file to test:

version: '3'
services:
  fluentd:
    command: ["fluentd", "-c", "/etc/fluentd.conf"]
    image: fluent/fluentd:v1.10.4-1.0
    volumes:
      - ./fluentd.conf:/etc/fluentd.conf
    ports:
      - 5140:5140/udp
  test:
    image: alpine
    logging:
      driver: syslog
      options:
        syslog-address: "udp://127.0.0.1:5140"
        tag: test
        syslog-format: rfc5424
    depends_on:
      - fluentd
docker-compose up fluentd
docker-compose run test echo ""
docker-compose run test echo "hello fluentd"

Expected behavior

Parse the logs like so:

2020-05-18 19:14:05.000000000 +0000 syslog.docker.containers.daemon.info: {"host":"default","ident":"test","pid":"2139","msgid":"test","extradata":"-","message":""}

Your Environment

  • Fluentd 1.10.4-1
  • Operating system: MacOS 10.4, and Ubuntu 18.04:
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
  • Kernel version: 4.15.0-1065-aws

Your Configuration

<source>
  @type syslog
  tag syslog.docker.containers
  port 5140
  protocol_type udp
  <parse>
    message_format rfc5424
  </parse>
  @log_level info
</source>

<match syslog.docker.containers.**>
  @type stdout
</match>

Your Error Log

fluentd_1  | fluentd -c /etc/fluentd.conf
fluentd_1  | 2020-05-18 19:18:43 +0000 [info]: parsing config file is succeeded path="/etc/fluentd.conf"
fluentd_1  | 2020-05-18 19:18:43 +0000 [info]: gem 'fluentd' version '1.10.4'
fluentd_1  | 2020-05-18 19:18:44 +0000 [warn]: 'protocol_type' parameter is deprecated: use transport directive
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: using configuration file: <ROOT>
fluentd_1  |   <source>
fluentd_1  |     @type syslog
fluentd_1  |     tag "syslog.docker.containers"
fluentd_1  |     port 5140
fluentd_1  |     protocol_type udp
fluentd_1  |     emit_unmatched_lines false
fluentd_1  |     @log_level "info"
fluentd_1  |     <parse>
fluentd_1  |       message_format rfc5424
fluentd_1  |     </parse>
fluentd_1  |   </source>
fluentd_1  |   <match syslog.docker.containers.**>
fluentd_1  |     @type stdout
fluentd_1  |   </match>
fluentd_1  | </ROOT>
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: starting fluentd-1.10.4 pid=6 ruby="2.5.8"
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/etc/fluentd.conf", "-p", "/fluentd/plugins", "--under-supervisor"]
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: adding match pattern="syslog.docker.containers.**" type="stdout"
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: adding source type="syslog"
fluentd_1  | 2020-05-18 19:18:44 +0000 [warn]: #0 'protocol_type' parameter is deprecated: use transport directive
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: #0 starting fluentd worker pid=20 ppid=6 worker=0
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: #0 listening syslog socket on 0.0.0.0:5140 with udp
fluentd_1  | 2020-05-18 19:18:44 +0000 [info]: #0 fluentd worker is now running worker=0
fluentd_1  | 2020-05-18 19:18:52 +0000 [warn]: #0 failed to parse message data="<30>1 2020-05-18T19:18:52Z default test 2139 test - "

Additional context

@repeatedly
Copy link
Member

This issue will be fixed by rfc5424 string parser.

@repeatedly repeatedly added the bug Something isn't working label Jun 1, 2020
@repeatedly repeatedly self-assigned this Jun 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants