Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

promtail docker_sd_config: long lines crash promtail parsing #6257

Closed
srstsavage opened this issue May 27, 2022 · 0 comments · Fixed by #6256
Closed

promtail docker_sd_config: long lines crash promtail parsing #6257

srstsavage opened this issue May 27, 2022 · 0 comments · Fixed by #6256

Comments

@srstsavage
Copy link
Contributor

Describe the bug
@jeschkies Using promtail's docker_sd_config, lines longer than than 64k result in an error that prevents further log lines from being consumed. The error is:

bufio.Scanner: token too long

This occurs because by default bufio.Scanner uses MaxScanTokenSize = 64 * 1024, and lines longer than the buffer will result in the above error.

See #6256 for a potential fix.

To Reproduce
Steps to reproduce the behavior:

  1. Started Promtail with docker_sd_config listening for Docker container logs
  2. Start a Docker container which generates log lines longer than 64k bytes
  3. Observe that parsing of the logs for the container fail when parsing long log lines and do not parse further lines

Expected behavior
Promtail parses Docker container log lines longer than 64k without issue.

Environment:

  • Infrastructure: bare metal
  • Deployment tool: ansible

Screenshots, Promtail config, or terminal output

From clients/pkg/promtail/targets/docker/targetmanager_test.go when a long log line is added to clients/pkg/promtail/targets/docker/testdata/flog.log and included in the tests:

level=warn msg="finished scanning logs lines with an error" err="bufio.Scanner: token too long"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant