Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in_process_exporter_metrics backtraces if process dies during scrape #9548

Open
PromyLOPh opened this issue Nov 4, 2024 · 0 comments · May be fixed by #9549
Open

in_process_exporter_metrics backtraces if process dies during scrape #9548

PromyLOPh opened this issue Nov 4, 2024 · 0 comments · May be fixed by #9549

Comments

@PromyLOPh
Copy link

Bug Report

Describe the bug

The process exporter input plugin dies when scraping alot of processes.

[2024/11/04 11:36:21] [engine] caught signal (SIGSEGV)
#0  0x7fc835eeefe6      in  __GI___readdir64() at inux/readdir64.c:37
#1  0x55fa41c295f8      in  process_proc_fds() at plugins/in_process_exporter_metrics/pe_process.c:767
#2  0x55fa41c2abcc      in  process_update() at plugins/in_process_exporter_metrics/pe_process.c:1131
#3  0x55fa41c2b059      in  pe_process_update() at plugins/in_process_exporter_metrics/pe_process.c:1181
#4  0x55fa41c1f14c      in  update_metrics() at plugins/in_process_exporter_metrics/pe.c:33
#5  0x55fa41c1f17e      in  cb_pe_collect() at plugins/in_process_exporter_metrics/pe.c:46
#6  0x55fa41a572a9      in  input_collector_fd() at src/flb_input_thread.c:166
#7  0x55fa41a57c2e      in  engine_handle_event() at src/flb_input_thread.c:181
#8  0x55fa41a57c2e      in  input_thread() at src/flb_input_thread.c:410
#9  0x55fa41ab4d8e      in  step_callback() at src/flb_worker.c:43
#10 0x7fc835ea8143      in  start_thread() at c:442
#11 0x7fc835f287db      in  clone3() at inux/x86_64/clone3.S:81
#12 0xffffffffffffffff  in  ???() at ???:0

To Reproduce

  • Run fluent-bit -p scrape_interval=1 -i process_exporter_metrics -o stdout.
  • Run while true; do for i in $(seq 10000); do sleep 5 & done; sleep 5; done at the same time.
  • Observe fluent-bit crashing.

Expected behavior

No crash.

Screenshots

n/a

Your Environment

  • Version used: commit 104aceb
  • Configuration: (none)
  • Environment name and version (e.g. Kubernetes? What version?): (none)
  • Server type and version: (none)
  • Operating System and version: Debian 12
  • Filters and plugins: process_exporter and stdout

Additional context

Discovered while debugging #9547

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant