input: use pthread_key to handle flb_libco_in_params

[nokute78]

#7061
Note: Valgrind reports too many errors. We should check that this patch causes it.

I modified to use pthread_key to handle flb_libco_in_params.
Currently flb_libco_in_params is a global variable, I suspect it causes race error.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• [N/A] Debug log output from testing the change

• [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• [N/A] Documentation required for this feature

Backporting

• [N/A] Backport to latest stable release.

Configuration

[SERVICE]
    Flush        0.5
    Daemon       Off
    Log_Level    info

[INPUT]
    name  event_type
    type  logs
    interval_sec 0
    interval_nsec 500

[INPUT]
    name  event_type
    type  metrics
    interval_sec 0
    interval_nsec 500

[OUTPUT]
    Name  vivo_exporter
    Match *

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[nokute78]

One of valgrind error is a data race of ring_buffer.
Please see #6776 (comment)
I think it is not releated this PR.

[nokute78]

One of valgrind error is send_metrics of in_event_type.
I think it is not related this PR.

Some cmt resources are created at send_metrics, but they are not released.

fluent-bit/plugins/in_event_type/event_type.c

Line 102 in 1d96914

static int send_metrics(struct flb_input_instance *ins)

We can test using following configuration.

[SERVICE]
    Flush        0.5
    Daemon       Off
    Log_Level    error

[INPUT]
    name  event_type
    type  metrics
    interval_sec 0
    interval_nsec 500

[OUTPUT]
    Name  vivo_exporter
    Match *

Valgrind error is like this.

==32282== 3,151 (136 direct, 3,015 indirect) bytes in 1 blocks are definitely lost in loss record 22,798 of 23,940
==32282==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x7A2010: cmt_create (cmetrics.c:45)
==32282==    by 0x41DDED: send_metrics (event_type.c:116)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9C8C22F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (88 direct, 3,063 indirect) bytes in 1 blocks are definitely lost in loss record 22,799 of 23,940
==32282==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x779F12: cmt_counter_create (cmt_counter.c:54)
==32282==    by 0x41DE50: send_metrics (event_type.c:118)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9C8C22F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (200 direct, 2,951 indirect) bytes in 1 blocks are definitely lost in loss record 22,800 of 23,940
==32282==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x792132: cmt_map_create (cmt_map.c:38)
==32282==    by 0x779FE5: cmt_counter_create (cmt_counter.c:69)
==32282==    by 0x41DE50: send_metrics (event_type.c:118)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9C8C22F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (120 direct, 3,031 indirect) bytes in 1 blocks are definitely lost in loss record 22,801 of 23,940
==32282==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x79236C: map_metric_create (cmt_map.c:106)
==32282==    by 0x79277A: cmt_map_metric_get (cmt_map.c:232)
==32282==    by 0x77A105: cmt_counter_inc (cmt_counter.c:106)
==32282==    by 0x41DF2E: send_metrics (event_type.c:126)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9C8C22F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (24 direct, 3,127 indirect) bytes in 1 blocks are definitely lost in loss record 22,802 of 23,940
==32282==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x7921F0: cmt_map_create (cmt_map.c:56)
==32282==    by 0x781E0C: cmt_summary_create (cmt_summary.c:85)
==32282==    by 0x41E552: send_metrics (event_type.c:171)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9C8C22F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (120 direct, 3,031 indirect) bytes in 1 blocks are definitely lost in loss record 22,803 of 23,940
==32282==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x79236C: map_metric_create (cmt_map.c:106)
==32282==    by 0x79277A: cmt_map_metric_get (cmt_map.c:232)
==32282==    by 0x77A105: cmt_counter_inc (cmt_counter.c:106)
==32282==    by 0x41DF2E: send_metrics (event_type.c:126)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9BC058F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (200 direct, 2,951 indirect) bytes in 1 blocks are definitely lost in loss record 22,804 of 23,940
==32282==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x792132: cmt_map_create (cmt_map.c:38)
==32282==    by 0x78677C: cmt_histogram_create (cmt_histogram.c:274)
==32282==    by 0x41E29A: send_metrics (event_type.c:148)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9BC058F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282== 
==32282== 3,151 (24 direct, 3,127 indirect) bytes in 1 blocks are definitely lost in loss record 22,805 of 23,940
==32282==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==32282==    by 0x7921F0: cmt_map_create (cmt_map.c:56)
==32282==    by 0x78677C: cmt_histogram_create (cmt_histogram.c:274)
==32282==    by 0x41E29A: send_metrics (event_type.c:148)
==32282==    by 0x41EC85: cb_collector_time (event_type.c:353)
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)
==32282==    by 0xB3DC2A: co_init (amd64.c:117)
==32282==    by 0x9BC058F: ???
==32282==    by 0x1CF539: input_pre_cb_collect (flb_input.h:513)

[edsiper]

thank you.

I pushed #7145 to fix the leaks in in_event_type and the lack of release of the ring buffer, thanks for the hints

[nokute78]

I also reported out_vivo_exporter bug.
#7203

Some monkey plugin may cause it.