out_loki: fix removing keys defined by labels #9766
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ajantti
requested review from
edsiper,
fujimotos,
koleini and
leonardo-albertovich
as code owners
Example Configurationservice:
flush: 1
log_level: debug
http_server: true
http_listen: 0.0.0.0
http_port: 2020
pipeline:
inputs:
- name: systemd
tag: host.*
read_from_tail: true
outputs:
- name: loki
host: 127.0.0.1
port: 3100
match: '*'
labels: $MESSAGE |
Valgrind/Debug output |
ajantti
added a commit
to ajantti/fluent-bit-docs
that referenced
this pull request
Dec 25, 2024
Related to the pull request fluent/fluent-bit#9766 The documentation doesn't make it clear, but the code seems to be trying to work this way. Also minor typo fix.
ajantti
added a commit
to ajantti/fluent-bit-docs
that referenced
this pull request
Jan 7, 2025
Related to the pull request fluent/fluent-bit#9766 The documentation doesn't make it clear, but the code seems to be trying to work this way. Also minor typo fix. Signed-off-by: jantti <jantti@tri-ace.co.jp>
When key is defined as a Loki label with labels, label_keys or label_map_path, the code is trying to remove them. However, if remove_keys is not defined, it never gets to the part that actually removes them. This looks like an obvious bug to me. I moved the debug message to inside the if (size > 0) statement, because earlier it would only print it if it was actually trying to remove something. So without moving it it would spam it every time. And absence of the message would then indicate size == 0. Not sure if this is the best, though. Signed-off-by: jantti <jantti@tri-ace.co.jp>
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
@patrick-stephens are you able to review this PR? |
|
Is this accidentally already fixed by #10563 ? |
Based on a quick test, I think it is. I didn't do extensive testing, but still, the code looks a bit wrong. I didn't reason through everything, but it feels like right now handling remove_keys_derived happens twice, once in prepare_remove_keys and once in remove_mpa_entry_create. But in prepare_remove_keys it only happens if the remove_keys is actually set in the config. The problem is that the actual adding keys for removal happens in fluent-bit/plugins/out_loki/loki.c Lines 1093 to 1100 in 6345fd1 Which is inside the earlier fluent-bit/plugins/out_loki/loki.c Line 1054 in 6345fd1 So it only gets triggered if the configuration actually sets remove_keys, even though the patterns might include keys that are derived, and initialized in fluent-bit/plugins/out_loki/loki.c Line 1051 in 6345fd1 So if the remove_mpa_entry_create is enough, it might be a good idea to remove the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When key is defined as a Loki label with labels, label_keys or label_map_path, the code is trying to remove them. However, if remove_keys is not defined, it never gets to the part that actually removes them. This looks like an obvious bug to me.
I moved the debug message to inside the if (size > 0) statement, because earlier it would only print it if it was actually trying to remove something. So without moving it it would spam it every time. And absence of the message would then indicate size == 0. Not sure if this is the best, though.
I moved the code that removes the keys to outside the if (ctx->remove_keys) statement.
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
fluent/fluent-bit-docs#1536
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.