Add custom suffix to identifiers in filestream input when needed #26669
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
botelastic
bot
added
the
needs_team
💔 Build Failed
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪Steps errors
Expand to view the steps failures
|
| Test | Results |
|---|---|
| Failed | 0 |
| Passed | 49282 |
| Skipped | 5396 |
| Total | 54678 |
kvch
force-pushed
the
feature-filebeat-filestream-add-suffix-to-source-identifiers
branch
from
1b31602
to
43d1730
Compare
so container parser can be supported completely
kvch
force-pushed
the
feature-filebeat-filestream-add-suffix-to-source-identifiers
branch
from
43d1730
to
d4182cb
Compare
kvch
changed the title
urso
reviewed
Jul 13, 2021
kvch
added
backport-v7.14.0
|
Pinging @elastic/agent (Team:Agent) |
botelastic
bot
removed
the
needs_team
urso
reviewed
Jul 13, 2021
| @@ -49,6 +49,8 @@ type CommonConfig struct { | |||
| } | |||
|
|
|||
| type Config struct { | |||
| Suffix string | |||
There was a problem hiding this comment.
I named it Suffix because from outside it is the string that is supposed to be added to the end of the state IDs. It does not matter if it contains a stream name or emojis describing the weather. The only thing that is relevant that it is a suffix.
urso
reviewed
Jul 13, 2021
| @@ -103,12 +106,16 @@ func NewConfig(pCfg CommonConfig, parsers []common.ConfigNamespace) (*Config, er | |||
| if err != nil { | |||
| return nil, fmt.Errorf("error while parsing container parser config: %+v", err) | |||
| } | |||
| if config.Stream != readjson.All { | |||
| suffix = config.Stream.String() | |||
There was a problem hiding this comment.
What if suffix (the stream) is already set?
There was a problem hiding this comment.
Only one suffix is supported at the moment.
There was a problem hiding this comment.
I have added an error.
|
note: I removed backport to 7.14. We are past FF. I don't think this is a bug fix, right? |
|
Would be nice to have a test in parser_test checking that the stream is correctly extracted from the config. Otherwise LGTM |
|
Added test |
urso
approved these changes
Jul 15, 2021
mergify bot
pushed a commit
that referenced
this pull request
Jul 15, 2021
) ## What does this PR do? This PR lets you inject suffixes to the state identifiers based on the configuration of the filestream input. For starters, it is needed by the container parser so tracking of different streams (stdout/stderr) can be done separately. ## Why is it important? Without this, the container input cannot be substituted with filestream input with a container parser. (cherry picked from commit 2876cfb)
kvch
added a commit
that referenced
this pull request
Jul 16, 2021
) (#26918) ## What does this PR do? This PR lets you inject suffixes to the state identifiers based on the configuration of the filestream input. For starters, it is needed by the container parser so tracking of different streams (stdout/stderr) can be done separately. ## Why is it important? Without this, the container input cannot be substituted with filestream input with a container parser. (cherry picked from commit 2876cfb) Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
mdelapenya
added a commit
to mdelapenya/beats
that referenced
this pull request
Jul 19, 2021
* master: (61 commits) Add disk queue unit tests based on the queuetest package [Heartbeat] redact authorization headers from logger (elastic#26892) Expose custom process metrics (elastic#26912) [gcp/billing] always quote table name identifier (elastic#26870) Add Beats central management removal to BCs (elastic#26400) Add custom suffix to identifiers in filestream input when needed (elastic#26669) Update asa-ftd-pipeline.yml (elastic#26265) Use common host parser in vsphere module (elastic#26904) [automation] Update go release version 1.16.6 (elastic#26860) Skip flaky test: filestream and harvester group (elastic#26728) [Filebeat] Remove alias fields from Suricata and Traefik module mappings (elastic#26627) docs: apm-server.auth (elastic#26831) [Automation] Update elastic stack version to 8.0.0-2f008f4a for testing (elastic#26881) Clarify the scope of start/end multiline example (elastic#26786) [Heartbeat]: update Node.js version for synthetics (elastic#26867) [fix][httpjson] Fix incorrect key for template data (elastic#26848) [httpjson] Add value_type parameter to httpjson transforms (elastic#26847) [Heartbeat]: capture error from journey/end events (elastic#26781) [Winlogbeat] Fixes for wineventlog experimental api (elastic#26826) Set agent.id to Fleet Agent ID for each metric/log monitoring input (elastic#26776) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR lets you inject suffixes to the state identifiers based on the configuration of the filestream input. For starters, it is needed by the container parser so tracking of different streams (stdout/stderr) can be done separately.
Why is it important?
Without this, the container input cannot be substituted with filestream input with a container parser.
Checklist
- [ ] I have made corresponding changes to the documentation- [ ] I have made corresponding change to the default configuration files- [ ] I have added an entry inCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.