Filebeat: Ensure module pipelines compatibility with previous versions of Elasticsearch (backport #26737) #27841
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
…s of Elasticsearch (#26737) Improve the pipeline/compatibility code so that all processors in a pipeline are scanned and acted-upon to ensure compatibility. This means: - Scan processors in on_failure section (both the pipeline's and each individual processor on-failure handler). - Scan the inner processor in a foreach processor. Add a new CI stage, module-compat-7.11, to filebeat/Jenkinsfile.yml and x-pack/filebeat/Jenkinsfile.yml, in order to run Filebeat modules tests under ES 7.11.2 to ensure that all pipelines are functional. This test uses a new flag, TESTING_FILEBEAT_SKIP_DIFF, to instruct the integration test to skip the comparison between the generated documents and the golden/expected files. The test will ensure that the pipeline loads, there are no ingest errors, the fields in the generated documents are valid and the number of returned documents matches the expected. This is intended to avoid having to maintain multiple versions of the golden files due to differences between ES versions and available processors. Also fixes the fortinet and threatintel modules pipelines so that they pass the new test, as some fields were left behind due to the uri_parts processor being removed. (cherry picked from commit 181cf69) # Conflicts: # x-pack/filebeat/module/threatintel/abuseurl/ingest/pipeline.yml # x-pack/filebeat/module/threatintel/anomali/ingest/pipeline.yml
mergify
bot
added
the
conflicts
Merged
3 tasks
botelastic
bot
added
the
needs_team
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
botelastic
bot
removed
the
needs_team
andrewkroh
added
Filebeat
botelastic
bot
removed
the
needs_team
andrewkroh
added
backport
needs_team
botelastic
bot
removed
the
needs_team
|
This pull request doesn't have a |
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
…beat tests (#19896) These setting has been removed as per elastic/elasticsearch#59262 causing the ES container to not launch. elasticsearch_1 | java.lang.IllegalArgumentException: unknown setting [script.cache.max_size] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
|
I added in #19896 b/c there was a dependency on it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an automatic backport of pull request #26737 done by Mergify.
Cherry-pick of 181cf69 has failed:
To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally
Mergify commands and options
More conditions and actions can be found in the documentation.
You can also trigger Mergify actions by commenting on this pull request:
@Mergifyio refreshwill re-evaluate the rules@Mergifyio rebasewill rebase this PR on its base branch@Mergifyio updatewill merge the base branch into this PR@Mergifyio backport <destination>will backport this PR on<destination>branchAdditionally, on Mergify dashboard you can:
Finally, you can contact us on https://mergify.io/