[FIX] [Compatibility] Get the valid index patterns to use in the app in Kibana 7.10.2 - 7.13.4 #3681

Desvelao · 2021-11-04T13:29:24Z

Description

This PR adds compatibility to Kibana version >= 7.11.x to manage how the index pattern data is retrieved to check if meets the Wazuh requirements to use.

The index pattern of Wazuh alerts should own a minimum set of fields (agent.id, manager.name, rule.groups and timestamp) to be considered usable in the app.

For Kibana <7.11.0, the index pattern fields are stored as the saved object's attribute.fields property and these are getting with the next request:

GET <KIBANA_PROTOCOL>://<KIBANA_HOST>:<KIBANA_PORT>/api/saved_objects/_find?type=index-pattern&fields=title&fields=fields&per_page=9999

but on Kibana 7.11.x or newer only the scripted fields are in the attributes.fields so is required getting the fields with:

GET <KIBANA_PROTOCOL>://<KIBANA_HOST>:<KIBANA_PORT>/api/index_patterns/_fields_for_wildcard?pattern=<INDEX_PATTERN_NAME>

More information in:

Kibana 7.11.0 changelog
PR that modified the cache of index pattern fields
New Index pattern API. This API is not used in the current app PR to maintain compatibility between Kibana versions

Changes

Added a function to check if the Kibana version (kibana.version property of package.json) satisfies a specific version.
Modified some logic when getting the index patterns or refreshing them to be compatible with 7.10.2 and 7.13.4. A Kibana change was modified as getting the index pattern fields.
Added a toast when an unknown index pattern field is detected in
Kibana >7.10.2 in Modules/<MODULE>/(Dashboard/Events)
- set its lifetime to 5 minutes instead of the default time to the toast for the different versions.

Notes about the fix

The fix uses the kibana.version property of the package.json. Set this property to the Kibana version you are using.
It was tested with Kibana 7.10.2 and 7.13.4 but should work in Kibana 7.11.x, 7.12.x. For 7.14.x or newer, this should be tested to check if works.

Tests

With a Kibana without index patterns, the app should create the index patterns (alerts, monitoring, statistics).
Go to Settings/Configuration and change the default index pattern to one that matches the name of Wazuh alerts indices, save and push the button to Execute the health check. The new index pattern should be created correctly.
Add some alerts that contain not existent fields, it could use the app sample data to generate alerts of AWS/GC by example, if there is no previous related data and appears new fields, go to Modules/Amazon Web Services/Events and explore the new alerts, expand the alert information and a message should be displayed saying you need to refresh the page to apply changes (7.10.2) or update de fields (>7.11.0)
Create an index pattern from Kibana>Stack management>Index patterns that matches the name of Wazuh alerts indices and go to the app, when opening the menu, the index pattern should be retrieved and these should appear in the index pattern selector of the app. Create some index pattern that doesn't matches with the Wazuh alerts and check these don't appear in the index pattern selector

The next condition should be fulfilled:

<=7.10.2: the fields should be in the attributes.fields of the index pattern
>=7.11.0: the fields should not be in the attributes.fields of the index pattern

GET <KIBANA_PROTOCOL>://<KIBANA_HOST>:<KIBANA_PORT>/api/saved_objects/_find?type=index-pattern&fields=title&fields=fields&per_page=9999

To test it (Gherkin):

Go to Wazuh without any Index Patterns created
Given the browser is logged in the Kibana app
When the browser enters Wazuh
Then the Index Pattern will be correctly created
Go Settings/Configuration and change a useful Index Pattern
Given the browser is logged in the Wazuh Kibana app
When the user change the Index Pattern from wazuh-alerts-* to wazuh-aler* or some Index Pattern name that will match with wazuh-alerts indices.
Then the Index Pattern Selector will appear
Go Settings/Configuration and change a unhelpful Index Pattern
Given the browser is logged in the Wazuh Kibana app
When the user change the Index Pattern from wazuh-alerts-* to wazuh-aler-* or some Index Pattern name that won't match with wazuh-alerts indices.
Then the Index Pattern Selector will not appear or it will appear without the option of this new Index Pattern.
Go Stack Management/Index Pattern and add an useful Index Pattern
Given the browser is logged in the Kibana app
When the user add an Index Pattern that will match with wazuh-alerts indices like wazuh-aler* and goes to Wazuh.
Then the Index Pattern Selector will appear at the right top of the screen
Go Stack Management/Index Pattern and add an unhelpful Index Pattern
Given the browser is logged in the Kibana app
When the user add an Index Pattern that will match with wazuh-alerts indices like wazuh-aler-* and goes to Wazuh.
Then the Index Pattern Selector will not appear at the right top of the screen or it will appear without the option of this new Index Pattern.
Atributes fields contain fields that aren't scripted fields in Kibana <= 7.10.2
Given the browser is logged in the Kibana app
When the user goes to <KIBANA_PROTOCOL>://<KIBANA_HOST>:<KIBANA_PORT>/api/saved_objects/_find?type=index-pattern&fields=title&fields=fields&per_page=9999
Then there will be some fields in atributes fields that are not scripted fields.
Atributes fields contain only scripted fields in Kibana > 7.10.2
Given the browser is logged in the Kibana app
When the user goes to <KIBANA_PROTOCOL>://<KIBANA_HOST>:<KIBANA_PORT>/api/saved_objects/_find?type=index-pattern&fields=title&fields=fields&per_page=9999
Then there will not be fields in atributes fields that are not scripted fields.
Check index pattern field
Given the browser is logged in the Kibana app with sample data of wazuh disabled
When the user goes to Stack Management/Index Pattern
And the user click on the Index Pattern that is using at this moment (wazuh-alerts-* by default)
And click in the refresh button of the Index Management Section
And go to Wazuh/Configuration/Sample Data and click on include the first one
And go to /Modules/Amazon
Then there will display an alert that you have to reload page and clicking on them, the Amazon dashboard and events will be displayed properly

Issues

This PR is related to a bug reported in #3310 (comment).

the fields for Kibana 7.10.2 and Kibana >=7.11.0 - Added a function to check if the Kibana version (`kibana.version` property of `package.json`) satisfies a specific version. - Modified some logic when getting the index patterns or refreshing them to be compatible with 7.10.2 and 7.13.4. An Kibana change modified as getting the index pattern fields. - Added a toast when an unknown index pattern field is detected in Kibana >7.10.2 in Modules/<MODULE>/(Dashboard/Events) and set its life time to 5 minutes. Note: tested with Kibana 7.10.2 and 7.13.4 but should works in Kibana 7.11.x and 7.12.x

…xes for the compatibility to get the index pattern data and their fields and refresh them

Desvelao · 2021-11-04T15:11:56Z

Changes

Replaced the references of version from 7.10.2 to 7.11

frankeros · 2021-11-05T13:15:31Z

public/components/common/modules/events.tsx

@@ -170,10 +171,11 @@ export class Events extends Component {
    if (!this.state.hasRefreshedKnownFields) {
      try {
        this.setState({ hasRefreshedKnownFields: true, isRefreshing: true });
-        await PatternHandler.refreshIndexPattern();
-
+        if(satisfyKibanaVersion('<7.11')){


'<7.11' could be a constant

Yes, it is a good idea, but I think it is not necessary. The variable name could be overcomplicated.

public/components/common/modules/events.tsx

public/react-services/saved-objects.js

Machi3mfl

CR ✅

public/react-services/saved-objects.js

frankeros

LGTM!

…lds-compatibility-7.10-7.13

* doc: update default package * Fixed compatibility wazuh 4.2 - kibana 7.13.4 (#3653) * Fixed compatibility wazuh 4.2 - kibana 7.13.4 * Changed xDomain in elastic chart * Added await in getSearchRequestBody * Updated CHANGELOG Co-authored-by: Ibarra Maximiliano <maximiliano.ibarra@wazuh.com> Co-authored-by: Franco Charriol <franco.charriol@wazuh.com> * Fixed interative register windows agent screen error (#3654) * Fixed interative register windows agent screen error * Added toLowercase * Updated CHANGELOG * Update CHANGELOG.md Co-authored-by: Ibarra Maximiliano <maximiliano.ibarra@wazuh.com> Co-authored-by: Franco Charriol <franco.charriol@wazuh.com> * Update CHANGELOG.md * fix: missing async * Fix missing import on build for kibana 7.14 (#3665) * changed import of saved searches * local copy of legacy/vis_update_state * Fixed breadcrumbs style compatibility for Kibana 7.14.2 (#3668) * Parcial breadcrumb style fix * Improved beadcrumb styling and text overflow * Added changelog * Fix remove token on logout kibana 7.13 (#3670) * Fixing EuiPannels in Overview Sections and disabled text in WzMenu (#3674) * Fixing EuiPannels in Overview Sections * Fixing disabled text in Wz Menu * Fix/3673 fixing double flyout clicking in a policy (#3676) * Fixing disabled buttons in wz menu * Fixing double flyout * Fixed Group Configuration and Management configuration error after trying to going back after you save (#3672) * Solving error * Adding Changelog.md * Fixing bug editting user * Fixed kibana settings conflict on health check (#3678) * [FIX] [Compatibility] Get the valid index patterns to use in the app in Kibana 7.10.2 - 7.13.4 (#3681) * fix(compatibility): Manage how to get the index pattern data and refresh the fields for Kibana 7.10.2 and Kibana >=7.11.0 - Added a function to check if the Kibana version (`kibana.version` property of `package.json`) satisfies a specific version. - Modified some logic when getting the index patterns or refreshing them to be compatible with 7.10.2 and 7.13.4. An Kibana change modified as getting the index pattern fields. - Added a toast when an unknown index pattern field is detected in Kibana >7.10.2 in Modules/<MODULE>/(Dashboard/Events) and set its life time to 5 minutes. Note: tested with Kibana 7.10.2 and 7.13.4 but should works in Kibana 7.11.x and 7.12.x * changelog: Add PR to changelog * fix(compatibility): Replaced references from 7.10.2 to 7.11 in the fixes for the compatibility to get the index pattern data and their fields and refresh them * bump: Wazuh 4.2.5 * Replaced window location after logout * Updated CHANGELOG * [FIX] [TELEMETRY] Hide the banner (#3709) * fix(telemetry): Hide the telemetry banner - Hide the telemetry banner from UI and set a flag in the saved object that controls if the banner was seen and dismissed - Changed the `wazuh` `public` `start` method to sync intead of `async` function * changelog: Add PR to changelog * fix: Add semicolon * fix(telemetry): Hide the telemetry banner when the app frontend is to be mounted. Note the requirement to access to the app to hide and dissmiss the telemetry notice. This means the banner is visible while the user doesn't open the Wazuh app if the notice wasn't dismissed. * [FIX] [HEALH CHECK] Fix `listValidIndexPatterns.find is not a function` error (#3698) * fix(health-check): Fix "listValidIndexPatterns.find is not a function" error - Ensure the `attributes.fields` exists in the index pattern data before parsing it - Removed returning with error message or error in `SavedObjects.getListOfIndexPatterns` and `SavedObjects.getListOfWazuhValidIndexPatterns` * changelog: Add PR to changelog * added delay into request deleteToken Co-authored-by: Franco Charriol <francocharriol@gmail.com> Co-authored-by: Maximiliano Ibarra <maximilianoaibarra@gmail.com> Co-authored-by: Ibarra Maximiliano <maximiliano.ibarra@wazuh.com> Co-authored-by: Franco Charriol <franco.charriol@wazuh.com> Co-authored-by: Federico Rodriguez <federico.rodriguez@wazuh.com> Co-authored-by: Alejandro Cuéllar Peinado <alejandro.cuellar@wazuh.com> Co-authored-by: Antonio <34042064+Desvelao@users.noreply.github.com> Co-authored-by: Matias Ezequiel Moreno <matiasezequielmoreno@MacBook-Pro-de-Matias.local> Co-authored-by: Matias Ezequiel Moreno <matiasmoreno876@gmail.com>

Desvelao added the type/enhancement Enhancement issue label Nov 4, 2021

Desvelao requested a review from a team November 4, 2021 13:29

Desvelao self-assigned this Nov 4, 2021

Desvelao added 2 commits November 4, 2021 14:33

changelog: Add PR to changelog

712c4df

fix(compatibility): Replaced references from 7.10.2 to 7.11 in the fi…

c6c7406

…xes for the compatibility to get the index pattern data and their fields and refresh them

frankeros reviewed Nov 5, 2021

View reviewed changes

Machi3mfl approved these changes Nov 5, 2021

View reviewed changes

CPAlejandro reviewed Nov 5, 2021

View reviewed changes

public/react-services/saved-objects.js Outdated Show resolved Hide resolved

public/react-services/saved-objects.js Outdated Show resolved Hide resolved

fix: apply request changes

e31ea48

Desvelao requested review from frankeros and CPAlejandro November 5, 2021 15:59

frankeros approved these changes Nov 8, 2021

View reviewed changes

Merge branch '4.2-7.10' into fix/get-valid-index-patterns-refresh-fie…

c8d80d6

…lds-compatibility-7.10-7.13

frankeros merged commit 854e080 into 4.2-7.10 Nov 8, 2021

frankeros deleted the fix/get-valid-index-patterns-refresh-fields-compatibility-7.10-7.13 branch November 8, 2021 13:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FIX] [Compatibility] Get the valid index patterns to use in the app in Kibana 7.10.2 - 7.13.4 #3681

[FIX] [Compatibility] Get the valid index patterns to use in the app in Kibana 7.10.2 - 7.13.4 #3681

Desvelao commented Nov 4, 2021 •

edited by CPAlejandro

Loading

Desvelao commented Nov 4, 2021 •

edited

Loading

frankeros Nov 5, 2021

Desvelao Nov 5, 2021

Machi3mfl left a comment

frankeros left a comment

[FIX] [Compatibility] Get the valid index patterns to use in the app in Kibana 7.10.2 - 7.13.4 #3681

[FIX] [Compatibility] Get the valid index patterns to use in the app in Kibana 7.10.2 - 7.13.4 #3681

Conversation

Desvelao commented Nov 4, 2021 • edited by CPAlejandro Loading

Description

Changes

Notes about the fix

Tests

Issues

Desvelao commented Nov 4, 2021 • edited Loading

Changes

frankeros Nov 5, 2021

Choose a reason for hiding this comment

Desvelao Nov 5, 2021

Choose a reason for hiding this comment

Machi3mfl left a comment

Choose a reason for hiding this comment

frankeros left a comment

Choose a reason for hiding this comment

Desvelao commented Nov 4, 2021 •

edited by CPAlejandro

Loading

Desvelao commented Nov 4, 2021 •

edited

Loading