Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 4.3.0 - Manual tests - Wazuh indexer - Second revision #2793

Closed
Rebits opened this issue Apr 18, 2022 · 11 comments
Closed

Release 4.3.0 - Manual tests - Wazuh indexer - Second revision #2793

Rebits opened this issue Apr 18, 2022 · 11 comments
Assignees

Comments

@Rebits
Copy link
Member

Rebits commented Apr 18, 2022

After wazuh/wazuh#12901 testing some issues were reported. With the creation of a new release candidate is required to evaluate the current status of those anomalous behaviors and perform a report with the global status.

Test information

Test name Wazuh indexer
Category Wazuh indexer
Deployment option Step by step
Main release issue wazuh/wazuh#10954
Release candidate # RC5

Test description

Environment to be deployed and configured

Node Service System Resources
node_1 wazuh_indexer_1 DEB 2GB - 2CPU
node_2 wazuh_indexer_2 RPM 2GB - 2CPU
node_3 wazuh_indexer_3 RPM 2GB - 2CPU
node_4 wazuh_master RPM 1GB - 1CPU
node_5 wazuh_worker_1 DEB 1GB - 1CPU
node_6 wazuh_worker_2 RPM 1GB - 1CPU

Test report procedure

Status Description
Not started
🔵 Tests and reviews are in progress
🟢 Tests and evaluations have been passed successfully
🟡 Possible improvement is suggested
🔴 There is some erroneous behavior

Reported issues to review

Status Issue Description Severity
🟢 wazuh/wazuh-documentation#4999 Additional steps need to be added to configure opensearch.yaml High
🟢 wazuh/wazuh-packages#1413 Fix the yml format of the config.yml file to generate the certificates Low
🔴 wazuh/wazuh-packages#1414 Restarting the wazuh-indexer service remains blocked indefinitely. High
🔴 wazuh/wazuh-packages#1749 Warnings of wazuh-indexer confusing for the user. Mention in documentation Low
🔴 wazuh/wazuh-packages#1885 Infinite loop when running indexer-security-init.sh High
🟢 wazuh/wazuh-documentation#5007 Add note in wazuh-indexer installation guide to suggest deleting downloaded files Low
🟢 wazuh/wazuh#12934 Invalid config.yml format when editing with vi editor (CRLF issue) Medium
🔴 wazuh/wazuh#12973 Update kibana_read_only role name Low
🔴 wazuh/wazuh-dashboard-plugins#3941 Permission error in wazuh-dashboard logs when running for first time Low
🔴 wazuh/wazuh#13013 Indices from wazuh-indexer are not created when reinstalling with yum Medium
🟢 wazuh/wazuh-documentation#5008 Suggestions to add to the wazuh-indexer documentation Low
@fedepacher
Copy link
Contributor

fedepacher commented Apr 18, 2022

Invalid config.yml format when editing with vi editor (CRLF issue)

Reported issues

Issue Description
wazuh/wazuh-packages#1421 Invalid config.yml format when editing with vi editor (CRLF issue)

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

Testing the usability and user experience when generating the necessary certificates for encrypted communication between components and cluster configuration.
In the pass, It was found the following error while generating the certificates using the script wazuh-certs-tool.sh by editing the config.yml file:

ERROR: The given information does not match with an IP address or a DNS.

Test 🟢

Description

It has been downloaded the following file:

# curl -sO https://packages-dev.wazuh.com/4.3/wazuh-certs-tool.sh
# curl -sO https://packages-dev.wazuh.com/4.3/config.yml

Edited the config.yml and replace the node names and IP values with the corresponding names and IP addresses. After that, the script to generate the certs has been run.

Details

Configuration file config.yml

nodes:
  # Wazuh indexer nodes
  indexer:
    - name: node-1
      ip: 172.18.1.10
    - name: node-2
      ip: 172.18.1.11
    - name: node-3
      ip: 172.18.1.12

  # Wazuh server nodes
  # Use node_type only with more than one Wazuh manager
  server:
    - name: wazuh-1
      ip: 172.18.1.13
      node_type: master
    - name: wazuh-2
      ip: 172.18.1.14
      node_type: worker

  # Wazuh dashboard nodes
  dashboard:
    - name: dashboard
      ip: 172.18.1.16

Console output results

[root@qactl-centos-8-1 vagrant]# bash ./wazuh-certs-tool.sh -A
18/04/2022 14:05:08 INFO: Admin certificates created.
18/04/2022 14:05:09 INFO: Wazuh indexer certificates created.
18/04/2022 14:05:09 INFO: Wazuh server certificates created.
18/04/2022 14:05:09 INFO: Wazuh dashboard certificates created.

Comments

The problem has been solved in RC6.

@fedepacher
Copy link
Contributor

fedepacher commented Apr 18, 2022

Update kibana_read_only role name

Reported issues

Issue Description
wazuh/wazuh#12973 Update kibana_read_only role name

Details

Wazuh version Install type Platform
4.3 Wazuh-Dashboard CentOS 8

Description

The file that contains the Wazuh-Dashboard configuration /etc/wazuh-dashboard/opensearch_dashboards.yml has a variable name kibana_read_only that could create confusion.

Test 🔴

Description

Wazuh-Dashboard has been installed and it has been checked if the kibana read_only keeps remaining in the /etc/wazuh-dashboard/opensearch _dashboards.yml file.

Details

Configuration file /etc/wazuh-dashboard/opensearch_dashboards.yml

server.host: 0.0.0.0
server.port: 443
opensearch.hosts: ["https://192.168.0.14:9200", "https://192.168.0.13:9200","https://192.168.0.17:9200"]
opensearch.ssl.verificationMode: certificate
#opensearch.username:
#opensearch.password:
opensearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opensearch_security.multitenancy.enabled: true
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
server.ssl.enabled: true
server.ssl.key: "/etc/wazuh-dashboard/certs/dashboard-key.pem"
server.ssl.certificate: "/etc/wazuh-dashboard/certs/dashboard.pem"
opensearch.ssl.certificateAuthorities: ["/etc/wazuh-dashboard/certs/root-ca.pem"]
uiSettings.overrides.defaultRoute: /app/wazuh?security_tenant=global

Comments

The variable name kibana_read_only still remain in RC6 but it will be solved in a future release as it says in this comment.

@fedepacher
Copy link
Contributor

Permission error in wazuh-dashboard logs when running for first time

Reported issues

Issue Description
wazuh/wazuh-dashboard-plugins#3941 Permission error in wazuh-dashboard logs when running for first time

Details

Wazuh version Install type Platform
4.3 Wazuh-Dashboard CentOS 8

Description

After doing a clean installation of the Wazuh-Dashboard, the following error can be seen in the log /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log:

ERROR  Could not check if the index .wazuh exists due to no permissions for create, delete or check

Test 🔴

Description

Wazuh-Dashboard has been installed and it has been checked if the /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log file.

Details

Log file /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log

[root@qactl-centos-8-6 vagrant]# cat /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log
{"date":"2022-04-18T22:36:30.020Z","level":"info","location":"initialize","message":"Wazuh dashboard index: .kibana"}
{"date":"2022-04-18T22:36:30.021Z","level":"info","location":"initialize","message":"App revision: 4301-1"}
{"date":"2022-04-18T22:36:30.021Z","level":"info","location":"initialize","message":"Total RAM: 970MB"}
{"date":"2022-04-18T22:36:30.308Z","level":"error","location":"initialize:checkKibanaStatus","message":"Could not check if the index .wazuh exists due to no permissions for create, delete or check"}
{"date":"2022-04-18T22:36:30.909Z","level":"error","location":"monitoring:getApiInfo","message":"connect ECONNREFUSED 127.0.0.1:55000"}
{"date":"2022-04-18T22:40:00.747Z","level":"error","location":"cron-scheduler|SaveDocument","message":"resource_already_exists_exception"}

Comments

The error messages still remain in RC6.

@fedepacher
Copy link
Contributor

fedepacher commented Apr 18, 2022

Infinite loop when restarting wazuh-indexer with configuration error

Reported issues

Issue Description
wazuh/wazuh-packages#1414 Infinite loop when restarting wazuh-indexer with configuration error

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

While testing for the wazuh-indexer package in 4.3.0-rc6, I noticed that if you restart the wazuh-indexer service with an error in the /etc/wazuh-indexer/opensearch.yml configuration file, the process does not end up staying in an infinite loop and without showing any type of error.

Test 🔴

Description

Edit the file /etc/wazuh-indexer/opensearch.yml and set network.host to the following value

network.host: asd
Details

Result

Restart the wazuh-indexer service:

systemctl restart wazuh-indexer

Comments

The process will be stuck indefinitely in RC6 as well and the explanation why is specified here.

@fedepacher
Copy link
Contributor

Request for changes on the Wazuh 4.3 installation guide

Reported issues

Issue Description
wazuh/wazuh-documentation#4999 Request for changes on the Wazuh 4.3 installation guide

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

As part of the manual tests of Wazuh 4.3 that are being performed, where the installation guide must be followed and verified, some errors were detected. Part of the errors found are related to the description of the steps that are needed to be followed to perform the installation and others appeared when trying to execute the installation commands.

Test 🟢

Description

It has been performance the step-by-step installation guide without any problem

Details

Result

Installation clean and completed succesfully

Comments

The step-by-step installation guide now it complete and well described steps in RC6.

@fedepacher
Copy link
Contributor

Fix the yml format of the config.yml file to generate the certificates

Reported issues

Issue Description
wazuh/wazuh-packages#1413 Fix the yml format of the config.yml file to generate the certificates

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

The content of the config.yml file does not comply with the actual yaml syntax.

Test 🟢

Description

It has been checked the content of config.yml file to check if the syntax complies with the actual yaml syntax.

Details

Result

nodes:
  # Wazuh indexer nodes
  indexer:
    - name: node-1
      ip: 192.168.0.14
    - name: node-2
      ip: 192.168.0.13
    - name: node-3
      ip: 192.168.0.17

  # Wazuh server nodes
  # Use node_type only with more than one Wazuh manager
  server:
    - name: wazuh-1
      ip: 192.168.0.19
      node_type: master
    - name: wazuh-2
      ip: 192.168.0.20
      node_type: worker

  # Wazuh dashboard nodes
  dashboard:
    - name: dashboard
      ip: 192.168.0.18

Comments

The content of the config.yml file now is what is expected in a yaml file in RC6.

@fedepacher
Copy link
Contributor

fedepacher commented Apr 18, 2022

Warnings messages when checking Wazuh-Indexer status

Reported issues

Issue Description
wazuh/wazuh-packages#1749 Warnings messages when checking Wazuh-Indexer status

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

Checking the status of wazuh-indexer the following WARNINGS come out in the console:

Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: An illegal reflective access operation has occurred
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/wazuh-indexer/plugins/opensearch-anomaly-detec>
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: All illegal access operations will be denied in a future release

Test 🔴

Description

To get WARNINGS messages the following command has been executed:

systemctl status wazuh-indexer
Details

Result

● wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-04-18 23:09:22 UTC; 23min ago
     Docs: https://documentation.wazuh.com
 Main PID: 51037 (java)
    Tasks: 47 (limit: 24918)
   Memory: 1.3G
   CGroup: /system.slice/wazuh-indexer.service
           └─51037 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.t>

Apr 18 23:09:08 qactl-centos-8-1 systemd[1]: Starting Wazuh-indexer...
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: An illegal reflective access operati>
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: Illegal reflective access by io.prot>
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: Please consider reporting this to th>
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: Use --illegal-access=warn to enable >
Apr 18 23:09:18 qactl-centos-8-1 systemd-entrypoint[51037]: WARNING: All illegal access operations will b>
Apr 18 23:09:22 qactl-centos-8-1 systemd[1]: Started Wazuh-indexer.

Comments

The WARNINGS messages remain in the status and in the documentation does not say anything about it in RC6 but the explanation why is still coming up is here.

@roronoasins
Copy link

Indices from wazuh-indexer are not created when reinstalling the indexer with yum

Reported issues

Issue Description
wazuh/wazuh#13013 Indices from wazuh-indexer are not created when reinstalling with yum

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

When we reinstall the component with yum, the alerts are not indexed anymore until we restart the services

Test 🔴

Description

After reinstalling the component It is needed to restart the related services so the indices are indexed again.

Details

Result

When installing the indexer all seems good.

[root@windexer1 vagrant]# curl -k -u admin:admin https://172.16.1.116:9200/_cat/indices?v
health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   security-auditlog-2022.04.19 hTYpX-rNS0Gmf1_YIwlStg   1   1         20            0    270.5kb        270.5kb
green  open   wazuh-alerts-4.x-2022.04.19  WeSQoVu9SXuaDx5NsZ2OJA   3   0        210            0    462.4kb        462.4kb
green  open   .opendistro_security         01ZQ5bUuRH-Vsbq3jmb2Bg   1   0          9            8     92.6kb         92.6kb
[root@wmaster vagrant]# filebeat test output
elasticsearch: https://172.16.1.116:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 172.16.1.116
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

Then, we uninstall it.

[root@windexer1 vagrant]# yum remove wazuh-indexer -y
Failed to set locale, defaulting to C.UTF-8
Dependencies resolved.
======================================================================================================
 Package                  Architecture      Version               Repository                     Size
======================================================================================================
Removing:
 wazuh-indexer            x86_64            4.3.0-1               @wazuh_pre_release            614 M

Transaction Summary
======================================================================================================
Remove  1 Package

Freed space: 614 M
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                              1/1 
  Running scriptlet: wazuh-indexer-4.3.0-1.x86_64                                                 1/1 
Stopping wazuh-indexer service... OK

  Erasing          : wazuh-indexer-4.3.0-1.x86_64                                                 1/1 
warning: /etc/wazuh-indexer/opensearch.yml saved as /etc/wazuh-indexer/opensearch.yml.rpmsave

  Running scriptlet: wazuh-indexer-4.3.0-1.x86_64                                                 1/1 
  Verifying        : wazuh-indexer-4.3.0-1.x86_64                                                 1/1 

Removed:
  wazuh-indexer-4.3.0-1.x86_64                                                                        

Complete!
[root@windexer1 vagrant]# rm -rf /var/lib/wazuh-indexer/
[root@windexer1 vagrant]# rm -rf /usr/share/wazuh-indexer/
[root@windexer1 vagrant]# rm -rf /etc/wazuh-indexer/

When reinstalling the component, the alerts indices are not indexed .

[root@windexer1 vagrant]# curl -k -u admin:admin https://172.16.1.116:9200/_cat/indices?v
health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .opendistro_security RFt51CGTQISsQlClomMYSQ   1   0          9            0     59.6kb         59.6kb

When restarting the services, the alerts show up.

[root@wmaster vagrant]# systemctl restart filebeat
[root@wmaster vagrant]# systemctl restart wazuh-manager
[root@windexer1 vagrant]# curl -k -u admin:admin https://172.16.1.116:9200/_cat/indices?v
health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   security-auditlog-2022.04.19 OKpSQQVKQMWWut_gwso-4w   1   1          2            0     29.1kb         29.1kb
green  open   wazuh-alerts-4.x-2022.04.19  _QRi7p3CT0m2-pd_zRKieA   3   0          0            0       208b           208b
green  open   .opendistro_security         RFt51CGTQISsQlClomMYSQ   1   0          9            0     59.6kb         59.6kb

@roronoasins
Copy link

Request for changes on the Wazuh 4.3 installation guide

Reported issues

Issue Description
wazuh/wazuh-documentation#5007 Request for changes on the Wazuh 4.3 installation guide

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

As part of the manual tests of Wazuh 4.3 that are being performed, where the installation guide must be followed and verified, some suggestions were provided. This was discussed with the team and we are not adding steps for removal recomendations in our guides.

Test 🟢

@fedepacher
Copy link
Contributor

Missing information in the documentation when configuring wazuh-Indexer

Reported issues

Issue Description
wazuh/wazuh-documentation#5008 Suggestions to add to the wazuh-indexer documentation

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

The /etc/wazuh-indexer/opensearch.yml file has some fields that are not well commented and some others fields that are not commented.

Test 🟢

Description

The requested information was added in this release.

Details

Result

In the following link, the missing information was added.

Comments

This issue was solved in RC6.

@fedepacher
Copy link
Contributor

fedepacher commented Apr 19, 2022

Infinite loop when running indexer-security-init.sh

Reported issues

Issue Description
wazuh/wazuh-packages#1885 Infinite loop when running indexer-security-init.sh

Details

Wazuh version Install type Platform
4.3 Wazuh-Indexer CentOS 8

Description

When trying to execute the indexer-security-init.sh script and getting an error (probably due to configuration or bad certificates), the wazuh-indexer remains in an infinite loop executing indexer-security-init.sh all the time, leaving the terminal unusable.

Test 🔴

Description

To achieve this problem, I installed 2 wazuh-indexer following the documentation of this link and I loaded bad certs to one of the nodes and then run the command in both nodes:

indexer-security-init.sh
Details

Result

[root@qactl-centos-8-8 vagrant]# /usr/share/wazuh-indexer/bin/indexer-security-init.sh
Security Admin v7
Will connect to 192.168.0.18:9300 ... done
Connected as CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US
OpenSearch Version: 1.2.4
OpenSearch Security Version: 1.2.4.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Cannot retrieve cluster state due to: null. This is not an error, will keep on trying ...
  Root cause: MasterNotDiscoveredException[null] (org.opensearch.discovery.MasterNotDiscoveredException/org.opensearch.discovery.MasterNotDiscoveredException)
   * Try running securityadmin.sh with -icl (but no -cl) and -nhnv (If that works you need to check your clustername as well as hostnames in your TLS certificates)
   * Make sure that your keystore or PEM certificate is a client certificate (not a node certificate) and configured properly in opensearch.yml
   * If this is not working, try running securityadmin.sh with --diagnose and see diagnose trace log file)
   * Add --accept-red-cluster to allow securityadmin to operate on a red cluster.

Comments

This problem remains in RC6 and the explanation why is here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants