Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wazuh-indexer testing: Installation and uninstallation footprint #2736

Closed
jmv74211 opened this issue Mar 29, 2022 · 1 comment
Closed

Wazuh-indexer testing: Installation and uninstallation footprint #2736

jmv74211 opened this issue Mar 29, 2022 · 1 comment
Assignees

Comments

@jmv74211
Copy link
Contributor

jmv74211 commented Mar 29, 2022

Parent issue
wazuh/wazuh#12901

It is requested to carry out a study on the impact of the installation and uninstallation of the Wazuh-indexer package at the level of files and operating system files.

Specifically, we want to know if any file not related to the installation of the Wazuh-indexer package has changed its permissions, group, owner... ("check-files data").

To test this, you can make use of QA's "check-files" tool, which obtains all this necessary information. The use cases to test are as follows:

Installation

  • Get initial system check-files
  • Install Wazuh-indexer package
  • Get the system check-files again
  • Make a comparison and analyze the results

Uninstallation

  • Get initial system check-files
  • Uninstall the package Wazuh-indexer.
  • Get the system check-files again
  • Make a comparison and analyze the results

Note: All research and results obtained should be attached in comments to this issue.

@jmv74211 jmv74211 changed the title Wazuh-indexer testing: Installation and uninstallation footprint Wazuh-indexer testing: Installation and uninstallation footprint Mar 29, 2022
@roronoasins
Copy link

roronoasins commented Mar 31, 2022

Research on the wazuh-indexer check-files when installing/uninstalling

Reported issues

Issue Description Severity
wazuh/wazuh-documentation#4999 Additional steps need to be added to configure opensearch.yaml High
wazuh/wazuh-packages#1749 Warnings of wazuh-indexer confusing for the user. Mention in documentation Low
wazuh/wazuh-documentation#5007 Add note in wazuh-indexer installation guide to suggest deleting downloaded and unneeded files Low

General information

Details

In order to obtain these check-files, the step-by-step guide for installing/uninstalling wazuh-indexer has been followed.

This has been tested in Ubuntu 20.04 and CentOS 8.

check-files overview

We call check-files to a metadata structure that we collect using our tool, here we have an example:

"/usr/share/wazuh-indexer/plugins/opensearch-anomaly-detection/checker-qual-2.11.1.jar": {
  "type": "file",
  "user": "wazuh-indexer",
  "group": "wazuh-indexer",
  "mode": "640",
  "permissions": "-rw-r-----",
  "last_update": "2022-01-14 03:58:54",
  "md5sum": "936884d7b04ee8f064df137bcdfe470d",
  "size": "196.40KB"
}

This contains fields like the type, owner, permissions, last_update(mtime=ctime+atime=property changes + content access), md5sum, size, etc.

After collecting the check-files in different scenarios(before installing, after installing, after uninstalling, ect.) we can get the diff between them looking for undesired changes.

How check-files diff is collected

The check-files are used to look for changes between the states. This changes are obtained using the test located in system/filesystem_integrity like this:

python3 -m pytest filesystem_integrity/test_check_file_system_integrity.py --before-file /path/to/file --after-file /path/to/file2 --output-path /desired/output/path

DEB installation check-files 🟢

Details

The final status is green because nothing has changed, just the last_update field and md5sum in some cases. Other fields like owner, group, permissions did not change. This means that the wazuh-indexer installation does not make any changes in the filesystem.

We can see the collected check-files here and the changes between them here.

Process
  • Prepare the installation
root@windexer:/home/vagrant# curl -sO https://packages-dev.wazuh.com/4.3/wazuh-certs-tool.sh
root@windexer:/home/vagrant# curl -sO https://packages-dev.wazuh.com/4.3/config.yml
root@windexer:/home/vagrant# nano config.yml
root@windexer:/home/vagrant# bash ./wazuh-certs-tool.sh -A
29/03/2022 14:44:27 INFO: Admin certificates created.
29/03/2022 14:44:27 INFO: Wazuh indexer certificates created.
29/03/2022 14:44:27 INFO: Wazuh server certificates created.
29/03/2022 14:44:27 INFO: Wazuh dashboard certificates created.
root@windexer:/home/vagrant# tar -cvf ./wazuh-certificates.tar -C ./wazuh-certificates/ .
./
./wazuh-1.pem
./root-ca.key
./admin-key.pem
./windexer-node-1-key.pem
./root-ca.pem
./dashboard.pem
./admin.pem
./windexer-node-1.pem
./dashboard-key.pem
./wazuh-1-key.pem
root@windexer:/home/vagrant# rm -r ./wazuh-certificates
root@windexer:/home/vagrant# apt install debconf adduser procps
Reading package lists... Done
Building dependency tree       
Reading state information... Done
adduser is already the newest version (3.116ubuntu1).
adduser set to manually installed.
debconf is already the newest version (1.5.66ubuntu1).
debconf set to manually installed.
procps is already the newest version (2:3.3.12-3ubuntu1.2).
procps set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 62 not upgraded.
root@windexer:/home/vagrant# curl -s https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/key/GPG-KEY-WAZUH | apt-key add -
OK
root@windexer:/home/vagrant# echo "deb https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/pre-release/apt/ unstable main" | tee -a /etc/apt/sources.list.d/wazuh_pre_release.list
deb https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/pre-release/apt/ unstable main
  • Before run the apt install wazuh-indexer command, get the filesystem check-files:

To obtain the check-files tool we can run the following command:

 curl -OL https://raw.githubusercontent.com/wazuh/wazuh-qa/2231-test-check-files-system/deps/wazuh_testing/wazuh_testing/scripts/check_files.py
root@windexer:/home/vagrant/wazuh-qa/deps/wazuh_testing# sudo python3 /home/vagrant/wazuh-qa/deps/wazuh_testing/wazuh_testing/scripts/check_files.py -o / -o /home/vagrant/check_files/check_files_before_wazuh_indexer_installation.json --ignore /sys /dev /proc /run /home/vagrant/check_files
2022-03-29 15:12:50,663 - INFO - Ignoring the following paths: ['/sys', '/dev', '/proc', '/run', '/home/vagrant/check_files']
2022-03-29 15:12:50,663 - INFO - Getting check-files data from /
2022-03-29 15:15:37,467 - INFO - The check-files data has been written in /home/vagrant/check_files/check_files_before_wazuh_indexer_installation.json file
  • Install wazuh-indexer
root@windexer:/home/vagrant/wazuh-qa/deps/wazuh_testing# apt install wazuh-indexer
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
    wazuh-indexer
0 upgraded, 1 newly installed, 0 to remove and 63 not upgraded.
Need to get 357 MB of archives.
After this operation, 639 MB of additional disk space will be used.
Get:1 https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/pre-release/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-1 [357 MB]
Fetched 357 MB in 1min 44s (3420 kB/s)                                                                                                     
Selecting previously unselected package wazuh-indexer.
(Reading database ... 66680 files and directories currently installed.)
Preparing to unpack .../wazuh-indexer_4.3.0-1_amd64.deb ...
Creating wazuh-indexer group... OK
Creating wazuh-indexer user... OK
Unpacking wazuh-indexer (4.3.0-1) ...
Setting up wazuh-indexer (4.3.0-1) ...
Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore
Processing triggers for systemd (237-3ubuntu10.52) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
  • After installing wazuh-indexer following the step-by-step guide the check-files item has been collected:
root@windexer:/home/vagrant/wazuh-qa/deps/wazuh_testing# sudo python3 /home/vagrant/wazuh-qa/deps/wazuh_testing/wazuh_testing/scripts/check_files.py -o /home/vagrant/check_files/check_files_after_wazuh_indexer_installation.json --ignore /sys /dev /proc /run /etc/wazuh-indexer /var/lib/wazuh-indexer /var/log/wazuh-indexer
2022-03-29 15:22:15,872 - INFO - Ignoring the following paths: ['/sys', '/dev', '/proc', '/run', '/home/vagrant/check_files']
2022-03-29 15:22:15,872 - INFO - Getting check-files data from /
2022-03-29 15:24:59,092 - INFO - The check-files data has been written in /home/vagrant/check_files/check_files_after_wazuh_indexer_installation.json file

DEB uninstallation check-files 🟢

Details

The final status is green because nothing has changed, just the last_update field in some cases. Other fields like owner, group, permissions, etc. did not change. This means that the wazuh-indexer uninstallation does not make any changes in the filesystem.

We can see the collected check-files here and the changes between them here.

Process
  • Remove the Wazuh repository:
root@windexer:/home/vagrant# rm /etc/apt/sources.list.d/wazuh_pre_release.list
  • Clean the Wazuh indexer installation:
root@windexer:/home/vagrant# apt remove --purge wazuh-indexer -y
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
    wazuh-indexer*
0 upgraded, 0 newly installed, 1 to remove and 63 not upgraded.
After this operation, 639 MB disk space will be freed.
(Reading database ... 67624 files and directories currently installed.)
Removing wazuh-indexer (4.3.0-1) ...
Stopping wazuh-indexer service... OK
(Reading database ... 66691 files and directories currently installed.)
Purging configuration files for wazuh-indexer (4.3.0-1) ...
Deleting configuration directory... OK
dpkg: warning: while removing wazuh-indexer, directory '/var/lib/wazuh-indexer' not empty so not removed
Processing triggers for systemd (237-3ubuntu10.52) ...
Processing triggers for ureadahead (0.100.0-21) ...
  • Remove the wazuh-certs-tool.sh and config.yml files downloaded previously
root@windexer:/home/vagrant# rm config.yml wazuh-certs-tool.sh
  • Finally, get the check-files after the uninstallation

To obtain the check-files tool we can run the following command:

 curl -OL https://raw.githubusercontent.com/wazuh/wazuh-qa/2231-test-check-files-system/deps/wazuh_testing/wazuh_testing/scripts/check_files.py
root@windexer:/home/vagrant# python3 /home/vagrant/check_files.py -o /home/vagrant/check_files/check_files_after_wazuh_indexer_uninstallation.json --ignore /sys /dev /proc /run /home/vagrant/check_files/
2022-03-29 15:47:09,569 - INFO - Ignoring the following paths: ['/sys', '/dev', '/proc', '/run', '/home/vagrant/check_files/']
2022-03-29 15:47:09,569 - INFO - Getting check-files data from /
2022-03-29 15:51:48,022 - INFO - The check-files data has been written in /home/vagrant/check_files/check_files_after_wazuh_indexer_uninstallation.json file

RPM installation check-files 🟢

Details

The final status is green because nothing has changed, just the last_update field and md5sum in some cases. Other fields like owner, group, permissions did not change. This means that the wazuh-indexer installation does not make any changes in the filesystem.

We can see the collected check-files here and the changes between them here.

Process
  • Prepare the installation
[root@windexer2 vagrant]# rpm --import https://packages.wazuh.com/key/GPG-KEY-WAZUH
[root@windexer2 vagrant]# echo -e '[wazuh_pre_release]\ngpgcheck=1\ngpgkey=https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/key/GPG-KEY-WAZUH\nenabled=1\nname=EL-$releasever - Wazuh\nbaseurl=https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/pre-release/yum/\nprotect=1' | tee /etc/yum.repos.d/wazuh_pre.repo
[wazuh_pre_release]
gpgcheck=1
gpgkey=https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/key/GPG-KEY-WAZUH
enabled=1
name=EL-$releasever - Wazuh
baseurl=https://s3-us-west-1.amazonaws.com/packages-dev.wazuh.com/pre-release/yum/
protect=1
  • Before run the apt install wazuh-indexer command, get the filesystem check-files:

To obtain the check-files tool we can run the following command:

 curl -OL https://raw.githubusercontent.com/wazuh/wazuh-qa/2231-test-check-files-system/deps/wazuh_testing/wazuh_testing/scripts/check_files.py
root@windexer:/home/vagrant/wazuh-qa/deps/wazuh_testing# sudo python3 /home/vagrant/wazuh-qa/deps/wazuh_testing/wazuh_testing/scripts/check_files.py -o / -o /home/vagrant/check_files/check_files_before_wazuh_indexer_installation.json --ignore /sys /dev /proc /run /home/vagrant/check_files
2022-03-29 15:12:50,663 - INFO - Ignoring the following paths: ['/sys', '/dev', '/proc', '/run', '/home/vagrant/check_files']
2022-03-29 15:12:50,663 - INFO - Getting check-files data from /
2022-03-29 15:15:37,467 - INFO - The check-files data has been written in /home/vagrant/check_files/check_files_before_wazuh_indexer_installation.json file
  • Install wazuh-indexer
[root@windexer2 vagrant]# yum install -y wazuh-indexer

EL-8 - Wazuh                                                                                                                                                                   1.5 MB/s | 5.4 MB     00:03    
Last metadata expiration check: 0:00:03 ago on Wed Mar 30 17:03:52 2022.
Dependencies resolved.
===============================================================================================================================================================================================================
    Package                                             Architecture                                 Version                                        Repository                                               Size
===============================================================================================================================================================================================================
Installing:
    wazuh-indexer                                       x86_64                                       4.3.0-1                                        wazuh_pre_release                                       361 M

Transaction Summary
===============================================================================================================================================================================================================
Install  1 Package

Total download size: 361 M
Installed size: 614 M
Downloading Packages:
wazuh-indexer-4.3.0-1.x86_64.rpm                                                                                                                                               7.8 MB/s | 361 MB     00:46    
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                          7.8 MB/s | 361 MB     00:46     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
    Preparing        :                                                                                                                                                                                       1/1 
    Running scriptlet: wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 
    Installing       : wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 
    Running scriptlet: wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 
Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore

    Verifying        : wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 

Installed:
    wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                                                 

Complete!

  • After installing wazuh-indexer following the step-by-step guide the check-files item has been collected:
[root@windexer2 vagrant]# sudo python3 check_files.py -o /home/vagrant/check_files/check_files_after_wazuh_indexer_installation.json --ignore /sys /dev /proc /run /home/vagrant/check_files
2022-03-30 17:10:27,207 - INFO - Ignoring the following paths: ['/sys', '/dev', '/proc', '/run', '/home/vagrant/check_files']
2022-03-30 17:10:27,207 - INFO - Getting check-files data from /
2022-03-30 17:10:49,317 - INFO - The check-files data has been written in /home/vagrant/check_files/check_files_after_wazuh_indexer_installation.json file

RPM uninstallation check-files 🟢

Details

The final status is green because nothing has changed, just the last_update field and md5sum in some cases. Other fields like owner, group, permissions did not change. This means that the wazuh-indexer installation does not make any changes in the filesystem.

We can see the collected check-files here and the changes between them here.

Process
  • Remove the Wazuh repository:
[root@windexer2 vagrant]# rm /etc/yum.repos.d/wazuh_pre.repo 
  • Clean the Wazuh indexer installation:

[root@windexer2 vagrant]# yum remove wazuh-indexer -y
Dependencies resolved.
===============================================================================================================================================================================================================
Package                                             Architecture                                 Version                                       Repository                                                Size
===============================================================================================================================================================================================================
Removing:
wazuh-indexer                                       x86_64                                       4.3.0-1                                       @wazuh_pre_release                                       614 M

Transaction Summary
===============================================================================================================================================================================================================
Remove  1 Package

Freed space: 614 M
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing        :                                                                                                                                                                                       1/1 
Running scriptlet: wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 
Stopping wazuh-indexer service... OK

Erasing          : wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 
Running scriptlet: wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 
Verifying        : wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                          1/1 

Removed:
wazuh-indexer-4.3.0-1.x86_64                                                                                                                                                                                 

Complete!
[root@windexer2 vagrant]# rm -rf /var/lib/wazuh-indexer/
[root@windexer2 vagrant]# rm -rf /usr/share/wazuh-indexer/
[root@windexer2 vagrant]# rm -rf /etc/wazuh-indexer/

  • Remove the wazuh-certs-tool.sh and config.yml files downloaded previously
[root@windexer2 vagrant]# rm config.yml wazuh-certs-tool.sh
  • Finally, get the check-files after the uninstallation

To obtain the check-files tool we can run the following command:

 curl -OL https://raw.githubusercontent.com/wazuh/wazuh-qa/2231-test-check-files-system/deps/wazuh_testing/wazuh_testing/scripts/check_files.py
[root@windexer2 vagrant]# python3 check_files.py -o /home/vagrant/check_files/check_files_after_wazuh_indexer_uninstallation.json --ignore /sys /dev /proc /run /home/vagrant/check_files
2022-03-30 17:15:24,733 - INFO - Ignoring the following paths: ['/sys', '/dev', '/proc', '/run', '/home/vagrant/check_files']
2022-03-30 17:15:24,733 - INFO - Getting check-files data from /
2022-03-30 17:15:46,017 - INFO - The check-files data has been written in /home/vagrant/check_files/check_files_after_wazuh_indexer_uninstallation.json file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants