Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bash scripts for MVP validation tests #482

Merged
merged 26 commits into from
Nov 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
277d2e9
Add MVP validation tests bash scripts
QU3B1M Oct 21, 2024
11f2d29
Add validations for generated index-patterns
QU3B1M Oct 22, 2024
5f75f03
Update scripts to support debian ARM
QU3B1M Oct 23, 2024
1dcd7d3
Update validations scripts to be able to use the generated package name
QU3B1M Oct 23, 2024
0a535b5
Add argument to define certificates path
QU3B1M Oct 23, 2024
bfdfc7d
Update OS detection on scripts
QU3B1M Oct 23, 2024
d00e12c
Add dependencies validations
QU3B1M Oct 23, 2024
2935bfe
Add usage description to each script and a simple README
QU3B1M Oct 24, 2024
0836924
Add dependencies validations
QU3B1M Oct 24, 2024
60f45e1
Fix typos
AlexRuiz7 Oct 28, 2024
1e2ac23
Merge branch 'master' into enhancement/478-validation-tests
AlexRuiz7 Oct 28, 2024
4e13c97
Apply SpellCheck linter recommendations
QU3B1M Oct 28, 2024
38d13e0
Skip checks related to SC2181 where the fix is not applicable
QU3B1M Oct 28, 2024
5a7f8ad
Remove unnecesary double quotes from certificates generation script
QU3B1M Oct 29, 2024
3e77550
Update variable quoting
QU3B1M Oct 29, 2024
256e242
Provision VMs with dependencies for the testing scripts
AlexRuiz7 Oct 29, 2024
03a6f53
Merge scripts 00 and 01 making it easier to get the package from GHA …
QU3B1M Oct 30, 2024
474e5dd
Optimize test scripts
AlexRuiz7 Oct 30, 2024
d176449
Add sleep after clister initialization
AlexRuiz7 Oct 31, 2024
23e86c1
Update README and improve scripts output logs
QU3B1M Oct 31, 2024
41914d5
Update execution guide on README
QU3B1M Oct 31, 2024
136242f
Add conditional to remove certs directory if already exists
QU3B1M Nov 1, 2024
1ceef3c
Add sleep to avoid requesting to the API before cluster is initialized
QU3B1M Nov 1, 2024
96fdef3
Add index force merge for the command_manager plugin index
QU3B1M Nov 1, 2024
4c4518e
Merge branch 'master' into enhancement/478-validation-tests
AlexRuiz7 Nov 5, 2024
3684bae
Avoid errors due to race conditions
AlexRuiz7 Nov 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion test-tools/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Basic cluster environment

This is a environment definition with the required configuration to be prepared to freshly install a Wazuh Indexer
This is an environment definition with the required configuration to be prepared to freshly install a Wazuh Indexer
cluster with two nodes using Vagrant and Libvirt to provision the Virtual Machines.

It also generates the node's required certificates using the `wazuh-certs-tool` and copy them to each node's `home`
Expand Down
22 changes: 15 additions & 7 deletions test-tools/Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ system("

Vagrant.configure("2") do |config|
config.vm.define "indexer_1" do |indexer_1|
indexer_1.vm.box = "generic/rhel9"
indexer_1.vm.box = "generic/alma9"
indexer_1.vm.synced_folder ".", "/vagrant"
indexer_1.vm.network "private_network", ip: "192.168.56.10"
indexer_1.vm.hostname = "node-1"
Expand All @@ -16,13 +16,17 @@ Vagrant.configure("2") do |config|
vb.cpus = "4"
end
indexer_1.vm.provision "shell", inline: <<-SHELL
sudo systemctl stop firewalld
sudo systemctl disable firewalld
sudo yum clean all
systemctl stop firewalld
systemctl disable firewalld
yum clean all
yum install curl jq unzip tar -y
# Add node-2 to /etc/hosts
sudo echo "192.168.56.11 node-2" >> /etc/hosts
echo "192.168.56.11 node-2" >> /etc/hosts
# Copy generated certificates
cp /vagrant/wazuh-certificates.tar /home/vagrant/wazuh-certificates.tar
# Copy test scripts
cp -r /vagrant/scripts /home/vagrant/scripts
chown -R vagrant:vagrant /home/vagrant/scripts
SHELL
end
config.vm.define "indexer_2" do |indexer_2|
Expand All @@ -35,12 +39,16 @@ Vagrant.configure("2") do |config|
vb.cpus = "4"
end
indexer_2.vm.provision "shell", inline: <<-SHELL
sudo systemctl stop ufw
sudo systemctl disable ufw
systemctl stop ufw
systemctl disable ufw
apt-get install curl jq unzip tar -y
# Add node-1 to /etc/hosts
echo "192.168.56.10 node-1" >> /etc/hosts
# Copy generated certificates
cp /vagrant/wazuh-certificates.tar /home/vagrant/wazuh-certificates.tar
# Copy test scripts
cp -r /vagrant/scripts /home/vagrant/scripts
chown -R vagrant:vagrant /home/vagrant/scripts
SHELL
end
end
88 changes: 88 additions & 0 deletions test-tools/scripts/00_run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/bin/bash

# Prompt the user for GitHub Token and artifact details securely
if [ -z "$GITHUB_TOKEN" ]; then
read -rsp 'Enter GitHub Token: ' GITHUB_TOKEN
echo ""
fi
export GITHUB_TOKEN

if [ -z "$RUN_ID" ]; then
read -rp 'Enter Action Run ID: ' RUN_ID
fi
export RUN_ID

if [ -z "$ARTIFACT_NAME" ]; then
read -rp 'Enter Artifact Name: ' ARTIFACT_NAME
fi
export ARTIFACT_NAME

# Define environment variables with default values if not provided
read -rp "Enter current node name (default: 'node-1'): " NODE_NAME
export NODE_NAME=${NODE_NAME:-"node-1"}

IP_ADDRESS=$(ip addr show eth1 2>/dev/null | grep 'inet ' | awk '{print $2}' | cut -d/ -f1)
if [ -z "$IP_ADDRESS" ]; then
IP_ADDRESS="127.0.0.1"
fi
read -rp "Enter IP of current node (default: '$IP_ADDRESS'): " NODE_IP
export NODE_IP=${NODE_IP:-$IP_ADDRESS}

export CERTS_PATH=${CERTS_PATH:-"/home/vagrant/wazuh-certificates.tar"}

# Optional variables for Node 2
read -rp 'Enter secondary Node name (optional): ' NODE_2
read -rp 'Enter IP of secondary Node (optional): ' IP_NODE_2

# Logging function with timestamps
log() {
echo "$(date +'%Y-%m-%d %H:%M:%S') - $1"
}

# Function to run a command and check for errors
run_command() {
local cmd=$1
log "Executing: $cmd"
if ! eval "$cmd"; then
log "Error executing: $cmd"
exit 1
else
log "Successfully executed: $cmd"
fi
}

# Main execution
log "Starting the script execution"

run_command "bash 01_download_and_install_package.sh -id $RUN_ID -n $ARTIFACT_NAME"

# Apply certificates
if [ -n "$NODE_2" ] && [ -n "$IP_NODE_2" ]; then
run_command "sudo bash 02_apply_certificates.sh -p $CERTS_PATH -n $NODE_NAME -nip $NODE_IP -s $NODE_2 -sip $IP_NODE_2"
else
run_command "sudo bash 02_apply_certificates.sh -p $CERTS_PATH -n $NODE_NAME -nip $NODE_IP"
fi

# Start indexer service
run_command "sudo bash 03_manage_indexer_service.sh -a start"

# Initialize cluster (assumes this step doesn't depend on Node 2 presence)
run_command "sudo bash 04_initialize_cluster.sh"
sleep 10

# Validate installed plugins
if [ -n "$NODE_2" ]; then
run_command "bash 05_validate_installed_plugins.sh -n $NODE_NAME -n $NODE_2"
else
run_command "bash 05_validate_installed_plugins.sh -n $NODE_NAME"
fi

# Validate setup and command manager
run_command "bash 06_validate_setup.sh"
run_command "bash 07_validate_command_manager.sh"

# Uninstall indexer
log "Running 08_uninstall_indexer.sh"
run_command "sudo bash 08_uninstall_indexer.sh"

log "All tasks completed successfully."
173 changes: 173 additions & 0 deletions test-tools/scripts/01_download_and_install_package.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
#!/bin/bash

# SPDX-License-Identifier: Apache-2.0
# The OpenSearch Contributors require contributions made to
# this file be licensed under the Apache-2.0 license or a
# compatible open source license.

# Tool dependencies
DEPENDENCIES=(curl jq unzip)
# Default package revision
PKG_REVISION="0"
# Wazuh indexer repository
REPO="wazuh/wazuh-indexer"

# Function to display usage help
usage() {
echo "Usage: $0 --run-id <RUN_ID> [-v <PKG_VERSION>] [-r <PKG_REVISION>] [-n <PKG_NAME>]"
echo
echo "Parameters:"
echo " -id, --run-id The GHA workflow execution ID."
echo " -v, --version (Optional) The version of the wazuh-indexer package."
echo " -r, --revision (Optional) The revision of the package. Defaults to '0' if not provided."
echo " -n, --name (Optional) The package name. If not provided, it will be configured based on version and revision."
echo
echo "Please ensure you have the GITHUB_TOKEN environment variable set to access the GitHub repository, and all the dependencies installed: " "${DEPENDENCIES[@]}"
exit 1
}

# Parse named parameters
while [[ "$#" -gt 0 ]]; do
case $1 in
--artifact-id|-id) RUN_ID="$2"; shift ;;
--version|-v) PKG_VERSION="$2"; shift ;;
--revision|-r) PKG_REVISION="$2"; shift ;;
--name|-n) PKG_NAME="$2"; shift ;;
-h|--help) usage ;;
*) echo "Unknown parameter passed: $1"; usage ;;
esac
shift
done

# Validate all dependencies are installed
for dep in "${DEPENDENCIES[@]}"
do
if ! command -v "${dep}" &> /dev/null
then
echo "Error: Dependency '$dep' is not installed. Please install $dep and try again." >&2
exit 1
fi
done

# Check if RUN_ID is provided
if [ -z "$RUN_ID" ]; then
echo "Error: RUN_ID is required."
usage
fi

# Validate GITHUB_TOKEN environment variable
if [ -z "$GITHUB_TOKEN" ]; then
echo "Please ensure you have the GITHUB_TOKEN environment variable set to access the GitHub repository."
exit 1
fi

# Ensure either PKG_NAME or both PKG_VERSION and PKG_REVISION are provided
if [ -z "$PKG_NAME" ] && { [ -z "$PKG_VERSION" ] || [ -z "$PKG_REVISION" ]; }; then
echo "Error: Either a package name (--name) or both a version (--version) and revision (--revision) must be provided."
usage
fi

# Detect OS and architecture
if [ -f /etc/os-release ]; then
. /etc/os-release
OS=$(echo "$NAME" | tr '[:upper:]' '[:lower:]')
else
echo "Unsupported OS."
exit 1
fi

# Determine package type if PKG_NAME is not provided
ARCH=$(uname -m)
case "$OS" in
*ubuntu* | *debian*)
PKG_FORMAT="deb"
if [ -z "$PKG_NAME" ]; then
[ "$ARCH" == "x86_64" ] && ARCH="amd64"
[ "$ARCH" == "aarch64" ] && ARCH="arm64"
PKG_NAME="wazuh-indexer_${PKG_VERSION}-${PKG_REVISION}_${ARCH}.${PKG_FORMAT}"
fi
;;
*centos* | *fedora* | *rhel* | *"red hat"* | *alma*)
PKG_FORMAT="rpm"
if [ -z "$PKG_NAME" ]; then
PKG_NAME="wazuh-indexer-${PKG_VERSION}-${PKG_REVISION}.${ARCH}.${PKG_FORMAT}"
fi
;;
*)
echo "Unsupported OS."
exit 1
;;
esac

# Check if the package is already present
if [ -f "$PKG_NAME" ]; then
echo "Package $PKG_NAME found locally. Reusing existing package."
else
# Fetch the list of artifacts
echo "Fetching artifacts list..."
RUN_URL="https://api.github.com/repos/${REPO}/actions/artifacts"
RESPONSE=$(curl -s -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $GITHUB_TOKEN" -H "X-GitHub-Api-Version: 2022-11-28" "$RUN_URL?name=$PKG_NAME")

# Check if the curl command was successful
if [ $? -ne 0 ]; then
echo "Error: Failed to fetch artifacts."
exit 1
fi

# Check if the artifact from the specified workflow run ID exists
echo "Checking ${PKG_NAME} package is generated for workflow run ${RUN_ID}"
ARTIFACT=$(echo "$RESPONSE" | jq -e ".artifacts[] | select(.workflow_run.id == $RUN_ID)")

if [ -z "$ARTIFACT" ]; then
echo "Error: Wazuh indexer package not found."
exit 1
fi

ARTIFACT_ID=$(echo "$ARTIFACT" | jq -r '.id')
echo "Wazuh indexer artifact detected. Artifact ID: $ARTIFACT_ID"

# Download the package
ARTIFACT_URL="https://api.github.com/repos/${REPO}/actions/artifacts/${ARTIFACT_ID}/zip"
echo "Downloading wazuh-indexer package from GitHub artifactory..."
echo "(It could take a couple of minutes)"

if ! curl -L -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"$ARTIFACT_URL" -o package.zip > /dev/null 2>&1; then
echo "Error downloading package."
exit 1
fi
echo "Package downloaded successfully"

# Unzip the package
echo "Decompressing wazuh-indexer package..."
unzip ./package.zip
rm package.zip

# shellcheck disable=SC2181
if [ $? -ne 0 ]; then
echo "Error unzipping package."
exit 1
fi
echo "Package decompressed"
fi

# Install the package
echo "Installing wazuh-indexer package..."
case "$PKG_FORMAT" in
"deb")
sudo dpkg -i "$PKG_NAME" > /dev/null 2>&1
;;
"rpm")
sudo rpm -i "$PKG_NAME" > /dev/null 2>&1
;;
esac

# shellcheck disable=SC2181
if [ $? -ne 0 ]; then
echo "Error installing package."
exit 1
fi

echo "Package installed successfully."
Loading