Skip to content

Latest commit

 

History

History
159 lines (127 loc) · 4.73 KB

3_0_4.rst

File metadata and controls

159 lines (127 loc) · 4.73 KB
Raw

3.0.4

v3.0.4 - May 11, 2022

This is a minor release with changes and enhancements relating to Skyline cluster. If you are not running a Skyline cluster then you can skip v3.0.4 and stay on v3.0.3

Changes

  • http_alerter - discard after x resend attempts (4566)
  • flux worker - Only rename if exists - flux.workers.metrics_sent to aet.flux.workers.metrics_sent (3824)
  • Correct cluster call api parameters append _timestamp (4530)
  • get_cluster_data - cluster changes (3824)
  • ionosphere_performance - convert rows with nans to 0, drop rows with date 0 (3934)
  • metrics_manager - Create a single hash that contains all the flux.quota.namespace_metrics namespaces to allow for the cluster sync to make a single call rather than a call per namespace (4464)
  • metrics_manager - Reduce cluster logging (4568)
  • Updated version and release docs (4562)

Upgrade notes

  • These update instructions apply to upgrading from v3.0.0 or v3.0.3
  • There are no changes to the DB in v3.0.3
  • There are no changes to settings.py in v3.0.3

How to update from v3.0.0/v3.0.3

  • Download the new release tag or clone/update to get it to a temp location, ready to be deployed.
  • Copy your settings.py from v3.0.0/v3.0.3 and replace the v3.0.3 settings.py
  • Stop the Skyline apps, backup the database and Redis, move to the new version, start the Skyline apps.
# Get the new version
NEW_SKYLINE_VERSION="v3.0.4"    # Your new Skyline version
OLD_SKYLINE_VERSION="v3.0.3"    # Your old Skyline version
# OLD_SKYLINE_VERSION="v3.0.0"    # Your old Skyline version

CURRENT_SKYLINE_PATH="/opt/skyline/github/skyline"                 # Your Skyline path
NEW_SKYLINE_PATH="${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"  # Your new Skyline path

mkdir -p "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
cd "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
git clone https://github.com/earthgecko/skyline .
git checkout "$NEW_SKYLINE_VERSION"

# settings.py
cp "$NEW_SKYLINE_PATH/skyline/settings.py" "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak"

# Create a new settings.py file in the new version based on your existing
# settings.py file
cat "${CURRENT_SKYLINE_PATH}/skyline/settings.py" > "$NEW_SKYLINE_PATH/skyline/settings.py"

# **TEST**
# Test your new settings.py BEFORE continuing with the upgrade,
# test_settings.sh runs the validate_settings.py that the Skyline apps run
# when they start
$NEW_SKYLINE_PATH/bin/test_settings.sh

# Stop/disable any/all service controls like monit, etc that are controlling
# Skyline services.

# NOTE ALL SKYLINE SERVICES ARE LISTED HERE, REMOVE TO ONES YOU DO NOT RUN
# or do not wish to run.
SKYLINE_SERVICES="analyzer
analyzer_batch
mirage
crucible
boundary
ionosphere
luminosity
panorama
webapp
vista
snab
flux
horizon"
for i in $SKYLINE_SERVICES
do
  systemctl stop "$i"
done
  • Move your current Skyline directory to a backup directory and move the new Skyline v3.0.4 with your new settings.py from the temp location to your working Skyline directory, (change your paths as appropriate) e.g.
mv "$CURRENT_SKYLINE_PATH" "${CURRENT_SKYLINE_PATH}.${OLD_SKYLINE_VERSION}"
mv "$NEW_SKYLINE_PATH" "$CURRENT_SKYLINE_PATH"

# Set permission on the dump dir
chown skyline:skyline "$CURRENT_SKYLINE_PATH"/skyline/webapp/static/dump
  • Start the all Skyline services (change as appropriate for your set up) e.g.
# NOTE ALL SKYLINE SERVICES ARE LISTED HERE, REMOVE TO ONES YOU DO NOT RUN
# apart from the new thunder Skyline app

# Start all other Skyline services
SKYLINE_SERVICES="horizon
flux
panorama
webapp
vista
analyzer
analyzer_batch
mirage
crucible
boundary
ionosphere
luminosity
snab
thunder"
for i in $SKYLINE_SERVICES
do
  systemctl start "$i"
  if [ $? -ne 0 ]; then
    echo "failed to start $i"
  else
    echo "started $i"
  fi
done
# Restart any/all service controls like monit, etc that are controlling
# Skyline services.
  • Check the logs
# How are they running
tail -n 20 /var/log/skyline/*.log

# Any errors - each app
find /var/log/skyline -type f -name "*.log" | while read skyline_logfile
do
  echo "#####
# Checking for errors in $skyline_logfile"
  cat "$skyline_logfile" | grep -B2 -A10 -i "error ::\|traceback" | tail -n 60
  echo ""
  echo ""
done

Congratulations, you are now running the best open source anomaly detection cluster stack in the world (probably).