4_0_0.rst

=====
4.0.0
=====

v4.0.0 - xxx xx, 2023

This is a **MAJOR** release with changes and enhancements across the board,
including changes and additions to backend components.

Notable new features and functionality
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are a plethera of changes and additions in v4.0.0

Added support for Prometheus/VictoriaMetrics - ingesting, storing and analysing
labelled metrics or tagged metrics from Prometheus, VictoriaMetrics and via
plain HTTP JSON.

This version also starts making use of the very, very long awaited
RedisTimeseries for storing labelled metric timeseries data.

Mirage has been significantly improved in its processing methodology to process
many metrics per run rather than a single metric per run.  This reduces the
occurrences of WATERFALL ALERTS to practically zero.  Mirage has also been
enhanced with additional algorithms to identify normal daily peaks and to
identify irregular, unstable timeseries and run an additional analysis on
these types of metrics at a much longer time period due to the sparsity of
normal events in a weekly period.  These two additional analysis methodologies
vastly decrease false positives on metrics which display these behaviours. 

There is a plethera of additions to custom algorithms and the addition of a new
adhoc analysis service called Skyline/Vortex which does adhoc analysis on any
24hr or 7day (overrrideable) uploaded timeseries or timeseries in the system
and uses any combination/s of custom algorithms that the user chooses to analyse
the timeseries and return results.  Vortex can be used directly via a HTTP API
and there is also a webapp UI for it as well.

There is also the addition of numerous mainstream custom algorithms to choose
from that can now also be used in any part of the analysis pipeline or at least
be used with Mirage and Vortex (for which these are intended to be used with,
they are not intended to be used as custom_algorithms in Analyzer):

- sigma
- matrixprofile
- spectral_residual
- lof
- isolation_forest
- dbscan
- pca
- prophet
- one_class_svm
- anomalous_daily_peak
- irregular_unstable
- mstl
- autoarima

Learning repetitive patterns has been added.

An adhoc analysis service has been added called Vortex.

There is a new internal monitoring service named Thunder that handles and routes
internal alerts relating to metric namespaces and monitors the internal
availability and functioning of the Skyline apps.

There has been a lot of computationally performance improvements by refactoring
all/most viable methods in algorithms and data processing into numba functions
where appropriate.

The webapp has a number of new pages and features.

There are significant changes to how both webapp and flux are run by gunicorn,
both apps have changed to using gevent for the unicorn ``worker_class``.

There are also a number of new dependencies and dependency updates as well.

Noteable Backend changes
========================

Flux and Webapp changed to gevent
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are significant changes to how both webapp and flux are run by gunicorn,
both apps have changed to using gevent for the unicorn ``worker_class``.  This
is a significant operational change.  Further there is also **one** remaining
webapp process that needs to be started as the default sync ``worker_class`` due
to not being able to get flask, gunicorn, gevent and the use of multiprocessing
in tsfresh webapp functions to work as desired.  Although there are ways to run
gevent and multiprocessing together, none of the recommended methods or other
attempted methods worked with flask, gunicorn, gevent and tsfresh.  The simplest
solution therefore is to just have an additional gunicorn process start using
sync to serve the few tsfresh related webapp requests, which have always worked
and with this method still do.  It is a tradeoff incurring a new process as
opposed to determining a convoluted method for flask, gunicorn, gevent and
tsfresh to work together and maintaining that.

numba optimisations
~~~~~~~~~~~~~~~~~~~

The starting of apps is a little slower in some cases and the first analysis run
on some apps can take longer than normal.  This is because gaining performance
on every analysis thereafter has the tradeoff in that all the numba algorithms
and numba data processing functions must be compiled on their initialisation.
Thereafter the performance improvements are gained on every function call.

RedisTimeseries
~~~~~~~~~~~~~~~

There are 2 available of methods to deploy and use RedisTimeseries, one being use
the normal OSS Redis and compile RedisTimeseries and load the module via the include
or use redis-stack-server.  Upgrading Redis or changing to redis-stack-server is
beyond the scope of this document as it will be vary OS/version (and there are lots
of options).  There is a guide at https://earthgecko-skyline.readthedocs.io/en/latest/upgrading/redis-to-redis-stack-server.html
that explains the upgrade process, but if you run your own Redis build/config/pattern
then you may have your own update/upgrade strategy.

That said, if your want to run Skyline with labelled metrics you will need a version
of Redis that has the RedisTimeseries module.  Consider not just using the redis.io
packages repos for yum/apt as they provide redis-stack-server RC (release candidate)
packages at times and in terms of Skyline Redis needs to be stable, using the
download binaries for your OS is a more stable method.

matrixprofile replaced by stumpy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The matrix-profile-foundation/matrixprofile library is no longer maintained and
this has been replaced with the stumpy matrixprofile implementation.  This
affects the matrixprofile custom_algorithm which will automatically use the
stumpy version if available.  matrixprofile has been removed as a dependency.

Prometheus/VictoriaMetrics metrics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Accept labelled metrics data from Prometheus, InfluxDB (telegraf) and VictoriaMetrics.
- Use VictoriaMetrics as Skyline's longer term TSDB for labelled metrics.

Changes
~~~~~~~

The changes list is LONG, so listed at the end of this page.

Upgrade notes
~~~~~~~~~~~~~

- These update instructions apply to upgrading from v3.0.4
- There are are changes to the DB in v4.0.0
- There are changes to settings.py in v4.0.0
- There are changes to dependencies in v4.0.0
- There is a new service called thunder in v4.0.0

How to update from v3.0.4
~~~~~~~~~~~~~~~~~~~~~~~~~

- Build a new Python version and virtualenv, due to this being a major version
  release we use a new virtualenv which all the required libraries and do not
  affect your running install so you can rollback if necessarily.
- Download the new release tag or clone/update to get it to a temp location,
  ready to be deployed.
- Copy your settings.py from v3.0.4 and update/replace the v3.0.4 settings.py
  with your new v4.0.0 settings.py file
- Stop the Skyline apps
- Backup the database and Redis
- OPTIONALLY upgrade MariaDB and/or Redis (redis-stack-server)
- Update the Python dependencies (requirements.txt)
- Move to the new version,
- Start the Skyline apps.

Ensure you read through all the steps below first and assess the steps you wish
to do and if any paths, etc need to be changed for your set up.  Also you need
to make a decision and assess whether you want to upgrade to use redis-stack-server.

Here are the steps....

- Get the new version.

.. code-block:: bash

    # Get the new version
    NEW_SKYLINE_VERSION="v4.0.0"    # Your new Skyline version
    OLD_SKYLINE_VERSION="v3.0.4"    # Your old Skyline version

    CURRENT_SKYLINE_PATH="/opt/skyline/github/skyline"                 # Your Skyline path
    NEW_SKYLINE_PATH="${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"  # Your new Skyline path

    mkdir -p "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
    cd "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
    git clone https://github.com/earthgecko/skyline .
    git checkout "$NEW_SKYLINE_VERSION"

    # settings.py
    cp "$NEW_SKYLINE_PATH/skyline/settings.py" "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak"

    # Create a new settings.py file in the new version based on your existing
    # settings.py file
    cat "${CURRENT_SKYLINE_PATH}/skyline/settings.py" > "$NEW_SKYLINE_PATH/skyline/settings.py"

    # You can diff the new settings with your settings that you just copied
    diff "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak" "$NEW_SKYLINE_PATH/skyline/settings.py"

- You are probably going to want to diff those settings in a visual diff like Meld.
- Deploy the new Python version and a new virtualenv and install the dependencies.

.. code-block:: bash

  # DEPLOY A NEW Python 3.8.17 virtualenv
  PYTHON_VERSION="3.8.17"
  PYTHON_MAJOR_VERSION="3.8"
  PYTHON_VIRTUALENV_DIR="/opt/python_virtualenv"
  mkdir -p "${PYTHON_VIRTUALENV_DIR}/versions/${PYTHON_VERSION}"
  mkdir -p "${PYTHON_VIRTUALENV_DIR}/projects"
  cd "${PYTHON_VIRTUALENV_DIR}/versions/${PYTHON_VERSION}"
  wget -q "https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz"
  tar -zxvf "Python-${PYTHON_VERSION}.tgz"
  cd ${PYTHON_VIRTUALENV_DIR}/versions/${PYTHON_VERSION}/Python-${PYTHON_VERSION}
  ./configure --prefix=${PYTHON_VIRTUALENV_DIR}/versions/${PYTHON_VERSION}
  make -j4
  make altinstall

  # Create a new skyline-py3817 virtualenv
  PROJECT="skyline-py3817"
  cd "${PYTHON_VIRTUALENV_DIR}/projects"
  virtualenv --python="${PYTHON_VIRTUALENV_DIR}/versions/${PYTHON_VERSION}/bin/python${PYTHON_MAJOR_VERSION}" "$PROJECT"
  ln -sf "${PYTHON_VIRTUALENV_DIR}/projects/skyline" "${PYTHON_VIRTUALENV_DIR}/projects/${PROJECT}"

  # Ensure the requirements are installed in your new Python 3.8.17 virtualenv
  cd "${PYTHON_VIRTUALENV_DIR}/projects/${PROJECT}"
  source bin/activate

  # Deploy the new requirements
  "bin/pip${PYTHON_MAJOR_VERSION}" install -r "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"/requirements.txt

  deactivate
  cd

  # Fix python-daemon
  cp "/opt/python_virtualenv/projects/${PROJECT}/lib/python${PYTHON_MAJOR_VERSION}/site-packages/daemon/runner.py" "/opt/python_virtualenv/projects/${PROJECT}/lib/python${PYTHON_MAJOR_VERSION}/site-packages/daemon/runner.py.original.bak"
  cat "$NEW_SKYLINE_PATH/utils/python-daemon/runner.2.3.0.py" > "/opt/python_virtualenv/projects/${PROJECT}/lib/python${PYTHON_MAJOR_VERSION}/site-packages/daemon/runner.py"
  # minor change related to unbuffered bytes I/O
  diff "/opt/python_virtualenv/projects/${PROJECT}/lib/python${PYTHON_MAJOR_VERSION}/site-packages/daemon/runner.py.original.bak" "/opt/python_virtualenv/projects/${PROJECT}/lib/python${PYTHON_MAJOR_VERSION}/site-packages/daemon/runner.py"

- Ensure you have added all the new (and changed) settings to `$NEW_SKYLINE_PATH/skyline/settings.py` and test them

.. code-block:: bash

  # Get the new version
  NEW_SKYLINE_VERSION="v4.0.0"    # Your new Skyline version
  CURRENT_SKYLINE_PATH="/opt/skyline/github/skyline"                 # Your Skyline path
  NEW_SKYLINE_PATH="${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"  # Your new Skyline path

  # You can diff the new settings with your settings that you just copied
  diff "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak" "$NEW_SKYLINE_PATH/skyline/settings.py"

  # UPDATE YOUR $NEW_SKYLINE_PATH/skyline/settings.py as appropriate

  # **TEST**
  # Test your new settings.py BEFORE continuing with the upgrade,
  # test_settings.sh runs the validate_settings.py that the Skyline apps run
  # when they start
  $NEW_SKYLINE_PATH/bin/test_settings.sh

- At this point we are going to stop Skyline services, update the DB and you
  can also upgrade MariaDB and/or Redis at this point if you choose to.

- Stop/disable any/all service controls like monit, etc that are controlling
  Skyline services.

.. code-block:: bash

  # Stop/disable any/all service controls like monit, etc that are controlling
  # Skyline services.

  # NOTE ALL SKYLINE SERVICES ARE LISTED HERE, REMOVE TO ONES YOU DO NOT RUN
  # or do not wish to run.
  SKYLINE_SERVICES="analyzer
  analyzer_batch
  mirage
  crucible
  boundary
  ionosphere
  luminosity
  panorama
  webapp
  vista
  snab
  flux
  horizon"
  for i in $SKYLINE_SERVICES
  do
    systemctl stop "$i"
  done

- Remove the numba cache directory.  This needs to be done to ensure that new
  jit cache files are compiled for numba jit functions with the updated Python
  version and required dependencies like numpy (even though they do not exist
  yet, this is for next time so it does not get forgetten).

.. code-block:: bash

  rm -rf /opt/skyline/.cache/numba/

- Backup the DB and apply the update

.. code-block:: bash

    # BACKUP THE DB AND APPLY THE NEW SQL
    BACKUP_DIR="/tmp"  # Where you want to backup the DB to
    MYSQL_USER="skyline"
    MYSQL_HOST="127.0.0.1"  # Your MySQL IP
    MYSQL_DB="skyline"  # Your MySQL Skyline DB name

    # Backup DB
    mkdir -p $BACKUP_DIR
    mysqldump -u$MYSQL_USER -p $MYSQL_DB > $BACKUP_DIR/pre.$NEW_SKYLINE_VERSION.$MYSQL_DB.sql

    # Check you dump exists and has data
    ls -al $BACKUP_DIR/pre.$NEW_SKYLINE_VERSION.$MYSQL_DB.sql

    # Update DB
    mysql -u$MYSQL_USER -p $MYSQL_DB < "${NEW_SKYLINE_PATH}/updates/sql/v4.0.0.sql"
    echo $?

- If you are wanting to upgrade MariaDB now is the time to do that.
- If you want to upgrade Redis or deploy redis-stack-server, now is the time to that.
- Move your current Skyline directory to a backup directory and move the new
  Skyline v4.0.0 with your new settings.py from the temp location to your
  working Skyline directory, (change your paths as appropriate) e.g.

.. code-block:: bash

  mv "$CURRENT_SKYLINE_PATH" "${CURRENT_SKYLINE_PATH}.${OLD_SKYLINE_VERSION}"
  mv "$NEW_SKYLINE_PATH" "$CURRENT_SKYLINE_PATH"

  # Set permission on the dump dir
  chown skyline:skyline "$CURRENT_SKYLINE_PATH"/skyline/webapp/static/dump

- Replace your /etc/skyline/skyline.conf to reflect the new Python version and
  PYTHON_VIRTUALENV_DIR

.. code-block:: bash

  mv /etc/skyline/skyline.conf /etc/skyline/skyline.conf.pre.v4.0.0.bak
  cp "$CURRENT_SKYLINE_PATH"/etc/skyline.conf /etc/skyline/
  diff /etc/skyline/skyline.conf.pre.v4.0.0.bak /etc/skyline/skyline.conf

- Backup and replace your systemd unit files to to reflect the new Python
  version and PYTHON_VIRTUALENV_DIR, **unless you control systemd units via
  configuration management or other**, then assess how to update your systemd
  unit files.  Note there is a new thunder service.

.. code-block:: bash

  mkdir /opt/skyline/github/skyline/etc/systemd/system.pre.v4.0.0.bak
  for i in $(find /opt/skyline/github/skyline/etc/systemd/system -type f)
  do
    unitfile=$(basename "$i")
    cp "/etc/systemd/system/${unitfile}" /opt/skyline/github/skyline/etc/systemd/system.pre.v4.0.0.bak/
    /bin/cp -f $i /etc/systemd/system/
  done
  systemctl daemon-reload


- Start the all Skyline services (change as appropriate for your set up) e.g.

.. code-block:: bash

    # NOTE ALL SKYLINE SERVICES ARE LISTED HERE, REMOVE TO ONES YOU DO NOT RUN
    # apart from the new thunder Skyline app

    # Start all other Skyline services
    SKYLINE_SERVICES="horizon
    flux
    panorama
    webapp
    vista
    analyzer
    analyzer_batch
    mirage
    crucible
    boundary
    ionosphere
    luminosity
    snab
    thunder"
    for i in $SKYLINE_SERVICES
    do
      systemctl start "$i"
      if [ $? -ne 0 ]; then
        echo "failed to start $i"
      else
        echo "started $i"
      fi
      systemctl enable "$i"
    done
    # Restart any/all service controls like monit, etc that are controlling
    # Skyline services.

- Check the logs.  You may see a few errors related to some missing Redis keys/hashes
  this is expected as they do not exist on starting the new version.

.. code-block:: bash

    # How are they running
    tail -n 20 /var/log/skyline/*.log

    # Any errors - each app
    find /var/log/skyline -type f -name "*.log" | while read skyline_logfile
    do
      echo "#####
    # Checking for errors in $skyline_logfile"
      cat "$skyline_logfile" | grep -B2 -A10 -i "error ::\|traceback" | tail -n 60
      echo ""
      echo ""
    done

Congratulations, you are now running the best OSS anomaly detection stack in the
world (probably).

Changes
-------

New slack messaging does not handle public channel (111)
thunder (1444)
panorama check file fails (1460)
Memory leak in Analyzer (1558)
Patterning Skyline Ionosphere - updated to tsfresh-0.3.0 (1658)
Update deps in Ionosphere (1758)
Update to tsfresh-0.3.0 (1778)
Shapelet extraction (1792)
tsfresh extract_features process stalling (1822)
ionosphere - mirage Redis data features (1828)
Ionosphere alerts (1830)
Ionosphere - Graphite now graphs (1842)
Ionosphere - features_profile matched graphite graphs (1852)
Ionosphere learn (1886)
Ionosphere learn - child like parent with evolutionary maturity (1886)
Handle non filesystem friendly metric names in check files (1904)
ionosphere_layers (1960)
Ionosphere - matches page (1996)
analyse_derivatives (2034)
analyse_derivatives - change in monotonicity (2050)
MySQL - Aborted_clients (2130)
Optimise Ionosphere DB usage (2132)
Update deps (2138)
Test skyline with bandit (2160)
luminosity (2270)
update deps for luminosity branch (2272)
pandas-0.22.0 and numpy (2274)
CORRELATE_ALERTS_ONLY (2360)
Add redis auth to Skyline and rebrow (2378)
Ionosphere validate learnt features profiles page (2430)
Ionosphere validate learnt features profiles page --> (2430)
luminosity_remote_data (2464)
FULL_DURATION feature profiles (2484)
illuminance (2580)
Update dependencies (2588)
mysql-connector-python - use_pure (2590)
Build Skyline on nodes at v1.2.8 (2596)
statsmodels-0.9.0 - cannot import name _representation (2600)
alert_slack (2618)
slack (2646)
CVE-2018-18074 - Skyline requests (2652)
Reduce mirage ionosphere alert loop (2682)
python-daemon does not support Python 3 buffer method (2692)
Failing to load metric vars (2708)
Prometheus to Skyline (2732)
Mutliple SQL Injection Security Vulnerabilities (2818)
Test redis-py upgrade (2824)
Skyline - Python 3.7.2 (2828)
Mirage - periodic_check (2882)
Initial Ionosphere echo load and Ionosphere feedback (2904)
ANALYZER_ENABLED setting (2916)
Update dependencies (2926)
Update dependencies (2964)
New slack messaging does not handle public channel (2986)
docker (3002)
Reduce multiprocessing Manager list usage (3034)
Reduce multiprocessing Manager list usage, this was (3034)
Update dependencies (3060)
SNAB (3068)
Add CUSTOM_ALERT_OPTS to settings (3194)
Allow Skyline to send metrics to another Carbon host (3250)
webapp api - unique_metrics (3252)
Review and test PR 124 (3256)
py3 (3262)
py3 (3263)
py3 Redis binary objects not strings (3266)
Deprecate string.replace for py3 (3270)
Always send anomaly_breakdown and exception metrics (3288)
Handle urllib2 in py3 (3290)
Record anomaly_end_timestamp (3306)
webapp api - derivative_metrics (3336)
http_alerter (3396)
Panorama - add metric to metrics table immediately (3442)
Allow flux to backfill (3444)
Add IONOSPHERE_MANAGE_PURGE (3462)
ionosphere.training_data Redis set (3472)
webapp api - training_data (3474)
batch_processing (3480)
analyzer_batch (3486)
webapp - crucible_process_metrics (3500)
ionosphere.untrainable_metrics (3508)
matched_or_regexed_in_list function (3512)
Sort all time series (3532)
webapp - upload_data endpoint (3538)
flux.uploaded_data_worker (3550)
Update deps (3556)
External alert config (3560)
custom_algorithms (3566)
Apply list to settings import (3572)
inference (3590)
Add timestamp to ENABLE_BOUNDARY_DEBUG output (3594)
Update Skyline to Python 3.8.3 and deps (3608)
Upgrade to slack v2 (3612)
Anomaly type classification (3642)
ROOMBA_DO_NOT_PROCESS_BATCH_METRICS (3650)
Handle multiple metrics in base_name conversion (3652)
IONOSPHERE_GRAPHITE_NOW_GRAPHS_OVERRIDE (3654)
Change mirage.last_check keys to timestamp value (3662)
flux - POST multiple metrics (3694)
FLUX_ZERO_FILL_NAMESPACES (3708)
Update sqlalchemy - SNYK-PYTHON-SQLALCHEMY-590109 (3720)
Validate Mirage running multiple processes (3730)
waterfall alerts (3734)
POC matrixprofile (3744)
POC SNAB (3748)
Add the anomaly_id to the http_alerter json (3772)
waterfall_alert - no analyzer triggered_algorithms in waterfall_panorama_data on MIRAGE_ALWAYS_METRICS (3776)
Handle single encoded forward slash requests to Graphite (3778)
skyline_functions - sanitise_graphite_url (3780)
snab_flux_load_test (3788)
flux - pickle to Graphite (3790)
Handle feedback metrics in Mirage and waterfall alerts (3800)
ANALYZER_DYNAMICALLY_ANALYZE_LOW_PRIORITY_METRICS (3808)
HORIZON_SHARDS (3820)
get_cluster_data (3824)
Add inactive columns to the metrics DB table (3828)
metrics_manager (3830)
custom_algorithms - run_before_3sigma parameter (3848)
webapp - yhat_values API endoint (3850)
Handle boring sparsely populated metrics in derivative_metrics (3856)
flux - try except everything (3864)
MIRAGE_ENABLE_HIGH_RESOLUTION_ANALYSIS (3866)
POC MIRAGE_ENABLE_HIGH_RESOLUTION_ANALYSIS (3868)
metrics_manager - check_data_sparsity (3870)
SNYK-PYTHON-PY-1049546 (3874)
ANALYZER_CHECK_LAST_TIMESTAMP (3884)
Test Python 3.8.6 and deps (3898)
Test Skyline with dependencies updates - tsfresh v0.17.9 (3926)
Update Skyline to use new tsfresh feature extraction method (3928)
ionosphere_performance (3934)
SNYK-PYTHON-PILLOW-1055461 and SNYK-PYTHON-PILLOW-1055462 (3940)
xlrd-2.0.1 removed support for xlsx format (3944)
luminosity - motifs (3956)
Handle secondary algorithms in yhat_values (3958)
SNYK-PYTHON-JINJA2-1012994 (3960)
custom_algorithm - adtk_level_shift (3970)
SNYK-PYTHON-PILLOW-1080635 (3972)
SNYK-PYTHON-AIOHTTP-1079232 (3974)
luminosity - classify_metrics (3978)
SNYK-PYTHON-URLLIB3-1085966 (3988)
Panorama - mirage not anomalous (3994)
EXTERNAL_SETTINGS (4000)
flux - aggregator.py and FLUX_AGGREGATE_NAMESPACES (4004)
SNYK-PYTHON-PILLOW-1090584 (4008)
Ionosphere - inference (4014)
Move mysql_select calls to SQLAlchemy (4022)
Change from scipy array to numpy array (4026)
refactoring (4030)
Mirage - removing feedback metrics to be processed (4048)
skyline.flux.worker.discarded metrics (4060)
ANALYZER_SKIP (4068)
CUSTOM_STALE_PERIOD (4076)
crucible - custom_algorithms (4118)
deps updates (4140)
webapp - stale_metrics API endpoint (4144)
check_data_sparsity - incorrect on low fidelity and inconsistent metrics (4146)
analyzer.metrics_manager.resolutions (4148)
metrics_manager - roomba batch processing metrics (4150)
DO_NOT_SKIP_SKYLINE_FEEDBACK_NAMESPACES (4152)
luminosity - cloudbursts (4164)
custom_algorithm - m66 (4180)
metrics_manager.boundary_metrics (4188)
functions.aws.send_sms (4196)
luminosity - cross_correlation_relationships (4264)
luminosity - related_namespaces (4278)
aet.metrics_manager.derivative_metrics Redis hash (4280)
Update dependencies (4282)
flux - telegraf (4284)
prometheus (4300)
matrixprofile - fN on big drops (4308)
flux - reload external_settings (4324)
webapp - panorama_plot_anomalies (4326)
BATCH_METRICS_CUSTOM_FULL_DURATIONS (4328)
SNYK-PYTHON-AIOHTTP-1584144 (4340)
Update dependencies (4344)
statsmodel 0.13.1 error (4346)
matplotlib 3.5.1 error (4348)
snyk - numpy and pillow updates (4362)
Prune old thunder.events (4364)
Switch git clone to https (4368)
webapp - handle url encoded chars (4374)
webapp - update_external_settings (4376)
flux - return reason with 400 (4380)
flux - quota (4400)
flux - external_settings - aggregation (4404)
settings - LOCAL_EXTERNAL_SETTINGS (4442)
webapp - inactive_metrics (4444)
Optimise horizon worker in_skip_list (4446)
Update dependencies (4452)
v0.19.1 (4456)
v0.19.1 (earthgecko-tsfresh) (4456)
flux - quota - cluster_sync (4464)
flux - remove_namespace_quota_metrics (4468)
Stationarity (4480)
Test alerts (4482)
settings - MONOTONIC_METRIC_NAMESPACES (4502)
Integrate opentelemetry (4514)
flux - opentelemetry traces (4516)
settings - LAST_KNOWN_VALUE_NAMESPACES (4518)
settings - ZERO_FILL_NAMESPACES (4520)
metrics_manager - derivative_metric_check (4528)
namespace.analysed_events (4530)
opentelemetry - rename InstrumentationLibrary to InstrumentationScope (4532)
Handle Redis failure (4536)
Plot matched timeseries (4540)
authoritative_node (4564)
mirage - process multiple metrics (4576)
webapp - api_get_fp_timeseries (4578)
custom_algorithm - skyline_ARTime (4584)
Ionosphere - inference - further validate all_in_range (4588)
flux - prometheus_alerts (4596)
functions.settings.manage_external_settings - use cache data on bad response (4608)
Support labelled metrics (4614)
memray (4620)
Change all dict copy to deepcopy (4624)
ionosphere.bulk.training (4650)
http_alerter - dotted_representation (4652)
ionosphere.learn_repetitive_patterns (4658)
settings.LUMINOSITY_CLOUDBURST_SKIP_METRICS (4662)
cloudburst active events only (4674)
analyzer - illuminance.all key (4676)
analyzer - anomalous metrics sets not flushing (4696)
Optimise Ionosphere Redis sets refresh when busy (4698)
algorithms - single series (4700)
numba optimisations (4702)
get_ionosphere_performance - allow multiple metric_like string (4704)
Add request_id and timing to ionosphere requests (4706)
ionosphere - store and cache fp minmax data (4708)
Reset mirage trigger_history on Ionosphere match (4712)
opentelemetry check is_instrumented_by_opentelemetry (4714)
dotted_representation breaking alert resend_queue (4720)
Handle alert and waterfall alert overlap (4722)
custom_algorithms - anomalous_daily_peak (4724)
vortex (4728)
flux vortex (4732)
mirage_vortex (4734)
custom_algorithms - sigma (4736)
Allow first_hour_average to handle different resolution (4738)
custom_algorithms - lof (4740)
custom_algorithms - skyline_prophet (4742)
custom_algorithms - spectral_residual (4744)
custom_algorithm - dbscan (4746)
custom_algorithm - isolation_forest (4748)
custom_algorithm - one_class_svm (4750)
custom_algorithm - pca (4750)
csv_to_timeseries (4754)
Use gevent gunicorn worker_class (4756)
Group metrics based on names and labels (4762)
analyzer_labelled_metrics - monotonicity checked incorrect classification (4766)
webapp - link to current VictoriaMetrics graph (4772)
Handle multiple metrics in victoriametrics response (4776)
v4.0.0 - update dependencies (4778)
Switch from matrixprofile to stumpy (4786)
functions.metrics_manager.manage_inactive_metrics (4792)
Deprecate run_script from crucible (4798)
Deprecate unittest2 (4800)
Manage NUMBA_CACHE_DIR (4806)
custom_algorithms - numba_cache_dirs (4808)
Handle error in sqlalchemy change (4816)
Deprecation of numpy asscalar function in luminol (4820)
vista - warn on remote_error (4824)
EXTERNAL_SETTINGS - handle key value in external_settings (4828)
webapp - panorama_plot_anomalies - all_events (4830)
webapp - api - get_all_activity (4834)
functions.metrics.get_namespace_metric.count (4838)
flux - prometheus - drop bucket metrics (4840)
flux - prometheus - x-test-only header (4840)
webapp - api - get_flux_test_metrics (4842)
Replace Redis getset with set with get (4844)
mirage - analyse.irregular.unstable.timeseries.at.30days (4848)
Allow for level to be passed in SYSLOG_OPTS (4852)
boundary - labelled_metrics (4854)
flux - prometheus - no 0s (4856)
analyzer_labelled_metrics - exceptions metrics set (4864)
Optimise luminosity for labelled_metrics (4872)
luminosity - related_metrics - labelled_metrics (4874)
labelled_metrics - resolution and data sparsity (4882)
Update dependencies - redis-py - CVE-2023-28858 (4884)
analyzer - operation_timings (4886)
analyzer - load_shedding (4888)
analyzer_labelled_metrics - use mrange (4890)
SNAB - labelled_metrics (4892)
labelled_metrics - SKYLINE_FEEDBACK_NAMESPACES (4894)
custom_algorithms - mstl (4896)
custom_algorithms - autoarima (4898)
Prevent training on metrics newer than 7 days (4902)
Handle send_graphite_metric failure (4904)
mute_alerts_on (4932)
Inefficient full_duration_timeseries_fill - last_known_value (4940)
vortex - m66 (4946)
vortex - adtk algorithms (4948)
custom_algorithm - spectral_entropy (4950)
vortex - consensus_count (4952)
webapp_features_profile - status (4958)
custom_algorithms - macd (4960)
Build and test skyline v4.0.0 (4962)
dawn - redis-stack-server (4964)
horizon.prometheus - handle TSDB not being able to parse LABELS (4968)
Ionosphere (922)
flux (956)