Merge pull request #427 from /issues/426

fixes #426 - 7.1.0 release
jantman · Sep 10, 2019 · 8790ab9 · 8790ab9
2 parents 8312a10 + 5dd674e
commit 8790ab9
Show file tree

Hide file tree

Showing 58 changed files with 5,041 additions and 754 deletions.
diff --git a/CHANGES.rst b/CHANGES.rst
@@ -1,10 +1,18 @@
 Changelog
 =========
 
-Unreleased Changes
+7.1.0 (2019-09-10)
 ------------------
 
 * `Issue #301 <https://github.com/jantman/awslimitchecker/issues/301>`__ - Distribute an official Docker image for awslimitchecker.
+* `Issue #421 <https://github.com/jantman/awslimitchecker/issues/421>`__
+
+  * Stop referencing deprecated ``botocore.vendored.requests.exceptions.ConnectTimeout`` in favor of new, and higher-level, ``botocore.exceptions.ConnectionError``
+  * In :py:meth:`awslimitchecker.utils._get_latest_version`, replace use of ``botocore.vendored.requests`` with ``urllib3``.
+
+* `Issue #324 <https://github.com/jantman/awslimitchecker/issues/324>`__ - Support loading :ref:`limit overrides <cli_usage.limit_overrides>` and/or :ref:`threshold overrides <cli_usage.threshold_overrides>` from a JSON file either stored locally or in S3 via new ``--limit-override-json`` and ``--threshold-override-json`` CLI options.
+* `Issue #418 <https://github.com/jantman/awslimitchecker/issues/418>`__ - Add support for sending runtime, limits, and usage to :ref:`<metric providers <cli_usage.metrics>` such as Datadog.
+* `Issue #419 <https://github.com/jantman/awslimitchecker/issues/419>`__ - Add support for alerts/notifications of thresholds crossed or failed runs (exceptions) via :ref:`<alert providers <cli_usage.alerts>` such as PagerDuty.
 
 7.0.0 (2019-08-13)
 ------------------

diff --git a/README.rst b/README.rst
@@ -101,6 +101,8 @@ What It Does
   an optional maximum time limit). See
   `Getting Started - Trusted Advisor <http://awslimitchecker.readthedocs.io/en/latest/getting_started.html#trusted-advisor>`_
   for more information.
+- Optionally send current usage and limit metrics to a metrics store, such as Datadog.
+- Optionally send warning/critical alerts to notification providers, such as PagerDuty.
 
 Requirements
 ------------

diff --git a/awslimitchecker/alerts/__init__.py b/awslimitchecker/alerts/__init__.py
@@ -0,0 +1,42 @@
+"""
+awslimitchecker/alerts/__init__.py
+
+The latest version of this package is available at:
+<https://github.com/jantman/awslimitchecker>
+
+################################################################################
+Copyright 2015-2019 Jason Antman <jason@jasonantman.com>
+
+    This file is part of awslimitchecker, also known as awslimitchecker.
+
+    awslimitchecker is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    awslimitchecker is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with awslimitchecker.  If not, see <http://www.gnu.org/licenses/>.
+
+The Copyright and Authors attributions contained herein may not be removed or
+otherwise altered, except to add the Author attribution of a contributor to
+this work. (Additional Terms pursuant to Section 7b of the AGPL v3)
+################################################################################
+While not legally required, I sincerely request that anyone who finds
+bugs please submit them at <https://github.com/jantman/awslimitchecker> or
+to me via email, and that you send any contributions or improvements
+either as a pull request on GitHub, or to me via email.
+################################################################################
+
+AUTHORS:
+Jason Antman <jason@jasonantman.com> <http://www.jasonantman.com>
+################################################################################
+"""
+
+from .base import AlertProvider
+from .dummy import Dummy
+from .pagerdutyv1 import PagerDutyV1
diff --git a/awslimitchecker/alerts/base.py b/awslimitchecker/alerts/base.py
@@ -0,0 +1,142 @@
+"""
+awslimitchecker/alerts/base.py
+
+The latest version of this package is available at:
+<https://github.com/jantman/awslimitchecker>
+
+################################################################################
+Copyright 2015-2019 Jason Antman <jason@jasonantman.com>
+
+    This file is part of awslimitchecker, also known as awslimitchecker.
+
+    awslimitchecker is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    awslimitchecker is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with awslimitchecker.  If not, see <http://www.gnu.org/licenses/>.
+
+The Copyright and Authors attributions contained herein may not be removed or
+otherwise altered, except to add the Author attribution of a contributor to
+this work. (Additional Terms pursuant to Section 7b of the AGPL v3)
+################################################################################
+While not legally required, I sincerely request that anyone who finds
+bugs please submit them at <https://github.com/jantman/awslimitchecker> or
+to me via email, and that you send any contributions or improvements
+either as a pull request on GitHub, or to me via email.
+################################################################################
+
+AUTHORS:
+Jason Antman <jason@jasonantman.com> <http://www.jasonantman.com>
+################################################################################
+"""
+
+import logging
+from abc import ABCMeta, abstractmethod
+
+logger = logging.getLogger(__name__)
+
+
+class AlertProvider(object):
+
+    __metaclass__ = ABCMeta
+
+    def __init__(self, region_name):
+        """
+        Initialize an AlertProvider class. This MUST be overridden by
+        subclasses. All configuration must be passed as keyword arguments
+        to the class constructor (these come from ``--alert-config`` CLI
+        arguments). Any dependency imports must be made in the constructor.
+        The constructor should do as much as possible to validate configuration.
+
+        :param region_name: the name of the region we're connected to
+        :type region_name: str
+        """
+        self._region_name = region_name
+
+    @abstractmethod
+    def on_success(self, duration=None):
+        """
+        Method called when no thresholds were breached, and run completed
+        successfully. Should resolve any open incidents (if the service supports
+        that functionality) or else simply return.
+
+        :param duration: duration of the usage/threshold checking run
+        :type duration: float
+        """
+        raise NotImplementedError()
+
+    @abstractmethod
+    def on_critical(self, problems, problem_str, exc=None, duration=None):
+        """
+        Method called when the run encountered errors, or at least one critical
+        threshold was met or crossed.
+
+        :param problems: dict of service name to nested dict of limit name to
+          limit, same format as the return value of
+          :py:meth:`~.AwsLimitChecker.check_thresholds`. ``None`` if ``exc`` is
+          specified.
+        :type problems: dict or None
+        :param problem_str: String representation of ``problems``, as displayed
+          in ``awslimitchecker`` command line output. ``None`` if ``exc`` is
+          specified.
+        :type problem_str: str or None
+        :param exc: Exception object that was raised during the run (optional)
+        :type exc: Exception
+        :param duration: duration of the run
+        :type duration: float
+        """
+        raise NotImplementedError()
+
+    @abstractmethod
+    def on_warning(self, problems, problem_str, duration=None):
+        """
+        Method called when one or more warning thresholds were crossed, but no
+        criticals and the run did not encounter any errors.
+
+        :param problems: dict of service name to nested dict of limit name to
+          limit, same format as the return value of
+          :py:meth:`~.AwsLimitChecker.check_thresholds`.
+        :type problems: dict or None
+        :param problem_str: String representation of ``problems``, as displayed
+          in ``awslimitchecker`` command line output.
+        :type problem_str: str or None
+        :param duration: duration of the run
+        :type duration: float
+        """
+        raise NotImplementedError()
+
+    @staticmethod
+    def providers_by_name():
+        """
+        Return a dict of available AlertProvider subclass names to the class
+        objects.
+
+        :return: AlertProvider class names to classes
+        :rtype: dict
+        """
+        return {x.__name__: x for x in AlertProvider.__subclasses__()}
+
+    @staticmethod
+    def get_provider_by_name(name):
+        """
+        Get a reference to the provider class with the specified name.
+
+        :param name: name of the AlertProvider subclass
+        :type name: str
+        :return: AlertProvider subclass
+        :rtype: ``class``
+        :raises: RuntimeError
+        """
+        try:
+            return AlertProvider.providers_by_name()[name]
+        except KeyError:
+            raise RuntimeError(
+                'ERROR: "%s" is not a valid AlertProvider class name' % name
+            )
diff --git a/awslimitchecker/alerts/dummy.py b/awslimitchecker/alerts/dummy.py
@@ -0,0 +1,125 @@
+"""
+awslimitchecker/alerts/dummy.py
+
+The latest version of this package is available at:
+<https://github.com/jantman/awslimitchecker>
+
+################################################################################
+Copyright 2015-2019 Jason Antman <jason@jasonantman.com>
+
+    This file is part of awslimitchecker, also known as awslimitchecker.
+
+    awslimitchecker is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    awslimitchecker is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with awslimitchecker.  If not, see <http://www.gnu.org/licenses/>.
+
+The Copyright and Authors attributions contained herein may not be removed or
+otherwise altered, except to add the Author attribution of a contributor to
+this work. (Additional Terms pursuant to Section 7b of the AGPL v3)
+################################################################################
+While not legally required, I sincerely request that anyone who finds
+bugs please submit them at <https://github.com/jantman/awslimitchecker> or
+to me via email, and that you send any contributions or improvements
+either as a pull request on GitHub, or to me via email.
+################################################################################
+
+AUTHORS:
+Jason Antman <jason@jasonantman.com> <http://www.jasonantman.com>
+################################################################################
+"""
+
+import logging
+from .base import AlertProvider
+
+logger = logging.getLogger(__name__)
+
+
+class Dummy(AlertProvider):
+    """
+    Dummy alert provider that just outputs alerts to STDOUT (this will
+    effectively duplicate awslimitchecker's CLI output).
+    """
+
+    def __init__(self, region_name, **_):
+        """
+        Initialize an AlertProvider class. This MUST be overridden by
+        subclasses. All configuration must be passed as keyword arguments
+        to the class constructor (these come from ``--alert-config`` CLI
+        arguments). Any dependency imports must be made in the constructor.
+        The constructor should do as much as possible to validate configuration.
+
+        :param region_name: the name of the region we're connected to
+        :type region_name: str
+        """
+        super(Dummy, self).__init__(region_name)
+
+    def on_success(self, duration=None):
+        """
+        Method called when no thresholds were breached, and run completed
+        successfully. Should resolve any open incidents (if the service supports
+        that functionality) or else simply return.
+
+        :param duration: duration of the usage/threshold checking run
+        :type duration: float
+        """
+        print('awslimitchecker in %s found no problems' % self._region_name)
+        if duration:
+            print('awslimitchecker run duration: %s' % duration)
+
+    def on_critical(self, problems, problem_str, exc=None, duration=None):
+        """
+        Method called when the run encountered errors, or at least one critical
+        threshold was met or crossed.
+
+        :param problems: dict of service name to nested dict of limit name to
+          limit, same format as the return value of
+          :py:meth:`~.AwsLimitChecker.check_thresholds`. ``None`` if ``exc`` is
+          specified.
+        :type problems: dict or None
+        :param problem_str: String representation of ``problems``, as displayed
+          in ``awslimitchecker`` command line output. ``None`` if ``exc`` is
+          specified.
+        :type problem_str: str or None
+        :param exc: Exception object that was raised during the run (optional)
+        :type exc: Exception
+        :param duration: duration of the run
+        :type duration: float
+        """
+        if exc is not None:
+            print('awslimitchecker in %s failed with exception: %s' % (
+                self._region_name, exc.__repr__()
+            ))
+        else:
+            print('awslimitchecker in %s found CRITICALS:' % self._region_name)
+            print(problem_str)
+        if duration:
+            print('awslimitchecker run duration: %s' % duration)
+
+    def on_warning(self, problems, problem_str, duration=None):
+        """
+        Method called when one or more warning thresholds were crossed, but no
+        criticals and the run did not encounter any errors.
+
+        :param problems: dict of service name to nested dict of limit name to
+          limit, same format as the return value of
+          :py:meth:`~.AwsLimitChecker.check_thresholds`.
+        :type problems: dict or None
+        :param problem_str: String representation of ``problems``, as displayed
+          in ``awslimitchecker`` command line output.
+        :type problem_str: str or None
+        :param duration: duration of the run
+        :type duration: float
+        """
+        print('awslimitchecker in %s found WARNINGS:' % self._region_name)
+        print(problem_str)
+        if duration:
+            print('awslimitchecker run duration: %s' % duration)