diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 00000000..422954aa --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,4 @@ +Contributing to awslimitchecker +=============================== + +See the [Development page of the documentation on ReadTheDocs](http://awslimitchecker.readthedocs.org/en/develop/development.html) for information on how to contribute to awslimitchecker. diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md new file mode 100644 index 00000000..82347402 --- /dev/null +++ b/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,71 @@ +Before opening an issue, please see the +[Getting Help](http://awslimitchecker.readthedocs.org/en/latest/getting_help.html) +page of the documentation. + +Please remove all of this template but the relevant section below, and fill in +each item in that section. + +## Feature Request + +If your feature request is for support of a service or limit not currently +supported by awslimitchecker, you can simply title the issue "add support for +" and add a simple description. + +For anything else, please fill in the following: + +### Feature Description + +Describe in detail the feature you would like to see implemented, especially +how it would work from a user perspective and what benefits it adds. Your description +should be detailed enough to be used to determine if code written for the feature +adequately solves the problem. + +### Use Cases + +Describe one or more use cases for why this feature will be useful. + +### Testing Assistance + +Indicate whether or not you will be able to assist in testing pre-release +code for the feature. + +## Bug Report + +When reporting a bug in awslimitchecker, please provide all of the following information, +as well as any additional details that may be useful in reproducing or fixing +the issue: + +### Version + +awslimitchecker version, as reported by ``awslimitchecker --version`` + +### Installation Method + +How was awslimitchecker installed (provide as much detail as possible, ideally +the exact command used and whether it was installed in a virtualenv or not). + +### Supporting Software Versions + +The output of ``python --version`` and ``virtualenv --version`` in the environment +that awslimitchecker is running in, as well as your operating system type and version. + +### Actual Output + +``` +Paste here the output of awslimitchecker (including the command used to run it), +run with the -vv (debug-level output) flag, that shows the issue. +``` + +### Expected Output + +Describe the output that you expected (what's wrong). If possible, after your description, +copy the actual output above and modify it to what was expected. + +### TrustedAdvisor + +If the bug/issue is related to TrustedAdvisor, which support contract your account has. + +### Testing Assistance + +Indicate whether or not you will be able to assist in testing pre-release +code for the feature. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 00000000..ccc6828f --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,23 @@ +Before submitting pull requests, please see the +[Development documentation](http://awslimitchecker.readthedocs.org/en/latest/development.html) +and specifically the [Pull Request Guidelines](http://awslimitchecker.readthedocs.org/en/latest/development.html#pull-requests). + +# Pull Request Checklist + +- [ ] Code should conform to the [Development Guidelines](http://awslimitchecker.readthedocs.org/en/latest/development.html#guidelines): + - [ ] pep8 compliant with some exceptions (see pytest.ini) + - [ ] 100% test coverage with pytest (with valid tests). If you have difficulty + writing tests for the code, feel free to ask for help or submit the PR without tests. + - [ ] Complete, correctly-formatted documentation for all classes, functions and methods. + - [ ] documentation has been rebuilt with ``tox -e docs`` + - [ ] Connections to the AWS services should only be made by the class's + ``connect()`` and ``connect_resource()`` methods, inherited from + [awslimitchecker.connectable.Connectable](http://awslimitchecker.readthedocs.org/en/latest/awslimitchecker.connectable.html) + - [ ] All modules should have (and use) module-level loggers. + - [ ] **Commit messages** should be meaningful, and reference the Issue number + if you're working on a GitHub issue (i.e. "issue #x - "). Please + refrain from using the "fixes #x" notation unless you are *sure* that the + the issue is fixed in that commit. + - [ ] Git history is fully intact; please do not squash or rewrite history. +- [ ] If you made changes to the ``versioncheck`` code, be sure to locally run the +``-versioncheck`` tox tests. diff --git a/.travis.yml b/.travis.yml index 44bd894c..81ee8455 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,24 +3,46 @@ sudo: false cache: directories: - "$HOME/.pip-cache/" -env: -- TOXENV=py26-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py27-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py32-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py33-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py34-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=pypy-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=pypy3-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py26-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py27-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py32-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py33-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=py34-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=pypy-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=pypy3-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=docs PIP_DOWNLOAD_CACHE=$HOME/.pip-cache -- TOXENV=integration -- TOXENV=integration3 +matrix: + include: + - python: "2.6" + env: TOXENV=py26-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "2.7" + env: TOXENV=py27-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.2" + env: TOXENV=py32-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.3" + env: TOXENV=py33-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.4" + env: TOXENV=py34-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.5" + env: TOXENV=py35-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "pypy" + env: TOXENV=pypy-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "pypy3" + env: TOXENV=pypy3-unit PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "2.6" + env: TOXENV=py26-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "2.7" + env: TOXENV=py27-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.2" + env: TOXENV=py32-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.3" + env: TOXENV=py33-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.4" + env: TOXENV=py34-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "3.5" + env: TOXENV=py35-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "pypy" + env: TOXENV=pypy-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "pypy3" + env: TOXENV=pypy3-versioncheck PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "2.7" + env: TOXENV=docs PIP_DOWNLOAD_CACHE=$HOME/.pip-cache + - python: "2.7" + env: TOXENV=integration + - python: "3.4" + env: TOXENV=integration3 install: - virtualenv --version; test $TOXENV = "py32-unit" -o $TOXENV = "py32-versioncheck" -o $TOXENV = "pypy3-unit" -o $TOXENV = "pypy3-versioncheck" && pip install --upgrade virtualenv==13.1.2 || /bin/true - git config --global user.email "travisci@jasonantman.com" diff --git a/CHANGES.rst b/CHANGES.rst index ae2b3b72..96c8feea 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -4,6 +4,16 @@ Changelog Pre-release (develop branch) ---------------------------- + +0.3.1 (2016-03-04) +------------------ + +* `#117 `_ fix Python 3.5 TravisCI tests and re-enable automatic testing for 3.5. +* `#116 `_ add t2.nano EC2 instance type; fix typo - "m4.8xlarge" should have been "m4.10xlarge"; update default limits for m4.(4|10)xlarge +* `#134 `_ Minor update to project description in docs and setup.py; use only _VERSION (not git) when building in RTD; include short description in docs HTML title; set meta description on docs index.rst. +* `#128 `_ Update Development and Getting Help documentation; add GitHub CONTRIBUTING.md file with link back to docs, as well as Issue and PR templates. +* `#131 `_ Refactor TrustedAdvisor interaction with limits for special naming cases (limits where the TrustedAdvisor service or limit name doesn't match that of the awslimitchecker limit); enable newly-available TrustedAdvisor data for some EC2 on-demand instance usage. + 0.3.0 (2016-02-18) ------------------ diff --git a/README.rst b/README.rst index 4a802eee..8805fdad 100644 --- a/README.rst +++ b/README.rst @@ -57,7 +57,7 @@ Develop: :target: https://readthedocs.org/projects/awslimitchecker/?badge=develop :alt: sphinx documentation for develop branch -A script and python module to check your AWS service limits and usage using `boto `_. +A script and python module to check your AWS service limits and usage, and warn when usage approaches limits. Users building out scalable services in Amazon AWS often run into AWS' `service limits `_ - often at the least convenient time (i.e. mid-deploy or when autoscaling fails). Amazon's `Trusted Advisor `_ @@ -145,8 +145,8 @@ For basic usage, see: See the `project documentation `_ for further information. -Bugs, Feature Requests ----------------------- +Bugs, Feature Requests, Support +------------------------------- Questions, comments, Bug reports and feature requests are happily accepted via the `GitHub Issue Tracker `_. @@ -155,6 +155,10 @@ Pull requests are always welcome. Please see the `Development `_ and `Getting Help `_ documentation for more information. +For paid support and development options, please see the +`Enterprise Support Agreements and Contract Development `_ +section of the documentation. + Changelog --------- diff --git a/awslimitchecker/checker.py b/awslimitchecker/checker.py index 920492a2..ea30acf5 100644 --- a/awslimitchecker/checker.py +++ b/awslimitchecker/checker.py @@ -122,19 +122,20 @@ def __init__(self, warning_threshold=80, critical_threshold=99, self.mfa_token = mfa_token self.region = region self.services = {} + for sname, cls in _services.items(): + self.services[sname] = cls(warning_threshold, critical_threshold, + account_id, account_role, region, + external_id, mfa_serial_number, + mfa_token) self.ta = TrustedAdvisor( + self.services, account_id=account_id, account_role=account_role, region=region, external_id=external_id, mfa_serial_number=mfa_serial_number, - mfa_token=mfa_token + mfa_token=mfa_token, ) - for sname, cls in _services.items(): - self.services[sname] = cls(warning_threshold, critical_threshold, - account_id, account_role, region, - external_id, mfa_serial_number, - mfa_token) def get_version(self): """ @@ -175,7 +176,7 @@ def get_limits(self, service=None, use_ta=True): if service is not None: to_get = {service: self.services[service]} if use_ta: - self.ta.update_limits(to_get) + self.ta.update_limits() for sname, cls in to_get.items(): if hasattr(cls, '_update_limits_from_api'): cls._update_limits_from_api() @@ -211,7 +212,7 @@ def find_usage(self, service=None, use_ta=True): if service is not None: to_get = {service: self.services[service]} if use_ta: - self.ta.update_limits(to_get) + self.ta.update_limits() for cls in to_get.values(): if hasattr(cls, '_update_limits_from_api'): cls._update_limits_from_api() @@ -409,7 +410,7 @@ def check_thresholds(self, service=None, use_ta=True): if service is not None: to_get = {service: self.services[service]} if use_ta: - self.ta.update_limits(to_get) + self.ta.update_limits() for sname, cls in to_get.items(): if hasattr(cls, '_update_limits_from_api'): cls._update_limits_from_api() diff --git a/awslimitchecker/limit.py b/awslimitchecker/limit.py index aba4a63e..504d8206 100644 --- a/awslimitchecker/limit.py +++ b/awslimitchecker/limit.py @@ -54,7 +54,8 @@ class AwsLimit(object): def __init__(self, name, service, default_limit, def_warning_threshold, def_critical_threshold, - limit_type=None, limit_subtype=None): + limit_type=None, limit_subtype=None, + ta_service_name=None, ta_limit_name=None): """ Describes one specific AWS service limit, as well as its current utilization, default limit, thresholds, and any @@ -65,7 +66,7 @@ def __init__(self, name, service, default_limit, :type name: string :param service: the :py:class:`~._AwsService` class that this limit is for - :type service_name: :py:class:`~._AwsService` + :type service: :py:class:`~._AwsService` :param default_limit: the default value of this limit for new accounts :type default_limit: int :param def_warning_threshold: the default warning threshold, as an @@ -80,6 +81,13 @@ def __init__(self, name, service, default_limit, such as "AWS::EC2::Instance" or "AWS::RDS::DBSubnetGroup". :param limit_subtype: resource sub-type for this limit, if applicable, such as "t2.micro" or "SecurityGroup" + :type limit_subtype: str + :param ta_service_name: The service name returned by Trusted Advisor + for this limit, if different from the name of ``service`` + :type ta_service_name: str + :param ta_limit_name: The limit name returned by Trusted Advisor for + this limit, if different from ``name``. + :type ta_limit_name: str :raises: ValueError """ if def_warning_threshold >= def_critical_threshold: @@ -103,6 +111,8 @@ def __init__(self, name, service, default_limit, self.crit_count = None self._warnings = [] self._criticals = [] + self._ta_service_name = ta_service_name + self._ta_limit_name = ta_limit_name def set_limit_override(self, limit_value, override_ta=True): """ @@ -379,6 +389,34 @@ def get_criticals(self): """ return self._criticals + @property + def ta_service_name(self): + """ + Return the effective Trusted Advisor service name that this limit's + data will have. This should be ``self._ta_service_name`` if set, + otherwise the name of ``self.service``. + + :return: Trusted Advisor service data name + :rtype: str + """ + if self._ta_service_name is not None: + return self._ta_service_name + return self.service.service_name + + @property + def ta_limit_name(self): + """ + Return the effective Trusted Advisor limit name that this limit's + data will have. This should be ``self._ta_limit_name`` if set, + otherwise ``self.name``. + + :return: Trusted Advisor limit data name + :rtype: str + """ + if self._ta_limit_name is not None: + return self._ta_limit_name + return self.name + class AwsLimitUsage(object): diff --git a/awslimitchecker/services/ec2.py b/awslimitchecker/services/ec2.py index af7e5a39..7d585d06 100644 --- a/awslimitchecker/services/ec2.py +++ b/awslimitchecker/services/ec2.py @@ -243,6 +243,8 @@ def _get_limits_instances(self): 'i2.8xlarge': (2, 20, 0), 'd2.4xlarge': (10, 20, 5), 'd2.8xlarge': (5, 20, 5), + 'm4.4xlarge': (10, 20, 5), + 'm4.10xlarge': (5, 20, 5) } limits = {} for i_type in self._instance_types(): @@ -258,7 +260,8 @@ def _get_limits_instances(self): self.warning_threshold, self.critical_threshold, limit_type='On-Demand instances', - limit_subtype=i_type + limit_subtype=i_type, + ta_limit_name='On-Demand instances - %s' % i_type ) # limit for ALL running On-Demand instances key = 'Running On-Demand EC2 instances' @@ -358,6 +361,7 @@ def _get_limits_networking(self): self.critical_threshold, limit_type='AWS::EC2::EIP', limit_subtype='AWS::EC2::VPC', + ta_service_name='VPC' # TA shows this as VPC not EC2 ) # the EC2 limits screen calls this 'EC2-Classic Elastic IPs' # but Trusted Advisor just calls it 'Elastic IP addresses (EIPs)' @@ -413,6 +417,7 @@ def _instance_types(self): :rtype: list """ GENERAL_TYPES = [ + 't2.nano', 't2.micro', 't2.small', 't2.medium', @@ -425,7 +430,7 @@ def _instance_types(self): 'm4.xlarge', 'm4.2xlarge', 'm4.4xlarge', - 'm4.8xlarge', + 'm4.10xlarge' ] PREV_GENERAL_TYPES = [ diff --git a/awslimitchecker/tests/services/test_ec2.py b/awslimitchecker/tests/services/test_ec2.py index 17d846c8..3a1be056 100644 --- a/awslimitchecker/tests/services/test_ec2.py +++ b/awslimitchecker/tests/services/test_ec2.py @@ -72,7 +72,7 @@ def test_init(self): def test_instance_types(self): cls = _Ec2Service(21, 43) types = cls._instance_types() - assert len(types) == 53 + assert len(types) == 54 assert 't2.micro' in types assert 'r3.8xlarge' in types assert 'c3.large' in types @@ -121,7 +121,7 @@ def test_get_limits_all(self): def test_get_limits_instances(self): cls = _Ec2Service(21, 43) limits = cls._get_limits_instances() - assert len(limits) == 54 + assert len(limits) == 55 # check a random subset of limits t2_micro = limits['Running On-Demand t2.micro instances'] assert t2_micro.default_limit == 20 @@ -140,6 +140,12 @@ def test_get_limits_instances(self): assert all_ec2.limit_type == 'On-Demand instances' assert all_ec2.limit_subtype is None assert 'Running On-Demand m4.4xlarge instances' in limits + for lname, lim in limits.items(): + assert lim.limit_type == 'On-Demand instances' + itype = lim.limit_subtype + if itype is not None: + assert lname == 'Running On-Demand %s instances' % itype + assert lim.ta_limit_name == 'On-Demand instances - %s' % itype def test_find_usage(self): with patch.multiple( @@ -454,6 +460,8 @@ def test_get_limits_networking(self): 'VPC security groups per elastic network interface', ] assert sorted(limits.keys()) == sorted(expected) + assert limits[ + 'VPC Elastic IP addresses (EIPs)'].ta_service_name == 'VPC' def test_update_limits_from_api(self): data = fixtures.test_update_limits_from_api diff --git a/awslimitchecker/tests/test_checker.py b/awslimitchecker/tests/test_checker.py index 231b4957..c3f60b39 100644 --- a/awslimitchecker/tests/test_checker.py +++ b/awslimitchecker/tests/test_checker.py @@ -105,10 +105,11 @@ def setup(self): def test_init(self): # dict should be of _AwsService instances - assert self.cls.services == { + services = { 'SvcFoo': self.mock_svc1, 'SvcBar': self.mock_svc2 } + assert self.cls.services == services # _AwsService instances should exist, but have no other calls assert self.mock_foo.mock_calls == [ call(80, 99, None, None, None, None, None, None) @@ -117,7 +118,7 @@ def test_init(self): call(80, 99, None, None, None, None, None, None) ] assert self.mock_ta_constr.mock_calls == [ - call(account_id=None, account_role=None, region=None, + call(services, account_id=None, account_role=None, region=None, external_id=None, mfa_serial_number=None, mfa_token=None) ] assert self.mock_svc1.mock_calls == [] @@ -169,10 +170,11 @@ def test_init_thresholds(self): critical_threshold=22, ) # dict should be of _AwsService instances - assert cls.services == { + services = { 'SvcFoo': mock_svc1, 'SvcBar': mock_svc2 } + assert cls.services == services # _AwsService instances should exist, but have no other calls assert mock_foo.mock_calls == [ call(5, 22, None, None, None, None, None, None) @@ -181,7 +183,7 @@ def test_init_thresholds(self): call(5, 22, None, None, None, None, None, None) ] assert mock_ta_constr.mock_calls == [ - call(account_id=None, account_role=None, region=None, + call(services, account_id=None, account_role=None, region=None, external_id=None, mfa_serial_number=None, mfa_token=None) ] assert mock_svc1.mock_calls == [] @@ -213,10 +215,11 @@ def test_init_region(self): mocks['TrustedAdvisor'].return_value = mock_ta cls = AwsLimitChecker(region='myregion') # dict should be of _AwsService instances - assert cls.services == { + services = { 'SvcFoo': mock_svc1, 'SvcBar': mock_svc2 } + assert cls.services == services # _AwsService instances should exist, but have no other calls assert mock_foo.mock_calls == [ call(80, 99, None, None, 'myregion', None, None, None) @@ -225,8 +228,9 @@ def test_init_region(self): call(80, 99, None, None, 'myregion', None, None, None) ] assert mock_ta_constr.mock_calls == [ - call(account_id=None, account_role=None, region='myregion', - external_id=None, mfa_serial_number=None, mfa_token=None) + call(services, account_id=None, account_role=None, + region='myregion', external_id=None, mfa_serial_number=None, + mfa_token=None) ] assert mock_svc1.mock_calls == [] assert mock_svc2.mock_calls == [] @@ -261,10 +265,11 @@ def test_init_sts(self): region='myregion' ) # dict should be of _AwsService instances - assert cls.services == { + services = { 'SvcFoo': mock_svc1, 'SvcBar': mock_svc2 } + assert cls.services == services # _AwsService instances should exist, but have no other calls assert mock_foo.mock_calls == [ call(80, 99, '123456789012', 'myrole', 'myregion', None, @@ -276,12 +281,13 @@ def test_init_sts(self): ] assert mock_ta_constr.mock_calls == [ call( + services, account_id='123456789012', account_role='myrole', region='myregion', external_id=None, mfa_serial_number=None, - mfa_token=None + mfa_token=None, ) ] assert mock_svc1.mock_calls == [] @@ -320,10 +326,11 @@ def test_init_sts_external_id(self): mfa_token=None ) # dict should be of _AwsService instances - assert cls.services == { + services = { 'SvcFoo': mock_svc1, 'SvcBar': mock_svc2 } + assert cls.services == services # _AwsService instances should exist, but have no other calls assert mock_foo.mock_calls == [ call(80, 99, '123456789012', 'myrole', 'myregion', 'myextid', @@ -335,12 +342,13 @@ def test_init_sts_external_id(self): ] assert mock_ta_constr.mock_calls == [ call( + services, account_id='123456789012', account_role='myrole', region='myregion', external_id='myextid', mfa_serial_number=None, - mfa_token=None + mfa_token=None, ) ] assert mock_svc1.mock_calls == [] @@ -375,10 +383,7 @@ def test_get_limits(self): res = self.cls.get_limits() assert res == limits assert self.mock_ta.mock_calls == [ - call.update_limits({ - 'SvcFoo': self.mock_svc1, - 'SvcBar': self.mock_svc2, - }) + call.update_limits() ] assert self.mock_svc1.mock_calls == [ call.get_limits() @@ -410,9 +415,7 @@ def test_get_limits_service(self): res = self.cls.get_limits(service='SvcFoo') assert res == {'SvcFoo': limits['SvcFoo']} assert self.mock_ta.mock_calls == [ - call.update_limits({ - 'SvcFoo': self.mock_svc1, - }) + call.update_limits() ] assert self.mock_svc1.mock_calls == [ call.get_limits() @@ -426,9 +429,7 @@ def test_get_limits_service_with_api(self): res = self.cls.get_limits(service='SvcBar') assert res == {'SvcBar': limits['SvcBar']} assert self.mock_ta.mock_calls == [ - call.update_limits({ - 'SvcBar': self.mock_svc2, - }) + call.update_limits() ] assert self.mock_svc1.mock_calls == [] assert self.mock_svc2.mock_calls == [ @@ -446,10 +447,7 @@ def test_find_usage(self): call.find_usage() ] assert self.mock_ta.mock_calls == [ - call.update_limits({ - 'SvcFoo': self.mock_svc1, - 'SvcBar': self.mock_svc2, - }) + call.update_limits() ] def test_find_usage_no_ta(self): @@ -470,7 +468,7 @@ def test_find_usage_service(self): ] assert self.mock_svc2.mock_calls == [] assert self.mock_ta.mock_calls == [ - call.update_limits({'SvcFoo': self.mock_svc1}) + call.update_limits() ] def test_find_usage_service_with_api(self): @@ -481,7 +479,7 @@ def test_find_usage_service_with_api(self): call.find_usage() ] assert self.mock_ta.mock_calls == [ - call.update_limits({'SvcBar': self.mock_svc2}) + call.update_limits() ] def test_set_threshold_overrides(self): @@ -684,10 +682,7 @@ def test_check_thresholds(self): } } assert self.mock_ta.mock_calls == [ - call.update_limits({ - 'SvcFoo': self.mock_svc1, - 'SvcBar': self.mock_svc2 - }), + call.update_limits(), ] assert self.mock_svc1.mock_calls == [ call.check_thresholds() @@ -707,7 +702,7 @@ def test_check_thresholds_service(self): } } assert self.mock_ta.mock_calls == [ - call.update_limits({'SvcFoo': self.mock_svc1}) + call.update_limits() ] assert self.mock_svc1.mock_calls == [ call.check_thresholds() @@ -724,7 +719,7 @@ def test_check_thresholds_service_api(self): } } assert self.mock_ta.mock_calls == [ - call.update_limits({'SvcBar': self.mock_svc2}) + call.update_limits() ] assert self.mock_svc1.mock_calls == [] assert self.mock_svc2.mock_calls == [ diff --git a/awslimitchecker/tests/test_limit.py b/awslimitchecker/tests/test_limit.py index 3304b33c..b0a15ecf 100644 --- a/awslimitchecker/tests/test_limit.py +++ b/awslimitchecker/tests/test_limit.py @@ -60,6 +60,7 @@ class TestAwsLimit(object): def setup(self): self.mock_svc = Mock(spec_set=_AwsService) + type(self.mock_svc).service_name = 'mysname' def test_init(self): limit = AwsLimit( @@ -80,6 +81,32 @@ def test_init(self): assert limit.api_limit is None assert limit.def_warning_threshold == 7 assert limit.def_critical_threshold == 11 + assert limit._ta_service_name is None + assert limit._ta_limit_name is None + + def test_init_ta_names(self): + limit = AwsLimit( + 'limitname', + self.mock_svc, + 3, + 7, + 11, + ta_service_name='foo', + ta_limit_name='bar' + ) + assert limit.name == 'limitname' + assert limit.service == self.mock_svc + assert limit.default_limit == 3 + assert limit.limit_type is None + assert limit.limit_subtype is None + assert limit.limit_override is None + assert limit.override_ta is True + assert limit.ta_limit is None + assert limit.api_limit is None + assert limit.def_warning_threshold == 7 + assert limit.def_critical_threshold == 11 + assert limit._ta_service_name == 'foo' + assert limit._ta_limit_name == 'bar' def test_init_valueerror(self): with pytest.raises(ValueError) as excinfo: @@ -522,6 +549,26 @@ def test_set_threshold_override(self): assert limit.crit_percent == 3 assert limit.crit_count == 4 + def test_ta_service_name_default(self): + limit = AwsLimit('limitname', self.mock_svc, 100, 1, 2) + assert limit.ta_service_name == 'mysname' + + def test_ta_service_name_overridden(self): + limit = AwsLimit( + 'limitname', self.mock_svc, 100, 1, 2, ta_service_name='foo' + ) + assert limit.ta_service_name == 'foo' + + def test_ta_limit_name_default(self): + limit = AwsLimit('limitname', self.mock_svc, 100, 1, 2) + assert limit.ta_limit_name == 'limitname' + + def test_ta_limit_name_overridden(self): + limit = AwsLimit( + 'limitname', self.mock_svc, 100, 1, 2, ta_limit_name='foo' + ) + assert limit.ta_limit_name == 'foo' + class TestAwsLimitUsage(object): diff --git a/awslimitchecker/tests/test_trustedadvisor.py b/awslimitchecker/tests/test_trustedadvisor.py index 42612d0a..f4f20099 100644 --- a/awslimitchecker/tests/test_trustedadvisor.py +++ b/awslimitchecker/tests/test_trustedadvisor.py @@ -41,6 +41,7 @@ from botocore.exceptions import ClientError from awslimitchecker.trustedadvisor import TrustedAdvisor from awslimitchecker.services.base import _AwsService +from awslimitchecker.limit import AwsLimit import pytest # https://code.google.com/p/mock/issues/detail?id=249 @@ -65,7 +66,7 @@ def setup(self): self.mock_client_config = Mock() type(self.mock_client_config).region_name = 'us-east-1' type(self.mock_conn)._client_config = self.mock_client_config - self.cls = TrustedAdvisor() + self.cls = TrustedAdvisor({}) self.cls.conn = self.mock_conn self.mock_svc1 = Mock(spec_set=_AwsService) @@ -76,7 +77,7 @@ def setup(self): } def test_init(self): - cls = TrustedAdvisor() + cls = TrustedAdvisor({}) assert cls.conn is None assert cls.account_id is None assert cls.account_role is None @@ -85,9 +86,15 @@ def test_init(self): assert cls.external_id is None assert cls.mfa_serial_number is None assert cls.mfa_token is None + assert cls.all_services == {} def test_init_sts(self): - cls = TrustedAdvisor(account_id='aid', account_role='role', region='r') + mock_svc = Mock(spec_set=_AwsService) + mock_svc.get_limits.return_value = {} + cls = TrustedAdvisor( + {'foo': mock_svc}, + account_id='aid', account_role='role', region='r' + ) assert cls.conn is None assert cls.account_id == 'aid' assert cls.account_role == 'role' @@ -96,10 +103,13 @@ def test_init_sts(self): assert cls.external_id is None assert cls.mfa_serial_number is None assert cls.mfa_token is None + assert cls.all_services == {'foo': mock_svc} def test_init_sts_external_id(self): - cls = TrustedAdvisor(account_id='aid', account_role='role', region='r', - external_id='myeid') + cls = TrustedAdvisor( + {}, account_id='aid', account_role='role', region='r', + external_id='myeid' + ) assert cls.conn is None assert cls.account_id == 'aid' assert cls.account_role == 'role' @@ -116,11 +126,11 @@ def test_update_limits(self): with patch('%s._update_services' % pb, autospec=True) as mock_update_services: mock_poll.return_value = mock_results - self.cls.update_limits(self.services) + self.cls.update_limits() assert mock_connect.mock_calls == [call(self.cls)] assert mock_poll.mock_calls == [call(self.cls)] assert mock_update_services.mock_calls == [ - call(self.cls, mock_results, self.services) + call(self.cls, mock_results) ] def test_get_limit_check_id(self): @@ -427,19 +437,21 @@ def test_poll_dont_have_ta(self): assert res == {} def test_update_services(self): - - def se_set(lname, val): - if lname == 'blam': - raise ValueError("foo") - - mock_autoscale = Mock(spec_set=_AwsService) - mock_ec2 = Mock(spec_set=_AwsService) - mock_ec2._set_ta_limit.side_effect = se_set - mock_vpc = Mock(spec_set=_AwsService) - services = { - 'AutoScaling': mock_autoscale, - 'EC2': mock_ec2, - 'VPC': mock_vpc, + mock_as_foo = Mock(spec_set=AwsLimit) + mock_as_bar = Mock(spec_set=AwsLimit) + mock_ec2_baz = Mock(spec_set=AwsLimit) + mock_vpc = Mock(spec_set=AwsLimit) + ta_services = { + 'AutoScaling': { + 'foo': mock_as_foo, + 'bar': mock_as_bar + }, + 'EC2': { + 'baz': mock_ec2_baz + }, + 'VPC': { + 'VPC Elastic IP addresses (EIPs)': mock_vpc + }, } ta_results = { 'AutoScaling': { @@ -459,7 +471,8 @@ def se_set(lname, val): } with patch('awslimitchecker.trustedadvisor' '.logger', autospec=True) as mock_logger: - self.cls._update_services(ta_results, services) + self.cls.ta_services = ta_services + self.cls._update_services(ta_results) assert mock_logger.mock_calls == [ call.debug("Updating TA limits on all services"), call.info("TrustedAdvisor returned check results for unknown " @@ -468,47 +481,64 @@ def se_set(lname, val): "service '%s'", 'OtherService'), call.info("Done updating TA limits on all services"), ] - assert mock_autoscale.mock_calls == [ - call._set_ta_limit('bar', 40), - call._set_ta_limit('foo', 20), + assert mock_as_foo.mock_calls == [ + call._set_ta_limit(20) + ] + assert mock_as_bar.mock_calls == [ + call._set_ta_limit(40) + ] + assert mock_ec2_baz.mock_calls == [ + call._set_ta_limit(5) ] - assert mock_ec2.mock_calls == [ - call._set_ta_limit('baz', 5), - call._set_ta_limit('blam', 10), - call._set_ta_limit('VPC Elastic IP addresses (EIPs)', 11) + assert mock_vpc.mock_calls == [ + call._set_ta_limit(11) ] - def test_update_services_no_ec2(self): + def test_make_ta_service_dict(self): + mock_ec2 = Mock(spec_set=_AwsService) + mock_el1 = Mock(spec_set=AwsLimit) + type(mock_el1).name = 'el1' + type(mock_el1).ta_service_name = 'EC2' + type(mock_el1).ta_limit_name = 'el1' + mock_el2 = Mock(spec_set=AwsLimit) + type(mock_el2).name = 'el2' + type(mock_el2).ta_service_name = 'Foo' + type(mock_el2).ta_limit_name = 'el2' + mock_ec2.get_limits.return_value = { + 'mock_el1': mock_el1, + 'mock_el2': mock_el2 + } - mock_autoscale = Mock(spec_set=_AwsService) mock_vpc = Mock(spec_set=_AwsService) - services = { - 'AutoScaling': mock_autoscale, - 'VPC': mock_vpc, + mock_vl1 = Mock(spec_set=AwsLimit) + type(mock_vl1).name = 'vl1' + type(mock_vl1).ta_service_name = 'VPC' + type(mock_vl1).ta_limit_name = 'other name' + mock_vl2 = Mock(spec_set=AwsLimit) + type(mock_vl2).name = 'vl2' + type(mock_vl2).ta_service_name = 'Foo' + type(mock_vl2).ta_limit_name = 'other limit' + mock_vpc.get_limits.return_value = { + 'mock_vl1': mock_vl1, + 'mock_vl2': mock_vl2 } - ta_results = { - 'AutoScaling': { - 'foo': 20, - 'bar': 40, - }, + + svcs = { + 'EC2': mock_ec2, + 'VPC': mock_vpc + } + + expected = { 'EC2': { - 'baz': 5, + 'el1': mock_el1 }, 'VPC': { - 'VPC Elastic IP addresses (EIPs)': 11, + 'other name': mock_vl1 + }, + 'Foo': { + 'el2': mock_el2, + 'other limit': mock_vl2 } } - with patch('awslimitchecker.trustedadvisor' - '.logger', autospec=True) as mock_logger: - self.cls._update_services(ta_results, services) - assert mock_logger.mock_calls == [ - call.debug("Updating TA limits on all services"), - call.info("TrustedAdvisor returned check results for unknown " - "service '%s'", 'EC2'), - call.info("Done updating TA limits on all services"), - ] - assert mock_autoscale.mock_calls == [ - call._set_ta_limit('bar', 40), - call._set_ta_limit('foo', 20), - ] - assert mock_vpc.mock_calls == [] + self.cls.all_services = svcs + assert self.cls._make_ta_service_dict() == expected diff --git a/awslimitchecker/trustedadvisor.py b/awslimitchecker/trustedadvisor.py index b1d8e852..f693ac9c 100644 --- a/awslimitchecker/trustedadvisor.py +++ b/awslimitchecker/trustedadvisor.py @@ -55,11 +55,15 @@ class TrustedAdvisor(Connectable): service_name = 'TrustedAdvisor' api_name = 'support' - def __init__(self, account_id=None, account_role=None, region=None, - external_id=None, mfa_serial_number=None, mfa_token=None): + def __init__(self, all_services, account_id=None, account_role=None, + region=None, external_id=None, mfa_serial_number=None, + mfa_token=None): """ Class to contain all TrustedAdvisor-related logic. + :param all_services: :py:class:`~.checker.AwsLimitChecker` ``services`` + dictionary. + :type all_services: dict :param account_id: `AWS Account ID `_ (12-digit string, currently numeric) for the account to connect to @@ -92,8 +96,10 @@ def __init__(self, account_id=None, account_role=None, region=None, self.external_id = external_id self.mfa_serial_number = mfa_serial_number self.mfa_token = mfa_token + self.all_services = all_services + self.ta_services = self._make_ta_service_dict() - def update_limits(self, services): + def update_limits(self): """ Poll 'Service Limits' check results from Trusted Advisor, if possible. Iterate over all :py:class:`~.AwsLimit` objects for the given services @@ -105,7 +111,7 @@ def update_limits(self, services): """ self.connect() ta_results = self._poll() - self._update_services(ta_results, services) + self._update_services(ta_results) def _poll(self): """ @@ -190,7 +196,7 @@ def _get_limit_check_id(self): "name 'Service Limits'.") return (None, None) - def _update_services(self, ta_results, services): + def _update_services(self, ta_results): """ Given a dict of TrustedAdvisor check results from :py:meth:`~._poll` and a dict of Service objects passed in to :py:meth:`~.update_limits`, @@ -203,30 +209,35 @@ def _update_services(self, ta_results, services): """ logger.debug("Updating TA limits on all services") for svc_name in sorted(ta_results.keys()): - limits = ta_results[svc_name] - if svc_name not in services: + svc_results = ta_results[svc_name] + if svc_name not in self.ta_services: logger.info("TrustedAdvisor returned check results for " "unknown service '%s'", svc_name) continue - service = services[svc_name] - for lim_name in sorted(limits.keys()): - try: - # @TODO - if we have ANY MORE special cases, we need a - # better way of handling this - maybe with a mapping - if svc_name == 'VPC' and lim_name == 'VPC Elastic IP ' \ - 'addresses (EIPs)': - # this limit belongs under EC2, but if we're only - # checking a specific list of services that doesn't - # include EC2, we don't want to set it. - if 'EC2' in services: - services['EC2']._set_ta_limit( - lim_name, limits[lim_name] - ) - else: - service._set_ta_limit(lim_name, limits[lim_name]) - except ValueError: + svc_limits = self.ta_services[svc_name] + for lim_name in sorted(svc_results): + if lim_name not in svc_limits: logger.info("TrustedAdvisor returned check results for " "unknown limit '%s' (service %s)", lim_name, svc_name) + continue + svc_limits[lim_name]._set_ta_limit(svc_results[lim_name]) logger.info("Done updating TA limits on all services") + + def _make_ta_service_dict(self): + """ + Build our service and limits dict. This is laid out identical to + ``self.all_services``, but keys limits by their ``ta_service_name`` + and ``ta_limit_name`` properties. + + :return: dict of TA service names to TA limit names to AwsLimit objects. + """ + res = {} + for svc_name in self.all_services: + svc_obj = self.all_services[svc_name] + for lim_name, lim in svc_obj.get_limits().items(): + if lim.ta_service_name not in res: + res[lim.ta_service_name] = {} + res[lim.ta_service_name][lim.ta_limit_name] = lim + return res diff --git a/awslimitchecker/version.py b/awslimitchecker/version.py index 3bd3c085..ac4e3c28 100644 --- a/awslimitchecker/version.py +++ b/awslimitchecker/version.py @@ -42,7 +42,7 @@ import logging logger = logging.getLogger(__name__) -_VERSION = '0.3.0' +_VERSION = '0.3.1' _PROJECT_URL = 'https://github.com/jantman/awslimitchecker' diff --git a/docs/source/conf.py b/docs/source/conf.py index 3f5d640c..9af1a090 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -14,9 +14,10 @@ import sys import os +import re # to let sphinx find the actual source... sys.path.insert(0, os.path.abspath("../..")) -from awslimitchecker.version import _get_version_info +from awslimitchecker.version import _get_version_info, _VERSION import sphinx.environment from docutils.utils import get_source_line @@ -25,6 +26,16 @@ # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) +is_rtd = os.environ.get('READTHEDOCS', None) != 'True' +readthedocs_version = os.environ.get('READTHEDOCS_VERSION', '') + +rtd_version = _get_version_info().version_str + +if (readthedocs_version in ['stable', 'latest', 'master'] or + re.match(r'^\d+\.\d+\.\d+', readthedocs_version)): + # this is a tag or stable/latest/master; show the actual version + rtd_version = _VERSION + # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. @@ -57,7 +68,7 @@ # General information about the project. project = u'awslimitchecker' -copyright = u'2015, Jason Antman' +copyright = u'2015, 2016 Jason Antman' author = u'Jason Antman' # The version info for the project you're documenting, acts as replacement for @@ -65,7 +76,7 @@ # built documents. # # The short X.Y version. -version = _get_version_info().version_str +version = rtd_version # The full version, including alpha/beta/rc tags. release = version @@ -116,7 +127,7 @@ # -- Options for HTML output ---------------------------------------------- -if os.environ.get('READTHEDOCS', None) != 'True': +if is_rtd: import sphinx_rtd_theme html_theme = 'sphinx_rtd_theme' html_theme_path = [ @@ -131,7 +142,7 @@ # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". -html_title = 'awslimitchecker v{v}'.format(v=version) +html_title = 'v{v} - check and report on AWS service usage and limits'.format(v=version) # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied diff --git a/docs/source/development.rst b/docs/source/development.rst index 998deb8c..562bf888 100644 --- a/docs/source/development.rst +++ b/docs/source/development.rst @@ -3,17 +3,39 @@ Development =========== +Any and all contributions to awslimitchecker are welcome. Guidelines for submitting +code contributions in the form of pull requests on `GitHub `_ +can be found below. For guidelines on submitting bug reports or feature requests, +please see the :ref:`Getting Help ` documentation. +For any contributions that don't fall into the above categories, please open an issue +for further assistance. + .. _development.pull_requests: Pull Requests ------------- +.. NOTE: be sure to update .github/PULL_REQUEST_TEMPLATE.md when changing this + Please cut all pull requests against the "develop" branch. I'll do my best to merge them as quickly as possible. If they pass all unit tests and have 100% coverage, it'll certainly be easier. I work on this project only in my personal time, so I can't always get things merged as quickly as I'd like. That being said, I'm committed to doing my best, and please call me out on it if you feel like I'm not. +.. _development.pull_request_guidelines: + +Pull Request Guidelines ++++++++++++++++++++++++ + +* Code should conform to the :ref:`Guidelines ` below. +* If you have difficulty writing tests for the code, feel free to ask for help or + submit the PR without tests. This will increase the amount of time it takes to + get merged, but I'd rather write tests for your code than write all the code myself. +* If you make changes to the ``versioncheck`` code, be sure to locally run the + ``-versioncheck`` tox tests. +* You've rebuilt the documentation using ``tox -e docs`` + .. _development.installing: Installing for Development @@ -41,14 +63,16 @@ To setup awslimitchecker for development: 4. Check out a new git branch. If you're working on a GitHub issue you opened, your branch should be called "issues/N" where N is the issue number. - .. _development.guidelines: Guidelines ---------- +.. NOTE: be sure to update .github/PULL_REQUEST_TEMPLATE.md when changing this + * pep8 compliant with some exceptions (see pytest.ini) * 100% test coverage with pytest (with valid tests) +* Complete, correctly-formatted documentation for all classes, functions and methods. * Connections to the AWS services should only be made by the class's :py:meth:`~awslimitchecker.connectable.Connectable.connect` and :py:meth:`~awslimitchecker.connectable.Connectable.connect_resource` methods, @@ -76,7 +100,9 @@ include 'NextToken' or another pagination marker, should be called through :py:func:`~awslimitchecker.utils.paginate_dict` with the appropriate parameters. 1. Add a new :py:class:`~.AwsLimit` instance to the return value of the - Service class's :py:meth:`~._AwsService.get_limits` method. + Service class's :py:meth:`~._AwsService.get_limits` method. If Trusted Advisor + returns data for this limit, be sure the service and limit names match those + returned by Trusted Advisor. 2. In the Service class's :py:meth:`~._AwsService.find_usage` method (or a method called by that, in the case of large or complex services), get the usage information via ``self.conn`` and/or ``self.resource_conn`` and pass it to the appropriate AwsLimit object via its @@ -89,6 +115,12 @@ include 'NextToken' or another pagination marker, should be called through class's ``_update_limits_from_api()`` method. 4. Ensure complete test coverage for the above. +In cases where the AWS service API has a different name than what is reported +by Trusted Advisor, or legacy cases where Trusted Advisor support is retroactively +added to a limit already in awslimitchecker, you must pass the +``ta_service_name`` and ``ta_limit_name`` parameters to the :py:class:`~.AwsLimit` +constructor, specifying the string values that are returned by Trusted Advisor. + .. _development.adding_services: Adding New Services @@ -235,6 +267,8 @@ work needed. See the guidelines below for information. Handling Issues and PRs ----------------------- +.. NOTE: be sure to update .github/PULL_REQUEST_TEMPLATE.md when changing this + All PRs and new work should be based off of the ``develop`` branch. PRs can be merged if they look good, and ``CHANGES.rst`` updated after the merge. @@ -269,11 +303,11 @@ Release Checklist 8. Confirm that README.rst renders correctly on GitHub. 9. Upload package to testpypi, confirm that README.rst renders correctly. - * Make sure your ~/.pypirc file is correct + * Make sure your ~/.pypirc file is correct (a repo called ``test`` for https://testpypi.python.org/pypi). * ``rm -Rf dist`` * ``python setup.py register -r https://testpypi.python.org/pypi`` * ``python setup.py sdist bdist_wheel`` - * ``twine upload -r https://testpypi.python.org/pypi dist/*`` + * ``twine upload -r test dist/*`` * Check that the README renders at https://testpypi.python.org/pypi/awslimitchecker 10. Create a pull request for the release to be merge into master. Upon successful Travis build, merge it. diff --git a/docs/source/getting_help.rst b/docs/source/getting_help.rst index 37c1d5d5..ea6887dd 100644 --- a/docs/source/getting_help.rst +++ b/docs/source/getting_help.rst @@ -4,21 +4,76 @@ Getting Help ============= +.. _getting_help.paid_support: + +Enterprise Support Agreements and Contract Development +------------------------------------------------------- + +For Commercial or Enterprise support agreements for awslimitchecker, +or for paid as-needed feature development or bug fixes, please contact Jason +Antman at jason@jasonantman.com. + .. _getting_help.reporting_bugs_and_questions: Reporting Bugs and Asking Questions ------------------------------------ +.. NOTE: be sure to update .github/ISSUE_TEMPLATE.md when changing this + Questions, bug reports and feature requests are happily accepted via the `GitHub Issue Tracker `_. -Pull requests are welcome. Issues that don't have an accompanying pull request -will be worked on as my time and priority allows, but I'll do my best to +Pull requests are welcome; see the :ref:`development` documentation for information +on PRs. Issues that don't have an accompanying pull request +will be worked on as my time and priority allows, and I'll do my best to complete feature requests as quickly as possible. Please take into account that I work on this project solely in my personal time, I don't get paid to work on it and I can't work on it for my day job, so there may be some delay in getting things implemented. -IRC ----- +.. _getting_help.guidelines_for_reporting_issues: + +Guidelines for Reporting Issues +------------------------------- + +Opening a `new issue on GitHub `_ +should pre-populate the issue description with a template of the following: + +.. _getting_help.feature_requests: + +Feature Requests +++++++++++++++++ + +If your feature request is for support of a service or limit not currently +supported by awslimitchecker, you can simply title the issue ``add support for +`` and add a simple description. +For anything else, please follow these guidelines: + +1. Describe in detail the feature you would like to see implemented, especially + how it would work from a user perspective and what benefits it adds. Your description + should be detailed enough to be used to determine if code written for the feature + adequately solves the problem. +2. Describe one or more use cases for why this feature will be useful. +3. Indicate whether or not you will be able to assist in testing pre-release + code for the feature. + +.. _getting_help.bug_reports: + +Bug Reports ++++++++++++ + +When reporting a bug in awslimitchecker, please provide all of the following information, +as well as any additional details that may be useful in reproducing or fixing +the issue: -I can often be found on `freenode `_ as "jantman". +1. awslimitchecker version, as reported by ``awslimitchecker --version``. +2. How was awslimitchecker installed (provide as much detail as possible, ideally + the exact command used and whether it was installed in a virtualenv or not). +3. The output of ``python --version`` and ``virtualenv --version`` in the environment + that awslimitchecker is running in. +4. Your operating system type and version. +5. The output of awslimitchecker, run with the ``-vv`` (debug-level output) flag + that shows the issue. +6. The output that you expected (what's wrong). +7. If the bug/issue is related to TrustedAdvisor, which support contract your account has. +8. Whether or not you are willing and able to assist in testing pre-release code + intended to fix the issue. diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst index a37ee5f4..0217f2c9 100644 --- a/docs/source/getting_started.rst +++ b/docs/source/getting_started.rst @@ -101,6 +101,15 @@ In addition, when assuming a role STS, you can use a `MFA device `_ +and `GetSessionToken `_ +API calls will throw errors when trying to access the IAM API (except for Session tokens, which will +work for IAM API calls only if an MFA token is used). Furthermore, Federation tokens cannot make use +of the STS AssumeRole functionality. If you attempt to use awslimitchecker with credentials generated +by these APIs (commonly used by organizations to hand out limited-lifetime credentials), you will likely +encounter errors. + .. _getting_started.regions: Regions diff --git a/docs/source/index.rst b/docs/source/index.rst index fca5e851..d2761a97 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,7 +1,5 @@ -.. awslimitchecker documentation master file, created by - sphinx-quickstart on Sat Jun 6 16:12:56 2015. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. +.. meta:: + :description: A script and python module to check your AWS service limits and usage, and warn when usage approaches limits. awslimitchecker ================ @@ -62,7 +60,7 @@ Develop: :target: https://readthedocs.org/projects/awslimitchecker/?badge=develop :alt: sphinx documentation for develop branch -A script and python module to check your AWS service limits and usage using `boto `_. +A script and python module to check your AWS service limits and usage, and warn when usage approaches limits. Users building out scalable services in Amazon AWS often run into AWS' `service limits `_ - often at the least convenient time (i.e. mid-deploy or when autoscaling fails). Amazon's `Trusted Advisor `_ @@ -111,6 +109,10 @@ Getting Help and Asking Questions See :ref:`getting_help`. +For paid support and development options, please see the +:ref:`Enterprise Support Agreements and Contract Development ` +section of the documentation. + Changelog --------- @@ -127,11 +129,10 @@ Contents Python Usage Required IAM Permissions Supported Limits + Getting Help Development Internals API - Getting Help - Indices and tables ================== diff --git a/docs/source/internals.rst b/docs/source/internals.rst index b443ee7e..5ddcb0e7 100644 --- a/docs/source/internals.rst +++ b/docs/source/internals.rst @@ -46,7 +46,7 @@ When :py:class:`~awslimitchecker.checker.AwsLimitChecker` is initialized, it als ``use_ta == True`` (the default), :py:meth:`~.TrustedAdvisor.update_limits` is called on the TrustedAdvisor instance. -:py:meth:`~.TrustedAdvisor.update_limits` polls Trusted Advisor data is from the Support API via +:py:meth:`~.TrustedAdvisor.update_limits` polls Trusted Advisor data from the Support API via :py:meth:`~.TrustedAdvisor._poll`; this will retrieve the limits for all "flaggedResources" items in the ``Service Limits`` Trusted Advisor check result for the current AWS account. It then calls :py:meth:`~.TrustedAdvisor._update_services`, passing in the Trusted Advisor check results and the @@ -59,10 +59,17 @@ in the TA result and attempts to call the ``Service``'s :py:meth:`~._AwsService. If a matching Service is not found, or if ``_set_ta_limit`` raises a ValueError (matching Limit not found for that Service), an error is logged. -Using this methodology, no additional code is needed to support new/additional Trusted Advisor limit checks; -*so long as* the Service and Limit name strings match between the Trusted Advisor API response and their -corresponding :py:class:`~._AwsService` and :py:class:`~.AwsLimit` instances, the TA limits will be automatically -added to the corresponding ``AwsLimit``. +When :py:class:`~awslimitchecker.checker.AwsLimitChecker` initializes +:py:class:`~awslimitchecker.trustedadvisor.TrustedAdvisor`, it passes in the +``self.services`` dictionary of all services and limits. At initialization time, +:py:class:`~awslimitchecker.trustedadvisor.TrustedAdvisor` iterates all services +and limits, and builds a new dictionary mapping the limit objects by the return +values of their :py:meth:`~awslimitchecker.limit.AwsLimit.ta_service_name` +and :py:meth:`~awslimitchecker.limit.AwsLimit.ta_limit_name` properties. This +allows limits to override the Trusted Advisor service and limit name that their +data comes from. In the default case, their service and limit names will be used +as they are set in the awslimitchecker code, and limits which have matching +Trusted Advisor data will be automatically populated. Service API Limit Information ----------------------------- diff --git a/docs/source/limits.rst b/docs/source/limits.rst index a3a6ca19..2138160f 100644 --- a/docs/source/limits.rst +++ b/docs/source/limits.rst @@ -46,6 +46,54 @@ updated from Trusted Advisor: * Elastic IP addresses (EIPs) + * Running On-Demand c1.medium instances + + * Running On-Demand c3.2xlarge instances + + * Running On-Demand c3.4xlarge instances + + * Running On-Demand c3.large instances + + * Running On-Demand c3.xlarge instances + + * Running On-Demand c4.2xlarge instances + + * Running On-Demand c4.large instances + + * Running On-Demand i2.2xlarge instances + + * Running On-Demand m1.medium instances + + * Running On-Demand m1.small instances + + * Running On-Demand m3.2xlarge instances + + * Running On-Demand m3.large instances + + * Running On-Demand m3.medium instances + + * Running On-Demand m3.xlarge instances + + * Running On-Demand m4.4xlarge instances + + * Running On-Demand m4.large instances + + * Running On-Demand m4.xlarge instances + + * Running On-Demand r3.2xlarge instances + + * Running On-Demand r3.4xlarge instances + + * Running On-Demand r3.large instances + + * Running On-Demand t1.micro instances + + * Running On-Demand t2.medium instances + + * Running On-Demand t2.micro instances + + * Running On-Demand t2.small instances + * VPC Elastic IP addresses (EIPs) * ELB @@ -145,17 +193,17 @@ Limit Default Elastic IP addresses (EIPs) :sup:`(TA)` :sup:`(API)` 5 Rules per VPC security group 50 Running On-Demand EC2 instances :sup:`(API)` 20 -Running On-Demand c1.medium instances 20 +Running On-Demand c1.medium instances :sup:`(TA)` 20 Running On-Demand c1.xlarge instances 20 -Running On-Demand c3.2xlarge instances 20 -Running On-Demand c3.4xlarge instances 20 +Running On-Demand c3.2xlarge instances :sup:`(TA)` 20 +Running On-Demand c3.4xlarge instances :sup:`(TA)` 20 Running On-Demand c3.8xlarge instances 20 -Running On-Demand c3.large instances 20 -Running On-Demand c3.xlarge instances 20 -Running On-Demand c4.2xlarge instances 20 +Running On-Demand c3.large instances :sup:`(TA)` 20 +Running On-Demand c3.xlarge instances :sup:`(TA)` 20 +Running On-Demand c4.2xlarge instances :sup:`(TA)` 20 Running On-Demand c4.4xlarge instances 10 Running On-Demand c4.8xlarge instances 5 -Running On-Demand c4.large instances 20 +Running On-Demand c4.large instances :sup:`(TA)` 20 Running On-Demand c4.xlarge instances 20 Running On-Demand cc2.8xlarge instances 20 Running On-Demand cg1.4xlarge instances 2 @@ -168,36 +216,37 @@ Running On-Demand g2.2xlarge instances 5 Running On-Demand g2.8xlarge instances 2 Running On-Demand hi1.4xlarge instances 2 Running On-Demand hs1.8xlarge instances 2 -Running On-Demand i2.2xlarge instances 8 +Running On-Demand i2.2xlarge instances :sup:`(TA)` 8 Running On-Demand i2.4xlarge instances 4 Running On-Demand i2.8xlarge instances 2 Running On-Demand i2.xlarge instances 8 Running On-Demand m1.large instances 20 -Running On-Demand m1.medium instances 20 -Running On-Demand m1.small instances 20 +Running On-Demand m1.medium instances :sup:`(TA)` 20 +Running On-Demand m1.small instances :sup:`(TA)` 20 Running On-Demand m1.xlarge instances 20 Running On-Demand m2.2xlarge instances 20 Running On-Demand m2.4xlarge instances 20 Running On-Demand m2.xlarge instances 20 -Running On-Demand m3.2xlarge instances 20 -Running On-Demand m3.large instances 20 -Running On-Demand m3.medium instances 20 -Running On-Demand m3.xlarge instances 20 +Running On-Demand m3.2xlarge instances :sup:`(TA)` 20 +Running On-Demand m3.large instances :sup:`(TA)` 20 +Running On-Demand m3.medium instances :sup:`(TA)` 20 +Running On-Demand m3.xlarge instances :sup:`(TA)` 20 +Running On-Demand m4.10xlarge instances 5 Running On-Demand m4.2xlarge instances 20 -Running On-Demand m4.4xlarge instances 20 -Running On-Demand m4.8xlarge instances 20 -Running On-Demand m4.large instances 20 -Running On-Demand m4.xlarge instances 20 -Running On-Demand r3.2xlarge instances 20 -Running On-Demand r3.4xlarge instances 10 +Running On-Demand m4.4xlarge instances :sup:`(TA)` 10 +Running On-Demand m4.large instances :sup:`(TA)` 20 +Running On-Demand m4.xlarge instances :sup:`(TA)` 20 +Running On-Demand r3.2xlarge instances :sup:`(TA)` 20 +Running On-Demand r3.4xlarge instances :sup:`(TA)` 10 Running On-Demand r3.8xlarge instances 5 -Running On-Demand r3.large instances 20 +Running On-Demand r3.large instances :sup:`(TA)` 20 Running On-Demand r3.xlarge instances 20 -Running On-Demand t1.micro instances 20 +Running On-Demand t1.micro instances :sup:`(TA)` 20 Running On-Demand t2.large instances 20 -Running On-Demand t2.medium instances 20 -Running On-Demand t2.micro instances 20 -Running On-Demand t2.small instances 20 +Running On-Demand t2.medium instances :sup:`(TA)` 20 +Running On-Demand t2.micro instances :sup:`(TA)` 20 +Running On-Demand t2.nano instances 20 +Running On-Demand t2.small instances :sup:`(TA)` 20 Security groups per VPC 100 VPC Elastic IP addresses (EIPs) :sup:`(TA)` :sup:`(API)` 5 VPC security groups per elastic network interface :sup:`(API)` 5 diff --git a/setup.py b/setup.py index 87a01d80..bafddf8e 100644 --- a/setup.py +++ b/setup.py @@ -83,7 +83,7 @@ awslimitchecker = awslimitchecker.runner:console_entry_point """, url=_PROJECT_URL, - description='A script and python module to check your AWS service limits and usage using boto.', + description='A script and python module to check your AWS service limits and usage, and warn when usage approaches limits.', long_description=long_description, install_requires=requires, keywords="AWS EC2 Amazon boto boto3 limits cloud", diff --git a/tox.ini b/tox.ini index 76af1efc..6848f860 100644 --- a/tox.ini +++ b/tox.ini @@ -174,7 +174,7 @@ recreate = True [testenv:docs] # this really just makes sure README.rst will parse on pypi -passenv = TRAVIS* CONTINUOUS_INTEGRATION AWS* +passenv = TRAVIS* CONTINUOUS_INTEGRATION AWS* READTHEDOCS* setenv = TOXINIDIR={toxinidir} TOXDISTDIR={distdir}