AWS Beijing region doesn't work.

[AnthonyWingerter]

Hello,

It appears when trying to connect to the AWS cn-north-1 region it still attempts to connect to ttps://support.us-east-1.amazonaws.com

Is there a way to change this behavior?

Thanks and regards,
-Anthony-

[root@av1c6ocvcu01 bin]# ./awslimitchecker --profile=cn --region cn-north-1 -vv
awslimitchecker 6.1.5 is AGPL-licensed free software; all users have a right to the full source code of this version. See https://github.com/jantman/awslimitchecker
2019-03-08 14:59:42,929 [DEBUG checker.py:206 - awslimitchecker.checker._boto_conn_kwargs() ] Using credentials profile: cn
2019-03-08 14:59:42,959 [DEBUG cloudtrail.py:116 - awslimitchecker.services.cloudtrail.get_limits() ] Gathering CloudTrail's limits from AWS
2019-03-08 14:59:43,100 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to dynamodb in region cn-north-1
2019-03-08 14:59:43,101 [DEBUG lambdafunc.py:82 - awslimitchecker.services.lambdafunc.get_limits() ] Getting limits for Lambda
2019-03-08 14:59:43,101 [DEBUG cloudtrail.py:116 - awslimitchecker.services.cloudtrail.get_limits() ] Gathering CloudTrail's limits from AWS
2019-03-08 14:59:43,102 [DEBUG lambdafunc.py:82 - awslimitchecker.services.lambdafunc.get_limits() ] Getting limits for Lambda
2019-03-08 14:59:43,112 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to support in region us-east-1
2019-03-08 14:59:43,113 [INFO trustedadvisor.py:162 - awslimitchecker.trustedadvisor._poll() ] Beginning TrustedAdvisor poll
2019-03-08 14:59:43,113 [DEBUG trustedadvisor.py:219 - awslimitchecker.trustedadvisor._get_limit_check_id() ] Querying Trusted Advisor checks
2019-03-08 14:59:43,115 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-08 14:59:43,115 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): support.us-east-1.amazonaws.com:443
2019-03-08 14:59:43,315 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://support.us-east-1.amazonaws.com:443 "POST / HTTP/1.1" 400 107
Traceback (most recent call last):
File "./awslimitchecker", line 11, in
sys.exit(console_entry_point())
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/runner.py", line 415, in console_entry_point
r.console_entry_point()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/runner.py", line 409, in console_entry_point
res = self.check_thresholds()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/runner.py", line 310, in check_thresholds
service=self.service_name)
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/checker.py", line 550, in check_thresholds
self.ta.update_limits()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/trustedadvisor.py", line 141, in update_limits
ta_results = self._poll()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/trustedadvisor.py", line 163, in _poll
tmp = self._get_limit_check_id()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/trustedadvisor.py", line 233, in _get_limit_check_id
raise ex
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the DescribeTrustedAdvisorChecks operation: The security token included in the request is invalid.

[jantman]

Anthony,

Hmm, I wasn't even aware of a China region, nor have I tested against it.

Per the Support API documentation, the Support API only has an endpoint in us-east-1.

If the China region(s) is isolated in some weird way that us-east-1 can't be reached, my only suggestion would be to disable Trusted Advisor support (--skip-ta) and then you'll need to explicitly specify your current service limit value for any limits that would normally be retrieved via TrustedAdvisor.

If anyone else has further information on this please feel free to comment, but for the time being I'm going to close this issue as the docs linked above make it very clear that the Support API only has one endpoint, in us-east-1, and the error pasted above makes it look like the new China regions work very differently from all other current/known regions (i.e. IAM is global, and creds that work in one region should work in all regions).

[bflad]

FYI, AWS partitions outside the standard ones usually have their own documentation. e.g.

• AWS China: https://www.amazonaws.cn/en/about-aws/regional-product-services/
• AWS GovCloud (US): https://docs.aws.amazon.com/govcloud-us/latest/ug-west/govcloud-ta.html

[AnthonyWingerter]

Here is some info I found.

https://docs.amazonaws.cn/en_us/aws/latest/userguide/awssupport.html

Feature Availability and Implementation Differences
The Amazon AWS implementation of AWS Support is unique in the following way:
• API access to AWS Trusted Advisor data via the AWS Support API is not supported.
https://docs.amazonaws.cn/en_us/aws/latest/userguide/awssupport.html

I tried the --skip-ta option and here are the results

[root@av1c6ocvcu01 bin]# ./awslimitchecker -u --profile=cn --region cn-north-1 --skip-ta -vv
awslimitchecker 6.1.5 is AGPL-licensed free software; all users have a right to the full source code of this version. See https://github.com/jantman/awslimitchecker
2019-03-11 11:06:12,267 [DEBUG checker.py:206 - awslimitchecker.checker._boto_conn_kwargs() ] Using credentials profile: cn
2019-03-11 11:06:12,286 [DEBUG cloudtrail.py:116 - awslimitchecker.services.cloudtrail.get_limits() ] Gathering CloudTrail's limits from AWS
2019-03-11 11:06:12,373 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to dynamodb in region cn-north-1
2019-03-11 11:06:12,374 [DEBUG lambdafunc.py:82 - awslimitchecker.services.lambdafunc.get_limits() ] Getting limits for Lambda
2019-03-11 11:06:12,374 [DEBUG cloudtrail.py:116 - awslimitchecker.services.cloudtrail.get_limits() ] Gathering CloudTrail's limits from AWS
2019-03-11 11:06:12,375 [DEBUG lambdafunc.py:82 - awslimitchecker.services.lambdafunc.get_limits() ] Getting limits for Lambda
2019-03-11 11:06:12,375 [DEBUG checker.py:354 - awslimitchecker.checker.find_usage() ] Finding usage for service: CloudTrail
2019-03-11 11:06:12,375 [DEBUG cloudtrail.py:61 - awslimitchecker.services.cloudtrail.find_usage() ] Checking usage for service CloudTrail
2019-03-11 11:06:12,387 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to cloudtrail in region cn-north-1
2019-03-11 11:06:12,389 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:12,389 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): cloudtrail.cn-north-1.amazonaws.com.cn:443
2019-03-11 11:06:14,402 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://cloudtrail.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 506
2019-03-11 11:06:14,406 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:38,257 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://cloudtrail.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 171
2019-03-11 11:06:38,258 [DEBUG cloudtrail.py:69 - awslimitchecker.services.cloudtrail.find_usage() ] Done checking usage.
2019-03-11 11:06:38,258 [DEBUG checker.py:354 - awslimitchecker.checker.find_usage() ] Finding usage for service: S3
2019-03-11 11:06:38,258 [DEBUG s3.py:60 - awslimitchecker.services.s3.find_usage() ] Checking usage for service S3
2019-03-11 11:06:38,325 [INFO connectable.py:108 - awslimitchecker.connectable.connect_resource() ] Connected to s3 (resource) in region cn-north-1
2019-03-11 11:06:38,331 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:38,331 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): s3.cn-north-1.amazonaws.com.cn:443
2019-03-11 11:06:42,582 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://s3.cn-north-1.amazonaws.com.cn:443 "GET / HTTP/1.1" 200 None
2019-03-11 11:06:42,600 [DEBUG s3.py:71 - awslimitchecker.services.s3.find_usage() ] Done checking usage.
2019-03-11 11:06:42,600 [DEBUG directoryservice.py:125 - awslimitchecker.services.directoryservice._update_limits_from_api() ] Setting DirectoryService limits from API
2019-03-11 11:06:42,624 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to ds in region cn-north-1
2019-03-11 11:06:42,626 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:42,626 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): ds.cn-north-1.amazonaws.com.cn:443
2019-03-11 11:06:44,061 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://ds.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 348
2019-03-11 11:06:44,064 [DEBUG checker.py:354 - awslimitchecker.checker.find_usage() ] Finding usage for service: Directory Service
2019-03-11 11:06:44,064 [DEBUG directoryservice.py:61 - awslimitchecker.services.directoryservice.find_usage() ] Checking usage for service Directory Service
2019-03-11 11:06:44,067 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:44,428 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://ds.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 348
2019-03-11 11:06:44,429 [DEBUG directoryservice.py:80 - awslimitchecker.services.directoryservice.find_usage() ] Done checking usage.
2019-03-11 11:06:44,452 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to autoscaling in region cn-north-1
2019-03-11 11:06:44,452 [INFO autoscaling.py:145 - awslimitchecker.services.autoscaling._update_limits_from_api() ] Querying EC2 DescribeAccountAttributes for limits
2019-03-11 11:06:44,454 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:44,454 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): autoscaling.cn-north-1.amazonaws.com.cn:443
2019-03-11 11:06:45,530 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://autoscaling.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 566
2019-03-11 11:06:45,531 [DEBUG checker.py:354 - awslimitchecker.checker.find_usage() ] Finding usage for service: AutoScaling
2019-03-11 11:06:45,531 [DEBUG autoscaling.py:61 - awslimitchecker.services.autoscaling.find_usage() ] Checking usage for service AutoScaling
2019-03-11 11:06:45,534 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:45,807 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://autoscaling.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 335
2019-03-11 11:06:45,810 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:46,092 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://autoscaling.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 350
2019-03-11 11:06:46,094 [DEBUG autoscaling.py:90 - awslimitchecker.services.autoscaling.find_usage() ] Done checking usage.
2019-03-11 11:06:46,094 [DEBUG cloudformation.py:107 - awslimitchecker.services.cloudformation._update_limits_from_api() ] Setting CloudFormation limits from API
2019-03-11 11:06:46,118 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to cloudformation in region cn-north-1
2019-03-11 11:06:46,120 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:46,120 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): cloudformation.cn-north-1.amazonaws.com.cn:443
2019-03-11 11:06:47,804 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://cloudformation.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 629
2019-03-11 11:06:47,806 [DEBUG cloudformation.py:115 - awslimitchecker.services.cloudformation._update_limits_from_api() ] API response contained unknown CloudFormation limit: StackOutputsLimit
2019-03-11 11:06:47,806 [DEBUG cloudformation.py:115 - awslimitchecker.services.cloudformation._update_limits_from_api() ] API response contained unknown CloudFormation limit: ConcurrentResourcesLimit
2019-03-11 11:06:47,806 [DEBUG checker.py:354 - awslimitchecker.checker.find_usage() ] Finding usage for service: CloudFormation
2019-03-11 11:06:47,806 [DEBUG cloudformation.py:63 - awslimitchecker.services.cloudformation.find_usage() ] Checking usage for service CloudFormation
2019-03-11 11:06:47,809 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:48,095 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://cloudformation.cn-north-1.amazonaws.com.cn:443 "POST / HTTP/1.1" 200 283
2019-03-11 11:06:48,097 [DEBUG cloudformation.py:78 - awslimitchecker.services.cloudformation.find_usage() ] Done checking usage.
2019-03-11 11:06:48,097 [DEBUG checker.py:354 - awslimitchecker.checker.find_usage() ] Finding usage for service: ApiGateway
2019-03-11 11:06:48,098 [DEBUG apigateway.py:61 - awslimitchecker.services.apigateway.find_usage() ] Checking usage for service ApiGateway
2019-03-11 11:06:48,142 [INFO connectable.py:90 - awslimitchecker.connectable.connect() ] Connected to apigateway in region cn-north-1
2019-03-11 11:06:48,142 [DEBUG apigateway.py:79 - awslimitchecker.services.apigateway._find_usage_apis() ] Finding usage for APIs
2019-03-11 11:06:48,145 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:48,145 [DEBUG connectionpool.py:813 - urllib3.connectionpool._new_conn() ] Starting new HTTPS connection (1): apigateway.cn-north-1.amazonaws.com.cn:443
2019-03-11 11:06:49,607 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://apigateway.cn-north-1.amazonaws.com.cn:443 "GET /restapis HTTP/1.1" 200 12
2019-03-11 11:06:49,608 [DEBUG apigateway.py:103 - awslimitchecker.services.apigateway._find_usage_apis() ] Found 0 APIs
2019-03-11 11:06:49,609 [DEBUG apigateway.py:106 - awslimitchecker.services.apigateway._find_usage_apis() ] Finding usage for per-API limits
2019-03-11 11:06:49,609 [DEBUG apigateway.py:157 - awslimitchecker.services.apigateway._find_usage_api_keys() ] Finding usage for API Keys
2019-03-11 11:06:49,611 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:49,951 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://apigateway.cn-north-1.amazonaws.com.cn:443 "GET /apikeys HTTP/1.1" 200 12
2019-03-11 11:06:49,952 [DEBUG apigateway.py:170 - awslimitchecker.services.apigateway._find_usage_certs() ] Finding usage for Client Certificates
2019-03-11 11:06:49,954 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:50,962 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://apigateway.cn-north-1.amazonaws.com.cn:443 "GET /clientcertificates HTTP/1.1" 200 12
2019-03-11 11:06:50,963 [DEBUG apigateway.py:196 - awslimitchecker.services.apigateway._find_usage_plans() ] Finding usage for Usage Plans
2019-03-11 11:06:50,965 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:51,314 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://apigateway.cn-north-1.amazonaws.com.cn:443 "GET /usageplans HTTP/1.1" 200 12
2019-03-11 11:06:51,315 [DEBUG apigateway.py:183 - awslimitchecker.services.apigateway._find_usage_vpc_links() ] Finding usage for VPC Links
2019-03-11 11:06:51,317 [DEBUG retry.py:210 - urllib3.util.retry.from_int() ] Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-03-11 11:06:51,643 [DEBUG connectionpool.py:393 - urllib3.connectionpool._make_request() ] https://apigateway.cn-north-1.amazonaws.com.cn:443 "GET /vpclinks HTTP/1.1" 400 59
Traceback (most recent call last):
File "./awslimitchecker", line 11, in
sys.exit(console_entry_point())
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/runner.py", line 415, in console_entry_point
r.console_entry_point()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/runner.py", line 405, in console_entry_point
self.show_usage()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/runner.py", line 256, in show_usage
service=self.service_name, use_ta=(not self.skip_ta))
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/checker.py", line 355, in find_usage
cls.find_usage()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/services/apigateway.py", line 69, in find_usage
self._find_usage_vpc_links()
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/awslimitchecker/services/apigateway.py", line 186, in _find_usage_vpc_links
for resp in paginator.paginate():
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/botocore/paginate.py", line 255, in iter
response = self._make_request(current_kwargs)
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/botocore/paginate.py", line 332, in _make_request
return self._method(**current_kwargs)
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/covisint/bin/limitchecker/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.BadRequestException: An error occurred (BadRequestException) when calling the GetVpcLinks operation: vpc link not supported for region cn-north-1

[jantman]

I guess I closed this rather prematurely given the information from @bflad - I've never worked with partitions outside of the standard ones and wasn't aware of this.

@AnthonyWingerter apologies for not following up on this sooner. So from that output, it looks like --skip-ta fixes the TA issue, but then there are additional problems with other services/API methods not supported in that region.

Does anyone have thoughts on how this should be handled? In the past I've seen that the errors raised when a service or method is not supported in a specific region aren't terribly similar, so catching and analyzing those errors requires a lot of code (and a lot of iterative development, which is hard since I don't have access to these partitions). On the other hand, I'm concerned that some hard-coded "skip these things in these regions" logic would be a maintenance nightmare. I'm certainly open to any opinions on how this could be handled...

[shivknight]

AWS just added a Support endpoint for Govcloud, https://support.us-gov-west-1.amazonaws.com. See https://aws.amazon.com/about-aws/whats-new/2019/08/aws-support-api-now-available-in-aws-govcloud/

[jantman]

Thanks to @djkiourtsis via #434, we now have support for assumed role partitions for STS, and alternate regions for use with Trusted Advisor (support API). This is currently only implemented at the python API level, but I'll be adding command-line arguments to match up with these new configuration options. I believe that will solve this issue.

[jantman]

@shivknight @AnthonyWingerter A tentative fix for this is present now in the develop branch, largely thanks to @djkiourtsis. It adds new keyword arguments to the AwsLimitChecker class for specifying the AWS partition and/or region to use for the support API (for Trusted Advisor), and also exposes those via new --ta-api-region and --role-partition command line options.

If any of you have the time to do a development installation of the develop branch and verify the fix, it would be greatly appreciated.

Regardless, this should be present in the next release. Many apologies for the delay on this.

[jantman]

This has been fixed in 8.0.0 which is now live on PyPI and Docker Hub. Apologies for any delays in getting this released.