403 Client Error: Forbidden for url: - both cloudflare and rackspacecloud

[stevekirtley]

I use Lexicon to handle DNS challenges for LetsEncrypt.

I've come to renew some certificates tonight and ran my trusty shell script which has been working for a few years now.

I received errors from Rackspace cloud of:

403 Client Error: Forbidden for url: https://dns.api.rackspacecloud.com/v1.0/758427/domains?name=stektest.com

I debug as best I could and found a valid auth token was being received back but all requests to the v1 DNS api were failing.

I assumed that Rackspace may have been having issues or changed their API so moved that domain to CloudFlare.

I set-up Lexicon using the following environment variables:

export PROVIDER='cloudflare'
export LEXICON_CLOUDFLARE_AUTH_USERNAME='my@valid.email'
export LEXICON_CLOUDFLARE_AUTH_API_KEY='KEY FROM CLOUDFLARE SET UP WITH ALL ZONE READ ACCESS'
export LEXICON_CLOUDFLARE_ZONE_ID='ZONE ID FROM CLOUDFLARE'

I tried a basic call to:
lexicon cloudflare --log_level DEBUG list stektest.com A

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.cloudflare.com/client/v4/zones/xxxxxxx
(xxx being the valid zone ID from the variable above).

Is there a known problem with these two providers?

Am I missing something obvious?

[mattgauf]

I discovered the issue with Rackspace just this evening while troubleshooting a script. Rackspace returns 403 for GET requests that include a 'request body', even when the request is properly authenticated.

providers/rackspace.py runs the _authenticate() method, and on line 91 it calls _get() to return the domain's ID for use in later API calls.

_get() Is found in providers/base.py and in turn calls the _request() method (which provider scripts override, so calling _get() just calls the _request() method found in that provider's script)

My solution isn't the most elegant but for Rackspace, it proved to be the most reliable solution:

I reviewed the rackspace.py script and decided that I could safely override the _get() call within rackspace.py. Here is the new method (which I placed just above _request() in providers/rackspace.py

def _get(self, url="/", query_params=None):
    if query_params is None:
        query_params = {}
    LOGGER.debug(
        "request tenant ID: %s", self._get_rackspace_option("auth_account")
    )
    full_url = (
        f"{self.api_endpoint}/{self._get_rackspace_option('auth_account')}{url}"
    )
    response = requests.get(
        url=full_url,
        params=query_params,
        headers={
            "X-Auth-Token": self._get_rackspace_option("auth_token"),
            "Content-Type": "application/json",
        },
    )
    # if the request fails for any reason, throw an error.
    response.raise_for_status()
    return response.json()

Once I've had a chance to sleep and ponder a more elegant solution I'll likely make a pull request.

Best,
-- M

[stevekirtley]

Ahh thanks Matt,

I started looking at it with a fresh head this morning and realised CloudFlare was working just fine when I used the right API key - but switching back to Rackspace with this override for _get() also works for me.

Nice work.

Steve