[DNS mode] Cloudflare New API Tokens

[mxvin]

Hello,
Cloudflare just releasing new API Tokens that can specify each API key for it's usage (Access Permission), that more secure than using Global API key.

As stated on https://api.cloudflare.com/#getting-started-requests (section API Tokens) implementing new API is just dirt easy by sending header "Authorization: Bearer <token>" instead of Global API that using header X-AUTH-EMAIL: & X-AUTH-KEY:

It's quite possible for adding new variable on account.conf like CF_API_Tokens=<tokens> and make some logic on dns_cf.sh that can deal with both new API Tokens & Global API header payload.

Regards.

[Neilpang]

fixed. please try with the latest dev branch.

acme.sh --upgrade -b dev

[Berzerker]

I believe this needs to be based on Zone ID, rather than account ID. Running a general account ID query with a specific zone being allowed (with no other zones being allowed) gives you an error that the account cannot list zones since the default API call tries to list all the entire account's information when it only has access to a specific zone.

{"success":false,"errors":[{"code":0,"message":"Requires permission \"com.cloudflare.api.account.zone.list\" to list zones of the account"}],"messages":[],"result":null}%

Running this with a specific Zone ID allows permission to list that specific zone, but even permissions across all zones will allow you to list that specific zone as well.

Running the API call with .../client/v4/zones/<zoneid> for that specific zone rather than .../client/v4/zones?account.id=<accountid> allows for more granular permissioning without reduced functionality.

Also, this error just throws a generic "invalid domain" even though the problem was permissioning on the token.

[AllenBalaj]

I'm having the same issue and had to allow the API token access to all zones to get this to work. It's better than what we had before since you can still limit access to only Zone and DNS settings, but it would be more secure to limit access to only those zones for which acme.sh needs DNS editing capabilities. For instance, I manage multiple small businesses' domains and DNS through Cloudflare, and would not want an acme.sh instance in one domain to have editing capabilities on another.

I like @Berzerker's idea, but how would this work with multiple domains, e.g. aa.com and bb.com? Perhaps the Zone ID variable should be comprised of a delimited list of all zones corresponding to the domains we're issuing certs for?

[Berzerker]

Making multiple calls to add the txt records then running all of the checks after is probably the easiest way with keeping the granular permissioning the tokens allow.

[AllenBalaj]

Are you suggesting multiple separate calls to acme.sh for each domain? Wouldn't that necessitate updating the environment variables before each call? How then would saved parameters and cron functionality be maintained?

[Berzerker]

I’m saying the acme.sh script should make multiple calls based on multiple zone IDs and/or account IDs set.

I’m agreeing with you.

[AllenBalaj]

Got it! Thanks for clarifying.

[Berzerker]

@Neilpang I don't think this should be closed. As per the last few comments, this isn't working 100% based on the functionality of the API Tokens.

As suggested, this should be switched to a Zone ID vs Account ID API call, with multiple calls being made if there are multiple domains/zones in play. Having it be Account ID call-based doesn't support setting permissions for specific zones as the token would not have access to list all of the zones of the account.

[AllenBalaj]

@Neilpang I'd be happy to submit a pull request for the CF ZoneID functionality if you have a specific preference as to how you'd like to see this implemented. Or would you rather we open a new issue and address the new functionality there?

[Neilpang]

@AllenBalaj

Thanks, show me the PR first.

[bartico6]

I have filed a bug report to CloudFlare that /v4/zones requires a global/account-wide zone list permission (even though you'd expect it to be able to list the zones it has access to).
I was told it was forwarded to the appropriate team, so we'll see how it plays out

[Berzerker]

IMO the way it works is expected functionality with that kind of an API call. If you call a global zone call, you'd expect it to list all of the zones on the account. Having it list all zones it only has access to is a bit misleading on the API call itself.

This is just an integration issue on the acme.sh end.

[bartico6]

I'd argue otherwise; consider that /v4/zones lists all zones your account has access to, and not all zones registered on CloudFlare. Following that thought process, the actor calling the API can use /v4/zones to list the zones that specific actor can access; this theoretically should mean tokens calling /v4/zones only see the zones those specific tokens can access, just like account-wide api keys can view all zones belonging to that account.
This is especially true if you consider that you can restrict the token to view "all zones" belonging to all accounts you may access; or just zones from one specific account - and if you select the latter it works as intended. It's just zone subset whitelisting that causes an issue.

[Berzerker]

I'm confused by your first line of thought. This is a CloudFlare-specific API call, how would it call zones not registered on CloudFlare?

Regardless, I wouldn't call it a bug. This certainly seems like intended functionality; you're probably better off requesting functionality change rather than reporting it as a bug.

[bartico6]

The first line states that the /v4/zones api returns zones you can access rather than all zones on Cloudflare; this means that the api already filters the zones on their network to the ones your account can access; similarly, if authenticated with a token, it should filter to the zones the token can access. The fact it requires a global list permission to list some entries in a scoped api call sounds like an oversight, moreso than an intended functionality. Especially that a few times the API has returned an unhandled server error in some specific configurations.

[Neilpang]

This issue was automatically closed by my fix. I just reopened it, in case you have more ideas.

Thanks

[brackenhill-mob]

The wiki needs updating badly for non-uber techies like me.

Like how do I get the CF_Account_ID value? "xxxxxxxx" isn't wonderfully helpful ;)

Am I right in thinking that I need to use the CF call here https://api.cloudflare.com/#user-user-details

If not ...?

Thanks

[Berzerker]

1. You can run this command:

curl -X GET "https://api.cloudflare.com/client/v4/zones" -H "X-Auth-Email: <ACCTEMAIL>" -H "X-Auth-Key: <GLOBALAPIKEY>" -H "Content-Type: application/json"

You can also pipe it into python -mjson.tool for a cleaner output.

Look for the id key under account key. The only other key in that object is name (to help you identify the right one).

2. You can log into your cloudflare account and the account ID should appear as a 32-character string in the URL after dash.cloudflare.com/

3. The Account ID appears on the right column of clicking on any domain towards the bottom.

[nwmcsween]

So what permissions does acme.sh exactly need to do com.cloudflare.api.account.zone.list? I gave it zone, zone, read but it still has the same issue

[nwmcsween]

hmm apparently acme.sh needs full access and cannot work with the limited access to a specific zone e.g. zone resources = include, all zones vs a specific zone

[Berzerker]

It needs access to all zones, since it does a query that needs to list all zones, but this is still a ton better than using the global key which has full administrative access to your account.

[nwmcsween]

It's still sort of bad as it allows acme.sh to edit any dns setting for any domain vs editing specific domains that use acme.sh

[Berzerker]

Of course it could be better, but I’ll take this for now until I or someone else has time to redo this.

[niallfleming]

Surely if you specified the domain id in the initial issueance command or whatever, you could skip the all zone list that requires the higher perms?

[Ramblurr]

The current implementation offers nothing really over using the global api key :/

Since the domain id (called the "account id" in cloudflare terms) is defined along with the API key, there is no need for acme.sh to fetch lists of domains.

[Cherry]

I ran into this issue today, and found that if you assign the API token permission to "Account -> Account Settings: Read", this implicitly grants com.cloudflare.api.account.zone.list, allowing zones to be listed.

Allowing the token to read all account settings isn't ideal, but I think it's a better solution than giving it access to all zones on an account.

[typoworx-de]

I ran into the same issue still having problems to get it running. I did fresh install of acme.sh (trying to migrate from certbot to integrate Cloudflare DNS for cert issue). I'm running acme.sh version v2.8.5.

I'm already using this permissions stil having the "invalid domain" error running acme.sh:

Is there still something wrong or missing? I'm using API Token created for acme.sh and my global api key.

[ptts]

@typoworx-de
I was just using the same permissions just even slightly more restrictive:

Permissions
Account.Account Settings → Read
Zone.Zone → Read
Zone.DNS → Edit
Account Resources
Include → "My personal Account only"
Zone Resources
Include → "My zone only"

Everything worked flawlessly.

What do you mean with this?

I'm using API Token created for acme.sh and my global api key.

You should be using your API Token and your CF_Account_ID. The point of using API Tokens is not having to use the global API key...

EDIT 2020-03-25: I started having issues again. Maybe I did something different the first time or I was trying to renew instead of issue a new certificate. (see my new post here)

[bengalih]

Agreed this is still an issue. CF I don't think will be doing anything about it despite several requests:

https://community.cloudflare.com/t/bug-zone-detail-by-name-requires-zone-list-permission/128042
https://community.cloudflare.com/t/bug-in-list-zones-endpoint-when-using-api-token/115048

@ptts permissions above work for me and seem to be the least restrictive you can grant using the token API to get this to work.

At the very least this should be updated in the docs to prevent people from constantly having to research this from scratch. Going to put in PR with doc link changes.

[bengalih]

Apparently I didn't need to PR a request to the docs, they were public edit.
I have referenced this from there so hopefully less people have grief. If this is not OK, please revert.

[Neilpang]

@bengalih
Thanks for your info.

It would be better if we have a dedicated wiki page to demonstrate the usage in detail.

[artooro]

This patch allows you to put the Zone ID into the Account ID field, and use a CloudFlare API Key that is restricted to the zone and has no account level access, and it will work fine.

dns_cf.sh.patch.zip

@Neilpang would this be an acceptable patch for a pull request, or would it be preferred to add yet an additional field called Zone ID to be used in this scenario?

[Neilpang]

@artooro Please give me more detailed explanation about your change.
I'm not always on cloudflare API, your explanation will save me a lot of time to recall the details, which can also help your change get merged.

[artooro]

@Neilpang no problem.
So the current issue with dns_cf.sh is that it requires you to use a Cloudflare API key that has access to all the domains in your account. This is because it makes an API call to list the domains in the account to get the zone ID which is needed to make the API requests to actually add/remove the DNS TXT records.

Myself and others would like to restrict API keys to the specific domain zone. My patch allows this to happen by the user putting a Zone ID into the Account ID field, and dns_cf.sh will try to use it and soft-fail to Account ID.
A cleaner solution would be to add an additional field to dns_cf.sh called Zone ID, I repurposed Account ID so that I could patch it on pfSense and not have to patch the UI/config system as well.

If this is merged officially, I think doing it as a new field would be best, so I'll go ahead and do that and create a merge request for it.

[ptts]

I was trying to use @artooro 's fix like this:

docker run --rm -it \
	-e "CF_Token=<MY_CF_TOKEN>" \
	-e "CF_Account_ID=<MY_CF_ACCOUNT_ID>" \
	-e "CF_Zone_ID=<MY_CF_ZONE_ID>" \
	-v "$DIR/../config/acme.sh:/acme.sh" \
	neilpang/acme.sh:dev \
		--issue \
		--dns dns_cf \
		-d "*.example.com" \
		-d "example.com"

Yet I always get:

[Wed Mar 25 10:30:55 UTC 2020] Using stage ACME_DIRECTORY: https://acme-staging-v02.api.letsencrypt.org/directory
[Wed Mar 25 10:30:56 UTC 2020] Multi domain='DNS:*.example.com,DNS:example.com'
[Wed Mar 25 10:30:56 UTC 2020] Getting domain auth token for each domain
[Wed Mar 25 10:30:58 UTC 2020] Getting webroot for domain='*.example.com'
[Wed Mar 25 10:30:58 UTC 2020] Getting webroot for domain='example.com'
[Wed Mar 25 10:30:58 UTC 2020] Adding txt value: yf6MdbFKbNK7x6-iNp76mQv-nPtmSLYnvIZb1RRZ1kdH for domain:  _acme-challenge.example.com
[Wed Mar 25 10:31:00 UTC 2020] **invalid domain**
[Wed Mar 25 10:31:00 UTC 2020] **Error add txt for domain:_acme-challenge.example.com**
[Wed Mar 25 10:31:00 UTC 2020] Please add '--debug' or '--log' to check more details.
[Wed Mar 25 10:31:00 UTC 2020] See: https://github.com/Neilpang/acme.sh/wiki/How-to-debug-acme.sh

In the debug logs I can see that my provided CF_Zone_ID is never used and it thus can't find the domain. It starts by calling https://api.cloudflare.com/client/v4/zones?name=example.com&account.id=<MY_ACCOUNT_ID>, does not get a response (ret='0') and then tries to call https://api.cloudflare.com/client/v4/zones?name=com&account.id=<MY_ACCOUNT_ID>, again acme.sh does not receive a response and fails.

My token permissions are:

<MY_DOMAIN_ACCOUNT> - Account Settings:Read
<MY_DOMAIN_ZONE> - Zone:Read, DNS:Edit

When I change the token permissions to include All zones it works.

Anything else I can try?

[moto8801]

Are you running 2.8.6? This hasn't been released yet, you have to clone the repo.

[laoyur]

Hi, what if I use 2 tokens for 2 domains ?

TOKEN1: ZONE1 - READ, DNS - EDIT
TOKEN2: ZONE2 - READ, DNS - EDIT

certs issued with these tokens:

CF_Token="TOKEN1" CF_Zone_ID="ZONE1" acme.sh --issue --dns dns_cf -d "example1.com"
CF_Token="TOKEN2" CF_Zone_ID="ZONE2" acme.sh --issue --dns dns_cf -d "example2.com"

Then I noticed that SAVED_CF_Token in ~/.acme.sh/accounts.conf was overwrote to TOKEN2. So TOKEN1 was lost and the domain example1.com will fail to renew later ?

[Neilpang]

@laoyur
Multiple tokens in single account is not supported yet.

[colinmcintosh]

Looks like this is working on version 2.8.6 with the fix from @artooro.

Here is what I did to get everything working in case people are still having trouble.

# Generate a new token at https://dash.cloudflare.com/profile/api-tokens
# Create a custom token with these settings:
#     Permissions:
#         Zone - DNS - Edit
#     Zone Resources:
#         Include - Specific Zone - <YOUR_ZONE_NAME>
#     IP Address Filtering: optional
#     TTL: optional
export CF_Token=<YOUR_CF_TOKEN>

# Get your Zone ID from the sidebar on the homepage of your Cloudflare Dashboard
# Make sure you are using the 32 character alphanumeric ID that looks something like 81501ef88ef9b34f24450b63145d4019
export CF_Zone_ID=<YOUR_ZONE_ID>

~/.acme.sh/acme.sh --issue -d <YOUR_DOMAIN> --dns dns_cf

Further expanded example here: https://gist.github.com/colinmcintosh/016088860d35f01658e545b5ba75ba41

[laoyur]

Looks like this is working on version 2.8.6 with the fix from @artooro.

Here is what I did to get everything working in case people are still having trouble.

# Generate a new token at https://dash.cloudflare.com/profile/api-tokens
# Create a custom token with these settings:
#     Permissions:
#         Zone - DNS - Edit
#     Zone Resources:
#         Include - Specific Zone - <YOUR_ZONE_NAME>
#     IP Address Filtering: optional
#     TTL: optional
export CF_Token=<YOUR_CF_TOKEN>

# Get your Account ID and Zone ID from the sidebar on the homepage of your Cloudflare Dashboard
# Make sure you are using the 32 character alphanumeric IDs that looks something like 81501ef88ef9b34f24450b63145d4019
export CF_Account_ID=<YOUR_ACCOUNT_ID>
export CF_Zone_ID=<YOUR_ZONE_ID>

~/.acme.sh/acme.sh --issue -d <YOUR_DOMAIN> --dns dns_cf

Further expanded example here: https://gist.github.com/colinmcintosh/016088860d35f01658e545b5ba75ba41

I can confirm CF_Account_ID is not required (not secure in fact) in your case.

[colinmcintosh]

@laoyur awesome! Thanks for the tip! I just tried this and it worked without the Account ID. I'll update the gist and my snippet to reflect CF_Account_ID not being needed.

[lachesis]

Where (if anywhere) does acme.sh keep track of these variables per-domain for the renewal cron? If I were to issue several star-certs on the same machine, how would acme.sh know which Zone ID and CF token to use for each renewal?

[qnxor]

@lachesis I just saw your post. I asked the same here #3026

I guess that for now we need to write our own cron jobs specifying the right tokens in each.

It would be neat to add this functionality in acme.sh given many folks have multiple domains and will create separate tokens for each.

[qnxor]

I just created two certs for two domains using two separate API tokens. Then I deleted the CF_* entries from account.conf (which were for the 2nd domain as it overrode the 1st). I then tried a staging --cron and both domains were reported to have renewed fine ... i haven't read how letsencrypt does things, but is that because the info in domain.com/domain.com.conf is sufficient and the token,zone,account IDs are no longer required after successfully getting a certificate? In other words, will both be perpetually renewed if done within 90 days? I notice Le_Order* and Le_Link* in there ...

@Neilpang could you confirm?

One (related) suggestion: maybe chmod 600 the account.conf and domain.com.conf files given they contain auth data?

[Neilpang]

is that because the info in domain.com/domain.com.conf is sufficient and the token,zone,account IDs are no longer required after successfully getting a certificate?

No, the let'sencrypt server has cached the verification status for a few days.

[qnxor]

Continuing here from #3026

I thought that might be the case and i'm already using the --accountconf workaround. But that seems unnecessary given there is already a domain conf file in domain/domain.conf, so why not cache the token,zone,account IDs there instead? Seems like the natural thing to do given everything domain specific is stored there, rather than keeping a 2nd file that is domain specific. Will also avoid messing with custom and extra cronjobs.

I'll try to do a PR for that when i get the chance.

[KalleDK]

Continuing here from #3026

I thought that might be the case and i'm already using the --accountconf workaround. But that seems unnecessary given there is already a domain conf file in domain/domain.conf, so why not cache the token,zone,account IDs there instead? Seems like the natural thing to do given everything domain specific is stored there, rather than keeping a 2nd file that is domain specific. Will also avoid messing with custom and extra cronjobs.

I'll try to do a PR for that when i get the chance.

I've made a patch, because I had the same problem
https://gist.github.com/KalleDK/8ea5adbdf0f885495a04f2cb1c188292

Don't know if any of this is usefull for a PR

[sg1888]

Has anybody submitted a PR for this? This should definitely be fixed. As it stands, you are unable to generate certs for domains in different Cloudflare accounts (you can only create tokens for domains in the same account). This should be saved in the domain.conf file, as suggested above.

How have you all handled the issue thus far?