[WIP] Query service_meta info using DNS TXT query API

[sodre]

This is the service part of feature request #2709

• Update documentation
• Return results for TXT query
• Return results for ANY query
• Return results for recursive DNS queries
• Review by hashicorp (@slackpad or @magiconair)

Kudos: @pierresouchay for closing #3881

[sodre]

@rtbellows,
when you get a chance, can you tell me what DNS TXT lookups are needed to support the ACME use-case? I want to make sure they are captured in the unit tests.

[pierresouchay]

@sodre This is a very welcomed idea.

(I don't remember if TXT records can be compressed however

Note that current SRV records suffer from metadata inflation of TXT records that reduce the data returned by SRV records (while it might hopefully soon benefit from #4131). Having this API would allow us to split SRV meta and TXT records information. That would be great, because it would allow to return far more SRV records on large clusters.

Having this PR merged could allow us to add an option to avoid returning TXT records in SRV records and being more efficient (since most people probably do not care about TXT meta in SRV records, only A/AAAA, not all metadata)

[neurostream]

@sodre @pierresouchay Most of the following was in response to your earlier question about "what DNS TXT lookups are needed to support the ACME use-case?" But I'm primarily watching this in hopes of realizing so many additional uses cases for ServiceMeta data via the DNS TXT Consul interface - this will be very exciting! Thanks!

TL;DR : this DNS use case, with the rfc1035- bit used by Node Meta, could make single name certs under the ".service." Consul subdomain via ACME pretty straight-forward.

So, some additional thoughts here about an ACME use case that I've also hoped to work though:

As an example, consider a Consul domain is .hashicorp.com ( instead of .consul ) and a "www" service will be resolvable via the DNS name www.service.hashicorp.com.

In the ACME use case with Let's Encrypt , the DNS challenge asks the requestor to create a TXT record adjacent in the domain where "www" is resolved. The ACME service responds with what they would like to query DNS with ( they give a DNS record name and value) so one can prove that one "control" the domain.

The DNS TXT record VALUE they generate seems to be a meaningful hash of something.
The DNS TXT record NAME, in the interactions I've automated so far, must look like "_acme-challenge.[plus the SSL DNS CommonName]".

For example:
DNS RECORD TYPE : TXT
DNS RECORD NAME: _acme-challenge.www.service.hashicorp.com
DNS RECORD VALUE: "MTdGOEJDNDItRkU1Qi00OUZFLUE1OTctOTZFQzA3OUZCQjcwCg"

If Consul's Service+Tag+Meta feature is used when updating the Service registration - and, re-using the existing "Node Meta" implementation behavior of prepending "rfc1035-" to the Service Meta key ( https://www.consul.io/docs/agent/dns.html#node-lookups - so that the DNS record value only returns the MetaValue part, without the the **MetaKey=**MetaValue of the ServiceMeta); then, it seems feasible for this service registering itself as "www" to actually retrieve an SSL certificate for "www.service.hashicorp.com" from a source like Let'sEncrypt this way ( assuming the Consul DNS service is reachable from from the ACME DNS challenge nodes).

With that said, these possible caveats came to mind:

• Consul service registration operates under a 'service.' subdomain for a single service name at a time, so Certificates for other parts of the domain would have to be handled some other way.
  ( for example, outside of Consul DNS, there is a practice of putting an _acme-challenge[.mydomain.com] CNAME to another DNS zone that can be updated with other tools. )
• ACME DNS-02 might help with the Wild Card, zone-apex scenario, or other DNS CN names needed in the Certificate ( [no-subdomain].service.hashicorp.com and *.service.hashicorp.com ) - the still under the .service subdomain.
  • ACMEv1 DNS-01 being one domain at a time and not being able to register a Consul service called '[empty/null]' for zone-apex or a service called '*' for wildcard.

Anyhow, super excited about this ServiceMeta DNS TXT record feature for other uses as well!

Cheers.

-Alan

[hanshasselberg]

Hello, I just stumbled over this and I am wondering if there is still interest?! From a first look this sounds nice, but I haven't looked at all the implications yet.

[sodre]

@i0rek, I don't currently have time to work on the implementation of this PR 😞.

[hanshasselberg]

@sodre Thank you for getting back to me! Would you agree that we can close this PR? I am asking because I don't have time to do that either.

[ghost]

Hey there,

This issue has been automatically locked because it is closed and there hasn't been any activity for at least 30 days.

If you are still experiencing problems, or still have questions, feel free to open a new one 👍.