-
-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
autohttps: Implement auto_https prefer_wildcard
option
#6146
Conversation
auto_https prefer_wildcard
option
Thanks Francis, this looks appealing. Will wait for one or two people to field-test it. |
This looks amazing and a super useful feature to reduce subdomain discovery through certificate transparency! I would suggest a small addition in order to minimise this even further. E.g. to make Caddy always use wildcards. This could maybe be implemented as another option for this directive, called Say I have two sites setup, for serviceA.domain.tld and serviceB.domain.tld, and use this setting, Caddy would then generate *.domain.tld and use it. If I have hostX.serviceA.domain.tld, hostY.serviceA.domain.tld, hostX.serviceB.domain.tld, hostY.serviceB.domain.tld, then all of those would be covered by the *.*.domain.tld certificate (does ACME even allow such certificates? 🤔). Alternatively, add another directive that allows users to specify a list of wildcard certs that will always be managed regardless of if they're used by sites or not, and then your feature here will be able to use those to avoid generating specific certs. A little less "magic" than the What do you think about this @francislavoie? I guess it might be necessary to give Caddy a list of domains we control in order for this to work when using multiple domains so it doesn't see domain1.tld and domain2.tld and tries to generate *.tld... 😂 |
@abjugard |
Right, unfortunate limitation but understandable. Then I think we can simplify the behaviour and not need to specify which domains are owned, but just let Caddy automatically figure out to make wildcards for the leftmost label. Perhaps we should just wait for this PR to get merged then I can take a crack at a |
I'm actually implementing part of this in CertMagic -- likely will become a "subject transformer" that allows changes to the subject name when managing certs, so, e.g. one can lob off the leftmost label and replace it with |
@francislavoie How would you feel if we/(I?) updated this PR to use the new SubjectTransformer introduced in the linked issue/commit? It should be relatively simple I think, if the global option is set, then set the SubjectTransformer for CertMagic configs to be a 3-line function. |
I'm not sure how that would look 🤔 maybe make a branch off this one if you think it's simple? |
I will try soon! :) |
Actually the SubjectTransformer might be slightly tangential to this, rather than directly related -- the transformer is for, like, "I have a.b.c and b.b.c, but I want you to manage a single wildcard instead of individual certs for each specific domain." Whereas this change is, "I have *.b.c and a.b.c, and I want a.b.c to use the wildcard cert." There's another aspect I want to consider as well, that is some users want just specific domains to be served under a wildcard, while the others shouldn't be. For example, in the config above, if there was also a site, I think I want to give this more thought, even before I implement anything here. This is a needed change though and I think it's a good start. I just don't want to commit to its API/syntax/implementation quite yet until we have a better picture of the bigger picture. |
I think that's already handled by my approach, because |
Ah, okay. Hmm. I might still wait on this until after 2.8 so I can give this a little more thought. We now have a way, in CertMagic, of mapping/transforming one subject name (domain name) to another, for the purposes of cert management. Even if this PR ends up being good as-is, I just want a little more time on it. |
Glad there is a PR for this and I really look forward to it. Thank you for your great work. One big advantage of having separate site blocks is when using Since these labels can be distributed across multiple docker-compose files, there can be some possibilities of misconfiguration somewhere (especially done by multiple people). With the current handle approach, if one of the subdomain is misconfigured, causes the ENTIRE wildcard site block to be removed, bringing down everything. With this PR, the failure would at least be localized (hopefully😊) |
962efa3
to
4baebcc
Compare
Could there not just be a It already seems to be an issue according to this report where an internal wildcard cert is being used instead of the LetsEncrypt one for an explicit site address. Alternatively, you could go the other way around like with Since the certmagic subject transformer feature is available and Caddy 2.8 is released, is there anything that can be done to assist moving this feature forward?
If you can provide a rough outline of what to test, I could put together configs to verify?
|
40d8022
to
c87b450
Compare
I am now wondering if we should make preferring wildcards the default behavior as @abjugard suggested above. In Slack, it was expressed that it would be a breaking change, but I am not sure if there are (m)any(?) use cases that would actually break. A wildcard cert is just as good as a subdomain cert. |
EDIT: Ignore me. I mistook the "prefer" wildcard cert to prefer provisioning an explicit subdomain with a wildcard cert (even if no wildcard was explicitly configured/requested).
Isn't it generally considered better practice to prefer explicit SAN for certs than a wildcard? Some businesses may have compliance requirements related to that expectation? That concern is only relevant if the private key was compromised for the attacker to leverage it before expiry/detection, but that may be sufficient time for the attacker? 🤷♂️ I don't operate at a scale where I'm paranoid about such personally, but I could understand it being a compliance concern elsewhere, which would make it a breaking change for those users? Thus you may want to introduce as opt-in, and switch to opt-out when a breaking change is acceptable? Unless you don't consider such implicit assumptions/trust in software to be a breaking change? You could also take the route of adding a notice (to release notes and pinned issue) for awareness before switching to opt-out, giving any potential users time to become aware of the change landing in future? To be fair though, I think when compliance matters that upgrading to newer releases would have a more strict policy than trusting semantic versioning 😅 So perhaps I'm rambling about a non-issue. |
If Caddy fails to obtain cert for *.example.com then foo.example.com won't have any cert too right? That seems problematic, especially because LetsEncrypt won't give you wildcard cert without DNS-01 challenge. |
The scenario you describe here isn't really a concern as the requirements for procuring a wildcard cert are strictly different from certs for specific subdomains. Caddy (I'm guessing) won't try to get a wildcard cert unless its able to, and if its able to then it will either succeed or fail in which case you've configured it wrong and it should correctly refuse to continue starting up the site in question. |
Wildcard certs require DNS challenges which have to be explicitly configured. If someone is configuring a wildcard into their server, then they probably don't intend to get subdomain certs at all. IMO.
Well, if we default to always preferring the wildcard cert, that changes this PR. I just want to do what is most expected and intuitive, and I feel like setting up wildcard hostnames is juuust explicit and involved enough that a user probably wants to use that cert for the rest of the subdomains. Maybe an inverse of this PR would be useful then, that is, to ignore wildcard certs. |
Well there is this report where someone does want some subdomains to not use wildcard. While the feature is named as prefer wildcard, I don't suppose there would be a way to ensure that only wildcards are provisioned when valid? As in if it fails, to opt-out of a fallback to explicit FQDN? This is due to CT public logs concern, which presently is avoided with a wildcard site address, but with this feature the benefit is explicit site addresses that could use a wildcard won't be as trustworthy for that convenience if wildcard provisioning falls back to explicit FQDNs.
Could we have an example of that? I have seen users mixing public + internal certs for the same domain, and earlier in this PR discussion suggested I've got a public wildcard and server, as well as local services that I can connect to over my LAN with split DNS using that same domain and wildcard cert but resolving to a different IP. That is quite a common practice for self-host enthusiasts. I'm not familiar with Cloudflare Origin, but assume that's different to a public CA cert like from LetsEncrypt with the bulk of devices will have trust store coverage. I could understand with
I would also suggest this approach 👍 You can always bump to a Caddy 3.0 release at a later point when it makes sense. A new major release doesn't need to be due to a massive rewrite / refactor. Or if acceptable switching the default after sufficient notice in a future minor release as suggested.
There's already some features marked as experimental right? Or has been in the past, that's a great approach 🚀 |
It's a cert issued by Cloudflare's non-publicly-trusted CA which only their servers trust, allowing HTTPS between Cloudflare and the upstream server without the upstream needing to have a publicly trusted cert. It was the go-to option before ACME & DNS challenge became more common-place.
I didn't make note of them, but one recently this week on discord that I can't really link to.
No plan to do a v3 anytime soon. Problem is it would be a breaking change to the entire plugin ecosystem because we'd need to change the Go package to have
Many, yes. Via godoc comments or elsewhere in the docs with |
c87b450
to
e3e27e4
Compare
Ok, I hear you. If we merge this, I would like for it to be an experimental / transitional feature, as we prepare to make using existing wildcards the norm/default behavior. This might be the kind of thing best learned from field experience, so I don't want to lock us into one choice either way. So, how about this for a plan:
Sound good? |
Yep! That's what I was trying to get across. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright. Sounds like a plan. Thanks for working on this. Let's try it out!
Actually I imagine it should work with acme automation disabled, does this work?:
|
e3e27e4
to
bdca47d
Compare
I'm not certain @kanashimia (would appreciate if you'd test it), but I think that's already handled without this feature. The |
bdca47d
to
4f4e13e
Compare
For reference, that was effectively covered here: #5216 (comment) With @francislavoie providing the From the perspective of a user with Caddyfile, the difference between external vs Caddy provisioned (internal / acme) certs with such behaviour is probably a bit unexpected. I've not tried the suggested Caddyfile, but assume Perhaps relevant for an eventual opt-out option once switching |
4f4e13e
to
dda73f4
Compare
This will go out with 2.9 beta 1. Would appreciate some field testing! |
I think I'm missing something about this. When I use a single wildcard domain, everything works as expected and it gets a cert. If I try to define multiple wildcard domains, it fails with multiple.Caddyfile.txt |
Interesting, thanks @coandco, looks like the Caddyfile adapter part is consolidating automation policies too aggressively. I'll try to make a fix. |
The previous config would end requesting a TLS certificate for each individual subdomain and not use the wildcard certificate. This change modifies the labels used on the containers to create host matchers and handlers to do the routing under a single wildcard Caddyfile site. This is a little trickier and more verbose while defining the labels but ends a much cleaner Caddyfile[1][2] and only requires a single certificate. Hopefully this will all be moot once the auto_https prefer_wildcard option is released in `2.9.x`. 1. https://caddyserver.com/docs/caddyfile/patterns#wildcard-certificates 2. https://caddy.community/t/docker-proxy-wildcard-subdomains/22170 3. caddyserver/caddy#6146
Closes #5447
This implements a new
auto_https prefer_wildcard
option, which drops automation policies for non-wildcard domains when there's already a wildcard in another policy.This allows users to flatten their config, and instead of using a pattern like https://caddyserver.com/docs/caddyfile/patterns#wildcard-certificates they can instead do something like:
This would only produce a single wildcard certificate, and no individual certificate for
foo.example.com
since it's already covered by the wildcard.This also allows specifying multiple arguments to
auto_https
, so you can do the following to set multiple Automatic HTTPS options. Previously only one could be set at a time, which was generally fine because there wasn't actually any usecase where it would be useful:I've only manually (visually) tested with a few simple usecases. Unfortunately we have a big lack of tests for the Automatic HTTPS logic because it manipulates config at runtime. I probably need help with testing this to make sure it doesn't have weird side effects. Thankfully, this should be safe/backwards compatible as long as users don't enable this option. We could call it experimental for now.