xpack.security.http.ssl.verification_mode is missing from the docs

[linghengqian]

Description

• xpack.security.http.ssl.verification_mode is missing from the docs for 8.1. Reference https://www.elastic.co/guide/en/elasticsearch/reference/8.1/security-settings.html .
• But in fact this property exists, refer to the third point of Resolution in https://www.elastic.co/guide/en/elasticsearch/reference/8.1/trb-security-setup.html, but there is no place to set this property point to its meaning.
• 
• In conclusion, I hope https://www.elastic.co/guide/en/elasticsearch/reference/8.1/security-settings.html can provide a description of this property.

[elasticmachine]

Pinging @elastic/es-docs (Team:Docs)

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[tvernum]

The fact that this isn't documented is intentional, because it shouldn't ever be set, and the fact that setup-passwords uses it is really a bug.
However, given we mention it in 1 part of the docs, and it is technically a valid setting (even if it shouldn't be used), we probably should document it (even if only to say that it is only useful for setup-passwords)

[linghengqian]

The fact that this isn't documented is intentional, because it shouldn't ever be set, and the fact that setup-passwords uses it is really a bug. However, given we mention it in 1 part of the docs, and it is technically a valid setting (even if it shouldn't be used), we probably should document it (even if only to say that it is only useful for setup-passwords)

• So, is there an improvement plan for docker compose for this page? ( https://www.elastic.co/guide/en/elastic-stack-get-started/8.1/get-started-stack-docker.html#docker-compose-file ) No one seems to notice this within [DOCS] Replace Docker Compose files and setup  #81133 .
• 

[lockewritesdocs]

@tvernum, should we remove that setting from the example docker-compose.yml file? We can document it in the security settings, but if it shouldn't be set, then I think that it makes sense to remove it from the example.

[lockewritesdocs]

@ywangd, can I get your input on this issue? Should we document xpack.security.http.ssl.verification_mode in the secure settings page and also remove it from the example docker-compose.yml file?

[ywangd]

@lockewritesdocs Yes it can be removed from the docker-compose.yml. When ES running as a server, this setting is completely useless and should not be set at all. Unfortunately, the setup-passwords tool shares the same settings with the ES server but running in a client mode. So it is useful only in this particular use case. Maybe in the long term, we should separate the settings used by the server from those for CLI tools. But for now documenting it explicilty and removing it from server side setting are the way to go.

[tvernum]

It turns out not to be just setup-passwords. It looks like reset-password has inherited the same behaviour.

I would propose that we migrate away from relying on verification mode, and instead start trusting the actual HTTP server cert instead.
CommandLineHttpClient.pinnedCaCertFingerprint already does something like that. We probably don't want to do exactly that (because there's no requirement that a node have a copy of its CA, so we may not know the fingerprint of the CA), but we could verify that the node provided the same cert that is configured in elasticsearch.yml

That would mean verification_mode is never needed anymore, and we can drop it everywhere, and simply document it as a "this is valid, but unncessary & discouraged".

[tvernum]

I just realised this issue doesn't explain why that setting shouldn't be set. I've explained it elsewhere (possibly on other issues) but it's worth covering here since this has become the main issue for tracking the lack of docs.

---

verification_mode defines how to verify the certificates presented by the other party in the SSL/TLS connection.

For TLS clients there are 3 values, all of which have meaning, and can be useful (although one of them is particularly dangerous and should be used with extreme caution).

• full verifies that the certificate is valid (e.g. it is within the not_before and not_after dates), chains to a trusted anchor (CA), and has a Subject Alternative Name (hostname) that matches the address to which the connection was intended
• certificate checks validity & trust chain (CA) but does not check the hostname (Subject Alternative Name).
• none checks nothing - neither validity, trust, or hostname. This is very dangerous, are ought to be avoided (but is occasionally necessary for debugging purposes, or to temporarily work around a transient issue such as expired certificates - but fixing the certificates is a much idea).

For TLS servers there is no such thing as hostname verification of the client's certificate. The server didn't open the connection so it doesn't have an address to check against. (Some servers do non-standard checks involving reverse DNS lookups, but that's not part of TLS or HTTPS).
So, when we are configuring TLS for a server, the full and certificate verification modes are identical. That means there are only 2 meaningful options - full and none.
However, client certificates are entirely optional in TLS. By default, they are not used, and in Elasticsearch you can set the client_authentication setting to none, optional or required.

Because of that, there's no case where setting verification mode to none makes sense.
If you configure:

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.client_authentication: required
xpack.security.http.ssl.verification_mode: none

It means, Clients must provide a certificate, but I don't care what certificate it is. That's nonsense. Don't do that. If you don't care about the certs, don't require them.

If you configure:

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.client_authentication: none
xpack.security.http.ssl.verification_mode: none

Then there are no certificates to verify (because client authentication is off), so the verification_mode is useless.

Given that xpack.security.http.ssl configures a server with configurable options for client_authentication, there is no need to have a configurable verification_mode.

---

The minor issue we have is that setup-passwords and reset-password configure their HTTPS client using the settings from xpack.security.http.ssl. That means that in this very specific case, xpack.security.http.ssl is used for both client and server settings. That was a bad idea, and we should fix it.

[justincr-elastic]

@tvernum thank you for the explanation. I have a followup question about mTLS to Elasticsearch.

In Kibana 8.3 documentation, it describes using a TLS client for Kibana authentication, and an Elasticsearch PKI realm for authorization. In other words, use role mapping from the Kibana TLS client's Subject DN to the kibana_system role.

• https://www.elastic.co/guide/en/kibana/8.3/elasticsearch-mutual-tls.html

In that scenario, the Kibana documention says to set xpack.security.http.ssl.client_authentication to optional. There is no mention of xpack.security.http.ssl.verification_mode, so I assume it would default to full.

What is design intent if xpack.security.http.ssl.verification_mode were changed to certificate? Is it design intent for full and certificate to behave the same way in that documented Kibana mTLS use case?

[tvernum]

Is it design intent for full and certificate to behave the same way in that documented Kibana mTLS use case?

Kibana mTLS doesn't use client certificates between Kibana and ES (hence needing to set client_authentication to optional). So the value of verification_mode has no consequence to Kibana mTLS.