monstache can not connect to elasticsearch v8 docker compose

[AsedMorteza]

hi @rwynn
When I use version 7 of Elastic and Kibana in Docker Compose, everything is fine, but when I change the version of the image file to 8, this error is displayed:

c-monstache | ERROR 2022/07/14 11:05:55 Unable to create Elasticsearch client: health check timeout: Head "http://es7:9200": dial tcp 172.20.0.2:9200: connect: connection refused: no Elasticsearch node available

[rwynn]

It may be because in Elasticsearch 8 security is enabled by default.
https://www.elastic.co/guide/en/elasticsearch/reference/current/manually-configure-security.html

You can configure monstache using config options elasticsearch-user / elasticsearch-password OR elasticsearch-pki-auth and transport level security with elasticsearch-pem-file and elasticsearch-validate-pem-file.

https://rwynn.github.io/monstache-site/config/

[AsedMorteza]

thanks for the reply
According to the content in the file elasticsearch.yml :

Enable security features

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents

xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12

Enable encryption and mutual authentication between cluster nodes

xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12

What should my config be like?
Thanks

[AsedMorteza]

It may be because in Elasticsearch 8 security is enabled by default. https://www.elastic.co/guide/en/elasticsearch/reference/current/manually-configure-security.html

You can configure monstache using config options elasticsearch-user / elasticsearch-password OR elasticsearch-pki-auth and transport level security with elasticsearch-pem-file and elasticsearch-validate-pem-file.

https://rwynn.github.io/monstache-site/config/

hi @rwynn
when (elasticsearch-pem-file = "/etc/elasticsearch/certs/transport.p12")
print this message:
Unable to create Elasticsearch client: open /etc/elasticsearch/certs/transport.p12: permission denied

and when ( [elasticsearch-pki-auth]
cert-file = "/etc/elasticsearch/certs/http_ca.crt")
print :
ERROR 2022/07/17 16:59:21 Unable to create Elasticsearch client: Elasticsearch client auth key file is empty

can you please help me ...

[rwynn]

Monstache only connects via the http/https endpoint. So you can generate a pem file and then reference that in your monstache config.

This configuration worked for me on a new Elasticsearch 8 install:

elasticsearch-urls = [ "https://localhost:9200" ]

elasticsearch-user="elastic"
elasticsearch-password="paste here"

elasticsearch-pem-file = "/home/ubuntu/http.pem"
elasticsearch-validate-pem-file = false

• regenerate a password as root for the elastic user if you don't already know it. Copy the output to your monstache config file replacing "paste here".

cd /usr/share/elasticsearch
bin/elasticsearch-reset-password -u elastic

• generate the pem file from the elasticsearch files as root (in 3rd command use output from 2nd command):

cd /usr/share/elasticsearch
bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password
openssl pkcs12 -in /etc/elasticsearch/certs/http.p12 -out /home/ubuntu/http.pem -nodes
chmod 644 /home/ubuntu/http.pem

[rwynn]

In your case the url to connect in docker compose would be https://es7:9200. Notice that is https not http.
You can mount the pem file as a volume into the monstache docker container.

[AsedMorteza]

@rwynn Thank you for your wonderful help!