self-signed certificates !!

[xorgxx]

Hi, I’ve been using Mercure for a long time now, and I love it. However, all this time, I’ve had to configure it like this:

http_client:
    default_options:
        verify_peer: false
        verify_host: false

This applies both in development and, unfortunately, in production as well. I’ve tried for a long time to resolve this issue, but without success!

I’m using Docker (latest version) and Mercure (latest version). Everything works well in both dev and prod, but as soon as I switch to:

http_client:
    default_options:
        verify_peer: true
        verify_host: true

I get the following error: "message": "Failed to send an update."

When I use this command:

curl --request POST --url https://xxxxxxx.com:9061/.well-known/mercure --header 'authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJtZXJjdXJlIjp7InB1Ymxpc2giOlsiKiJdLCJzdWJzY3JpYmUiOlsiaHR0cHM6Ly9leGFtcGxlLmNvbS9teS1wcml2YXRlLXRvcGljIiwie3NjaGVtZX06Ly97K2hvc3R9L2RlbW8vYm9va3Mve2lkfS5qc29ubGQiLCIvLndlbGwta25vd24vbWVyY3VyZS9zdWJzY3JpcHRpb25zey90b3BpY317L3N1YnNjcmliZXJ9Il0sInBheWxvYWQiOnsidXNlciI6Imh0dHBzOi8vZXhhbXBsZS5jb20vdXNlcnMvZHVuZ2xhcyIsInJlbW90ZUFkZHIiOiIxMjcuMC4wLjEifX19.KKPIikwUzRuB3DTpVw6ajzwSChwFw5omBMmMcWKiDcM' --header 'content-type: application/x-www-form-urlencoded' --data topic=https://example.com/my-private-topic --data 'data={"text":"My text value"}'

It returns: urn:uuid:47f74e92-4092-465a-956d-4fea867abfb9

here my config :

# docker-compose.yml
version: "3.9"

services:
  caddy:
    image: dunglas/mercure
    restart: unless-stopped

    container_name: mercure-update
    #environment:
      # Uncomment the following line to disable HTTPS
      #SERVER_NAME: ':80'
      #MERCURE_PUBLISHER_JWT_KEY: '!ChangeMe!'
      #MERCURE_SUBSCRIBER_JWT_KEY: '!ChangeMe!'
    # Uncomment the following line to enable the development mode
    command:  caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
    ports:
      - "9060:80"
      - "9061:443"

    volumes:
      - /volume1/docker/mercure/xxxxxx:/etc/caddy
      - /volume1/docker/mercure/xxxxxx/ssl:/etc/xxxxxx/ssl

# Learn how to configure the Mercure.rocks Hub on https://mercure.rocks/docs/hub/config
{
    # Debug mode (disable it in production!)
    #debug
    email xorg@i2p.i2p
    # HTTP/3 support
    #servers {
    #    protocol {
    #        experimental_http3
    #    }
    #}
}

{$SERVER_NAME:xxxxxx.com}:443
# using self-signed 
tls /etc/xxxxxx/ssl/RSA-cert.pem /etc/xxxxxx/ssl/RSA-privkey.pem
log
route {

    redir / /.well-known/mercure/ui/
    encode gzip

    mercure {

        # Enable the demo endpoint (disable it in production!)
        demo

        # Publisher JWT key
        publisher_jwt "!ChangeThisMercureHubJWTSecretKey!"

        # Subscriber JWT key
        subscriber_jwt "!ChangeThisMercureHubJWTSecretKey!"
       
        publish_origins *

        cors_origins *

        # Allow anonymous subscribers (double-check that it's what you want)
        anonymous

        # Enable the subscription API (double-check that it's what you want)
        subscriptions

    }

    respond "Not Found" 404
}

mercure:
    hubs:
        default:
            url: '%env(MERCURE_URL)%'
            public_url: '%env(MERCURE_PUBLIC_URL)%'
            jwt:
                secret: '%env(MERCURE_JWT_SECRET)%'
                publish: '*'

It seems that the Docker server images are working properly. The certificates on the Mercure server are from Let's Encrypt, the development application uses OpenSSL, and the production application uses Let's Encrypt.

So, my question is: Does Mercure support self-signed certificates in both development AND production environments?

thank.

[dunglas]

If you can access the hub in HTTPS using a browser or curl, this is a problem in your PHP container.

CA certs are probably missing or misconfigured in the PHP container/server. You need to install a bundle (usually the ca-certificates package on most distributions), see https://curl.se/docs/sslcerts.html.

[xorgxx]

Thank you for the fast answer. I will check this! 🦖

[xorgxx]

Well, so i have update dockerfile and add ca-certificates

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    cron g++ gettext libicu-dev openssl libc-client-dev libkrb5-dev ca-certificates \
   ....
RUN echo "openssl.cafile=/etc/ssl/certs/ca-certificates.crt" >> /usr/local/etc/php/conf.d/ssl.ini \
    && echo "openssl.capath=/etc/ssl/certs" >> /usr/local/etc/php/conf.d/ssl.ini
   .....

root@0588bd5caa20 /var/www # php -r "var_dump(openssl_get_cert_locations());"
array(8) {
  ["default_cert_file"]=>string(21) "/usr/lib/ssl/cert.pem"
  ["default_cert_file_env"]=> string(13) "SSL_CERT_FILE"
  ["default_cert_dir"]=>string(18) "/usr/lib/ssl/certs"
  ["default_cert_dir_env"]=>string(12) "SSL_CERT_DIR"
  ["default_private_dir"]=>string(20) "/usr/lib/ssl/private"
  ["default_default_cert_area"]=>string(12) "/usr/lib/ssl"
  ["ini_cafile"]=>string(34) "/etc/ssl/certs/ca-certificates.crt"
  ["ini_capath"]=>string(14) "/etc/ssl/certs"

root@0588bd5caa20 /var/www #  curl -v https://xxxxxxx.com:9061/.well-known/mercure
*   Trying xx.xx.xxx.xxx...
* TCP_NODELAY set
* Connected to xxxxx.com (xx.xx.xxx.xxx) port 9061 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

i still get error "update send fail"
will try again tomorrow ...

[dunglas]

I see that you are using a non-standard port. This is not supported by Let's Encrypt when using the HTTP-01 challenge (the default). You'll have to use the default port (443) or switch to a more complex ACME challenge such as DNS-01. https://community.letsencrypt.org/t/using-encrypt-for-non-standard-ports/20164/2

[xorgxx]

you mean this config

# docker-compose.yml
version: "3.9"

services:
  caddy:
    image: dunglas/mercure
    restart: unless-stopped

    container_name: mercure-update
    #environment:
      # Uncomment the following line to disable HTTPS
      #SERVER_NAME: ':80'
      #MERCURE_PUBLISHER_JWT_KEY: '!ChangeMe!'
      #MERCURE_SUBSCRIBER_JWT_KEY: '!ChangeMe!'
    # Uncomment the following line to enable the development mode
    command:  caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
    ports:
      - "9060:80"          <----
      - "9061:443"        <----

    volumes:
      - /volume1/docker/mercure/xxxxxx:/etc/caddy
      - /volume1/docker/mercure/xxxxxx/ssl:/etc/xxxxxx/ssl

Do you mean it's just a redirection to a container, or is there an incompatibility between xxxxx.com:9061 and the container running on port 443? Or are you asking if Let's Encrypt can't access xxxxx.com:9061 and can only reach xxxxx.com directly? To be honest, I'm not exactly sure where the issue lies—whether it's on the Mercure server, in my Symfony app, or with the browser or HTTP client I'm using.

[dunglas]

Basically, the public ports must be the standard ports or it won't work. Yo ease debugging, I strongly encourage using standard ports everywhere (on the host and in the container).

[xorgxx]

So, I’ve found a solution, but I’m not sure if it’s the best practice. Here’s what I did:

I installed the Mercure container on port 80. My server is running a system called WebStation, and I linked the container to WebStation, exposing port 80. Now, mercure.xxxxx.com is working. Instead of using the SSL certificate from the Mercure container, I provided my server’s SSL certificate to mercure.xxxxx.com. I checked it on SSL Labs, and it reports that everything is fine!

What do you think—does this sound like a solid solution? I know it might sound a bit crazy, and maybe it’s not entirely clear.

[xorgxx]

log mercure

WRN ts=1729064428.3628864 msg=failed to set GOMAXPROCS error=open /sys/fs/cgroup/cpu/cpu.cfs_quota_us: no such file or directory

INF ts=1729064428.363403 msg=using config from file file=/etc/caddy/Caddyfile

INF ts=1729064428.369608 msg=adapted config to JSON adapter=caddyfile

WRN ts=1729064428.3696916 msg=Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies adapter=caddyfile file=/etc/caddy/Caddyfile line=3

INF ts=1729064428.371863 logger=admin msg=admin endpoint started address=localhost:2019 enforce_origin=false origins=["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]

WRN ts=1729064428.3728101 logger=http.auto_https msg=server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server server_name=srv0 http_port=80

INF ts=1729064428.3748562 logger=tls.cache.maintenance msg=started background certificate maintenance cache=0xc00074c200

INF ts=1729064428.3778641 logger=http.log msg=server running name=srv0 protocols=["h1","h2","h3"]

INF ts=1729064428.3783524 msg=autosaved config (load with --resume flag) file=/config/caddy/autosave.json

INF ts=1729064428.3783813 msg=serving initial configuration

INF ts=1729064428.467714 logger=tls msg=storage cleaning happened too recently; skipping for now storage=FileStorage:/data/caddy instance=b06f60ba-a11e-4544-8231-b26ea770e689 try_again=1729150828.4677079 try_again_in=86399.999998562

INF ts=1729064428.4680552 logger=tls msg=finished cleaning storage units

INF ts=1729064484.9836862 logger=http.log.access msg=handled request request={"remote_ip":"172.21.0.1","remote_port":"41744","client_ip":"172.21.0.1","proto":"HTTP/1.1","method":"GET","host":"mercure.xxxxxx.com","uri":"/.well-known/mercure/ui/","headers":{"X-Forwarded-Port":["443"],"Connection":["close"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Site":["none"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["REDACTED"],"X-Forwarded-By":["192.168.1.17"],"X-Real-Ip":["192.168.1.40"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:131.0) Gecko/20100101 Firefox/131.0"],"Dnt":["1"],"Sec-Gpc":["1"],"X-Forwarded-Proto":["https"],"Accept-Language":["fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Mode":["navigate"],"Priority":["u=0, i"]}} bytes_read=0 user_id= duration=0.052490293 size=2662 status=200 resp_headers={"X-Content-Type-Options":["nosniff"],"X-Xss-Protection":["1; mode=block"],"Content-Security-Policy":["default-src 'self' mercure.rocks cdn.jsdelivr.net"],"Server":["Caddy"],"X-Frame-Options":["DENY"],"Content-Type":["text/html; charset=utf-8"],"Content-Encoding":["gzip"],"Vary":["Accept-Encoding"]}

xorg@NeoxNasFr:~$ curl -v https://mercure.xxxxx.com/.well-known/mercure
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /.well-known/mercure HTTP/2
> Host: mercure.xxxxx.com
> user-agent: curl/7.86.0
> accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 400 
< server: xxxxx.com
< date: Wed, 16 Oct 2024 08:07:33 GMT
< content-type: text/plain; charset=utf-8
< content-length: 27
< content-security-policy: default-src 'self' mercure.rocks cdn.jsdelivr.net
< x-content-type-options: nosniff
< x-frame-options: DENY
< x-xss-protection: 1; mode=block
< 
Missing "topic" parameter.
* TLSv1.2 (IN), TLS header, Supplemental data (23):