Console (SSL) not available since upgrading to 2.4.120

[lsampas]

Version

2.4.120

Installation Method

Cloud image (Amazon, Azure, Google)

Description

upgrading

Installation Type

Distributed

Location

cloud

Hardware Specs

Meets minimum requirements

CPU

8

RAM

32

Storage for /

345G

Storage for /nsm

n/a -- manager node

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

1Gbps to 10Gbps

Status

No, one or more services are failed (please provide detail below)

Salt Status

No, there are no failures

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

Upgrade the the latest version appeared to run OK on the manager. (We have a distributed install with 10+ sensors.)

However, after completion, the manager console is not available, and it appears that nginx is listening on 443 but in plaintext. e.g. https://x.x.x.x:443 fails with cipher mismatch, but http://x.x.x.x:443 gives a 400 "plain HTTP request was sent to HTTPS port" error.
checked certs in /etc/pki/managerssl.crt,key -- still there.
so-status shows all services running on the manager OK.
tried salt-call state.apply ssl.nginx -- works without apparent errors, but same result.
tried so-nginx-restart -- no difference.
tried deleting /etc/pki/managerssl.crt, key and then repeating above. (backed up old ones first)
new cert and key were created, but same error, even after repeating state.apply as above and restarting nginx.
so-status still shows OK.

went into so-nginx via docker exec, certs are there, too, just where they should be. nginx -t says syntax is OK.

nginx log in /opt/so/log/nginx (access.log) is indeed updated on plaintext http over 443 requests.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[dougburks]

What browser are you using? Have you tried using chromium or Google Chrome to see if that makes any difference?

[lsampas]

Same result with Edge. Firefox gives a tab crashed in either https://x.x.x.x or http://x.x.x.x:443
Curl gives the 400 bad request http to https port.

[lsampas]

on the sensors, so-status reports everything is running. so-suricata-restart and so-zeek-restart work fine. I'll look again Monday.

[dougburks]

Are you using the default Security Onion cert or have you replaced it with your own custom cert?

[lsampas]

We have custom. I tried regenerating our certs, made no difference. I was thinking of regenerating with an RSA key rather than EC or backing up and deleting to see if we get an auto-generated cert.

[dougburks]

Please try the following:

# Create a backup copy of nginx config
sudo cp /opt/so/saltstack/default/salt/nginx/etc/nginx.conf /root/nginx.conf.tlsv12

# Update the ssl_protocol lines
sudo sed -i 's|ssl_protocols TLSv1.2;|ssl_protocols TLSv1.2 TLSv1.3;|g' /opt/so/saltstack/default/salt/nginx/etc/nginx.conf

# Restart services
sudo salt-call state.apply nginx

[lsampas]

That did it. (I got impatient after state.apply didn't fix it immediately and ran an so-nginx-restart.)

Now I get to explain to management how we got better support from SO (and the founder no less) than we do for certain products we pay for.

Thank you!

[dougburks]

Glad that worked! We've created an issue to fix this in the next release:
#14252

[jmolletAMNH]

I just had a similar experience and it turns out our issue was inconsistent SSL protocol versions. We fixed it by adding TLSv1.3 to the ssl_protocols line in /opt/so/saltstack/default/salt/nginx/etc/nginx.conf.

ssl_protocols TLSv1.2; --> ssl_protocols TLSv1.2 TLSv1.3;