Service nut-server / upsd not starting automatically on CentOS / RockyLinux

[adrianTNT]

I have some trouble with NUT on RockyLinux 9.2, Network UPS Tools upsc 2.8.2
Later edit: driver = nutdrv_qx, NUT installed with yum: Package nut-2.8.2-2.el9.x86_64

I ran systemctl enable nut-server to add nut-server to auto start, but after reboot it says is inactive (dead), so command to get UPS data (e.g upsc myups1) says Connection refused.

I looked in /var/log/messages and since reboot there is no log line containing nut-, I understand it didn't even try to start, not sure where to look at next.

If I manually run service nut-server restart or service upsd restart then service starts OK and I can see the UPS status text when running upsc myups1

As a fix, I can auto start the service nut-server / upsd if I add it to a cronjob with @reboot:

 @reboot /usr/sbin/service nut-server restart > /dev/null 2>&1;

• I don't know if I configured something wrong or it is just not that reliable on RockyLinux / CentOS ? I can leave it starting with cron job at reboot.
• do I need to add any other NUT related services to auto-start ? I only need to get data of local UPS connected by USB, no network UPS related setup
• Is it normal for both upsd and nut-server to write same messages in /var/log/messages or does this indicate a config problem ?

Dec 15 11:36:12 mycomputer nut-server[2299]: Found 1 UPS defined in ups.conf
Dec 15 11:36:12 mycomputer upsd[2299]: Found 1 UPS defined in ups.conf

Thanks.

[jimklimov]

One thing that comes to mind - when you enabled the service, which symlinks did systemd report as created? Not sure about packaging, but vanilla services deliver a nut.target which "Wants" or "Requires" other units, and is wanted-by the system milestones. So check if it is enabled too?

[adrianTNT]

which symlinks did systemd report as created

You mean these ? It created 2

systemctl enable nut-server
Created symlink /etc/systemd/system/upsd.service → /usr/lib/systemd/system/nut-server.service.
Created symlink /etc/systemd/system/nut.target.wants/nut-server.service → /usr/lib/systemd/system/nut-server.service.

Sorry, I don't fully understand what you meant about "wants" but I remember I seen "wants" when I looked for nut related files, are these "want" files relevant to know what other services to enable ?

[root@localhost ~]# find /etc/ -name *nut*
/etc/ups/nut.conf
/etc/systemd/system/nut-driver@.service.d
/etc/systemd/system/nut-driver@.service.d/nut-driver-enumerator-generated-checksum.conf
/etc/systemd/system/nut-driver.target.wants
/etc/systemd/system/nut-driver.target.wants/nut-driver@myups1.service
/etc/systemd/system/nut-driver@myups1.service.d
/etc/systemd/system/nut-driver@myups1.service.d/nut-driver-enumerator-generated-checksum.conf
/etc/systemd/system/nut-driver@myups1.service.d/nut-driver-enumerator-generated-devicename.conf
/etc/systemd/system/nut.target.wants
/etc/systemd/system/nut.target.wants/nut-server.service
[root@localhost ~]#

A more serious thing I noticed is that if I disconnect USB cable and plug it back in, it fails to get data:

service nut-server status 
Data for UPS [myups1] is stale - check driver

Works again if I start driver with: upsdrvctl start which says it found a running copy, it stops that one and starts another one, then I can get data again by upsc myups1

I can get around these by writing monitoring scripts but not sure if NUT services should already do this by themselves.

[jimklimov]

Cheers, regarding upsdrvctl start - as you noted, it finds a running copy (wrapped under the nut-drvier@myups1.service you've shown a bit above) and there's actually a tug of war between the service and manually-started driver daemon. You can see more detail about that in https://github.com/networkupstools/nut/wiki/nut%E2%80%90driver%E2%80%90enumerator-(NDE) but the short of it is that upsdrvsvcctl restart may be a better choice for systemd/SMF augmented distributions.

I can't quickly see from the earlier posts what UPS device and driver you have, but normally they should re-connect in case of communications problems, and you would see the data become "not stale" after a while (there are several polling intervals involved in a driver, upsd data server, and upsmon client).

Regarding service start-up, there seem to be too few systemd mentions in the "screenshot" above, for comparison, on my test system I see the following, which I'd detail as they go in alphabetic order, below.

:; ls -ld `find /etc/systemd | grep -w nut | sort

lrwxrwxrwx 1 root root  30 Dec  5 03:32 /etc/systemd/system/multi-user.target.wants/nut.target -> /lib/systemd/system/nut.target
drwxr-xr-x 2 root root   4 Apr 19  2024 /etc/systemd/system/nut-driver.target.wants
lrwxrwxrwx 1 root root  39 Jan  2  2023 /etc/systemd/system/nut-driver.target.wants/nut-driver@dummy.service -> /lib/systemd/system/nut-driver@.service
lrwxrwxrwx 1 root root  39 Apr 19  2024 /etc/systemd/system/nut-driver.target.wants/nut-driver@eco650.service -> /lib/systemd/system/nut-driver@.service
drwxr-xr-x 2 root root   3 Nov  1  2022 /etc/systemd/system/nut-driver@.service.d
-rw-r--r-- 1 root root  81 Apr 19  2023 /etc/systemd/system/nut-driver@.service.d/nut-driver-enumerator-generated-checksum.conf
lrwxrwxrwx 1 root root   9 Jan  2  2023 /etc/systemd/system/nut-driver@dummy.service -> /dev/null
drwxr-xr-x 2 root root   4 Apr 19  2024 /etc/systemd/system/nut-driver@dummy.service.d
-rw-r--r-- 1 root root  74 Jul 23 00:47 /etc/systemd/system/nut-driver@dummy.service.d/nut-driver-enumerator-generated-checksum.conf
-rw-r--r-- 1 root root  39 Apr 19  2024 /etc/systemd/system/nut-driver@dummy.service.d/nut-driver-enumerator-generated-devicename.conf
drwxr-xr-x 2 root root   5 Apr 19  2024 /etc/systemd/system/nut-driver@eco650.service.d
-rw-r--r-- 1 root root  74 Apr 19  2024 /etc/systemd/system/nut-driver@eco650.service.d/nut-driver-enumerator-generated-checksum.conf
-rw-r--r-- 1 root root  40 Apr 19  2024 /etc/systemd/system/nut-driver@eco650.service.d/nut-driver-enumerator-generated-devicename.conf
-rw-r--r-- 1 root root 364 Apr 19  2024 /etc/systemd/system/nut-driver@eco650.service.d/nut-driver-enumerator-generated.conf
drwxr-xr-x 2 root root   7 Dec  5 03:32 /etc/systemd/system/nut.target.wants
lrwxrwxrwx 1 root root  46 Dec  5 03:32 /etc/systemd/system/nut.target.wants/nut-driver-enumerator.path -> /lib/systemd/system/nut-driver-enumerator.path
lrwxrwxrwx 1 root root  49 Dec  5 03:32 /etc/systemd/system/nut.target.wants/nut-driver-enumerator.service -> /lib/systemd/system/nut-driver-enumerator.service
lrwxrwxrwx 1 root root  37 Dec  5 03:32 /etc/systemd/system/nut.target.wants/nut-driver.target -> /lib/systemd/system/nut-driver.target
lrwxrwxrwx 1 root root  39 Dec  5 03:32 /etc/systemd/system/nut.target.wants/nut-monitor.service -> /lib/systemd/system/nut-monitor.service
lrwxrwxrwx 1 root root  38 Dec  5 03:32 /etc/systemd/system/nut.target.wants/nut-server.service -> /lib/systemd/system/nut-server.service

Generally note that in systemd the unitname.d directories allow "drop-in" customizations over the default unit as delivered by packaging and/or allowing instantiation (e.g. nut-driver@.service definition), and links to /dev/null mark masked services that should never get enabled or started. Enablement of targets (and other units) should have been part of packaging, or can be done manually as systemctl enable UNITNAME.

• Result of systemctl enable nut.target, so the system milestone multi-user.target "wants" (tries, does not fail if unsuccessful) to start NUT (whatever nut.target would pull in):

/etc/systemd/system/multi-user.target.wants/nut.target -> /lib/systemd/system/nut.target

• Result of systemctl enable nut-driver@{dummy,eco650}.service, done by NDE based on ups.conf contents:

/etc/systemd/system/nut-driver.target.wants
/etc/systemd/system/nut-driver.target.wants/nut-driver@dummy.service -> /lib/systemd/system/nut-driver@.service
/etc/systemd/system/nut-driver.target.wants/nut-driver@eco650.service -> /lib/systemd/system/nut-driver@.service

• Nominal customization of nut-driver@.service.d template, generated by NDE from ups.conf contents for tracking of "global" part of configuration:

/etc/systemd/system/nut-driver@.service.d
/etc/systemd/system/nut-driver@.service.d/nut-driver-enumerator-generated-checksum.conf

• The nut-driver@dummy.service never auto-starts; I can manage that (virtual) NUT driver as a program in testing whichever way I like (e.g. with command-line configuration):

/etc/systemd/system/nut-driver@dummy.service -> /dev/null

• Customizations for unit of a dummy UPS, generated by NDE from ups.conf contents:

/etc/systemd/system/nut-driver@dummy.service.d
/etc/systemd/system/nut-driver@dummy.service.d/nut-driver-enumerator-generated-checksum.conf
/etc/systemd/system/nut-driver@dummy.service.d/nut-driver-enumerator-generated-devicename.conf

• Customizations for unit of a for eco650 UPS, generated by NDE from ups.conf contents:

/etc/systemd/system/nut-driver@eco650.service.d
/etc/systemd/system/nut-driver@eco650.service.d/nut-driver-enumerator-generated-checksum.conf
/etc/systemd/system/nut-driver@eco650.service.d/nut-driver-enumerator-generated-devicename.conf
/etc/systemd/system/nut-driver@eco650.service.d/nut-driver-enumerator-generated.conf

• Result of systemctl enable nut-driver-enumerator.{path,service} nut-driver.target nut-monitor.service nut-server.service which "wants" those units to auto-start as part of nut.target which in turn would be "wanted" by the system milestone multi-user.target as seen above (note there are other NUT units, especially around NDE implementation variants, that may be or not be useful in a particular deployment):

/etc/systemd/system/nut.target.wants
/etc/systemd/system/nut.target.wants/nut-driver-enumerator.path -> /lib/systemd/system/nut-driver-enumerator.path
/etc/systemd/system/nut.target.wants/nut-driver-enumerator.service -> /lib/systemd/system/nut-driver-enumerator.service
/etc/systemd/system/nut.target.wants/nut-driver.target -> /lib/systemd/system/nut-driver.target
/etc/systemd/system/nut.target.wants/nut-monitor.service -> /lib/systemd/system/nut-monitor.service
/etc/systemd/system/nut.target.wants/nut-server.service -> /lib/systemd/system/nut-server.service

In your case it seems that no units were enabled by packaging (distro problem?) or you disabled some?

[jimklimov]

Posted the above to https://github.com/networkupstools/nut/wiki/NUT-systemd-service-units

[jimklimov]

Note that you can view driver logs in journalctl -lu nut-driver@myups1 or follow them in real-time (as e.g. you experiment with cable pulling) with journalctl -flu nut-driver@myups1. You may want to bump debug verbosity during the experiments, and tone it down later - see https://github.com/networkupstools/nut/wiki/Changing-NUT-daemon-debug-verbosity

[adrianTNT]

Thanks for the details Jim.
I did thought that upsdrvsvcctl restart is better than start, but it says Unrecognized argument: restart so I guess I am using a very old version ?

usage: /usr/sbin/upsdrvsvcctl [OPTIONS] (start | stop | shutdown) [<ups>]

Not sure how to check versions because I am confused by the many components, but the logs mention Network UPS Tools - UPS driver controller 2.8.2, yum says Package nut-2.8.2-2.el9.x86_64 installed.

Driver is nutdrv_qx I forgot to mention.

Ups if it matters is an Njoy Keen 800 USB

lsusb -vvv
idVendor      0x1a86 QinHeng Electronics
idProduct     0x7523 CH340 serial converter

In your case it seems that no units were enabled by packaging (distro problem?) or you disabled some?

You mean if I customised something during install ? I just installed it with yum (Package nut-2.8.2-2.el9.x86_64) without altering anything

I had this "APC" one installed first: apcupsd.x86_64 : APC UPS Power Control Daemon, then installed NUT, and then I uninstalled the APC one, I thought maybe they shared some components and uninstalling APC one removed part of NUT ? But since then I tried a yum remove and yum install nut again.

After changing the UPS name or driver name I need to run upsdrvsvcctl reconfigure and upsdrvsvcctl resync ?
I did that now and it created one more NUT file here:

find /etc/systemd | grep nut | sort
/etc/systemd/system/nut-driver@myups1.service.d
/etc/systemd/system/nut-driver@myups1.service.d/nut-driver-enumerator-generated-checksum.conf
/etc/systemd/system/nut-driver@myups1.service.d/nut-driver-enumerator-generated.conf <<< THIS ONE
/etc/systemd/system/nut-driver@myups1.service.d/nut-driver-enumerator-generated-devicename.conf
/etc/systemd/system/nut-driver@.service.d
/etc/systemd/system/nut-driver@.service.d/nut-driver-enumerator-generated-checksum.conf
/etc/systemd/system/nut-driver.target.wants
/etc/systemd/system/nut-driver.target.wants/nut-driver@myups1.service
/etc/systemd/system/nut.target.wants
/etc/systemd/system/nut.target.wants/nut-server.service

If Package nut-2.8.2-2.el9.x86_64 is old and there is nothing obvious to fix above, I will probably look into downloading from GitHub and compile myself, I don't ahve much experience with that thou.

[jimklimov]

My bad, no restart in either of upsdrv{,svc}ctl tools, indeed - just a stop and then start again (optionally with one device name as the last arg).

For custom compilations you can try to follow https://github.com/networkupstools/nut/wiki/Building-NUT-for-in%E2%80%90place-upgrades-or-non%E2%80%90disruptive-tests but to fix up the service units, probably this part of those instructions can suffice with whatever your packages delivered:

   { sudo systemctl stop nut-monitor nut-server || true ; } && \
   { sudo systemctl stop nut-driver.service || true ; } && \
   { sudo systemctl stop nut-driver.target || true ; } && \
   { sudo systemctl stop nut.target || true ; } && \
   sudo systemctl daemon-reload && \
   sudo systemd-tmpfiles --create && \
   sudo systemctl disable nut.target nut-driver.target nut-monitor nut-server nut-driver-enumerator.path nut-driver-enumerator.service && \
   sudo systemctl enable  nut.target nut-driver.target nut-monitor nut-server nut-driver-enumerator.path nut-driver-enumerator.service && \
   { sudo systemctl restart udev || true ; } && \
   sudo systemctl restart nut-driver-enumerator.service nut-monitor nut-server

Note the disable/enable of units - that is needed to surely record the symlinks they defined according to "wants" etc. properties. Primarily needed when these files could get updated too (as in a custom build) but should not hurt generally.

After changing the UPS name or driver name I need to run upsdrvsvcctl reconfigure and upsdrvsvcctl resync ?

Probably that should do it, probably either one would suffice; if the nut-driver-enumerator.path (or daemon variants) unit is enabled, edition of ups.conf should automatically trigger re-definition and reload-or-restart of impacted units, as long as systemd upholds its part of the deal (not 100% foolproof, e.g. they can not-notice the change when system is stressed), otherwise I call the script nut-driver-enumerator.sh directly (which is what upsdrvsvcctl resync does, and the tool is a somewhat more "stable API" for such things).

[pestevao]

Resuming.

For make it start automatically on Red Hat like systemd:

sudo systemctl enable nut-server
sudo systemctl enable nut-driver@UPS_NAME_ON_ups.conf_FILE
sudo systemctl enable nut.target

Greetings

[jimklimov]

@pestevao : note that nut-driver@... does not appear by itself, the nut-driver-enumerator (script or service) handles that wrapping, and should enable/start it as well. See more at https://github.com/networkupstools/nut/wiki/nut%E2%80%90driver%E2%80%90enumerator-(NDE)

And the other units (server, target) should be activated by the package or systemd-presets file delivered by NUT. Unless packagers botched their job, or you have a distro version with an older release that did not get modern features and fixes.