omr-admin.service stuck in reboot loop

[donbecker]

Hi,

I've deployed the VPS using the scripts on AWS EC2, using a Debian 10 image.

After using it for a few months and then rebooting, it appears that omr.admin.service is in a reboot loop:

tail -f /var/log/syslog

Dec 8 19:19:39 ip-172-31-14-148 OMR-Service: Restart OMR-Admin
Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: Stopping OMR-Admin...
Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: omr-admin.service: Main process exited, code=killed, status=15/TERM
Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: omr-admin.service: Succeeded.
Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: Stopped OMR-Admin.
Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: Started OMR-Admin.
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: Traceback (most recent call last):
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/bin/omr-admin.py", line 773, in
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: omr_config_data = json.load(f)
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/init.py", line 296, in load
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/init.py", line 348, in loads
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: return _default_decoder.decode(s)
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: raise JSONDecodeError("Expecting value", s, err.value) from None
Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Main process exited, code=exited, status=1/FAILURE
Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Failed with result 'exit-code'.
Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Service RestartSec=100ms expired, scheduling restart.
Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Scheduled restart job, restart counter is at 1.
Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: Stopped OMR-Admin.
Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: Started OMR-Admin.
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: Traceback (most recent call last):
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/bin/omr-admin.py", line 773, in
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: omr_config_data = json.load(f)
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/init.py", line 296, in load
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/init.py", line 348, in loads
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: return _default_decoder.decode(s)
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: raise JSONDecodeError("Expecting value", s, err.value) from None
Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Dec 8 19:19:52 ip-172-31-14-148 OMR-Service: Restart OMR-Admin

It appears that I'm still successfully passing traffic thru the VPS but I'd like to fix this.

Please let me know if you need any other info to help.

[Ysurac]

I would need the VPS script release used (available in /etc/motd).
Check also that /etc/openmptcprouter-vps-admin/omr-admin-config.json

[donbecker]

"File "/usr/bin/omr-admin.py", line 773"

lines 772-773:

with open('/etc/openmptcprouter-vps-admin/omr-admin-config.json') as f:
omr_config_data = json.load(f)

So I suspect this is throwing an error trying to load the config file.

Checking the omr-admin-config.json file, I find it's completely empty, last edit 11/15. I've never modified this file.

Luckily I had a manual backup of the entire box dated 11/13, and I was able to copy the config over.

With 2 cellular internet connections and omr-admin service in reboot loop I was getting approx 60M down.

Once I updated the config, the service restarted and it has not restarted since.

I now get approx 96M down.

Back in the Luci panel for my local OpenMPTCP router (System -> OpenMPTCPRouter -> Status).... the VPS server in the diagram now shows green (was showing some error about admin connection before).

Any idea what might cause this config file to be completely empty?