Cronicle start problem - PID incorrectly attributed - EASY FIX

[cliffshaw]

Summary

TLDR: Cronicle wrongly thinks it's running when it's not.

This screenshot says it all. Cronicle thinks it's running because another process is running that has the stale PID.

This machine has only been live for 3 hours due to a reboot. Yet the PID file it wrote was from yesterday.

Steps to reproduce the problem

Steal a PID from any running job. Cronicle will not check whether this PID is stale and actually taken up by another process.

Operating system and version?

Ubuntu 22.04

Node.js version?

20 something.

Cronicle software version?

0.9.53

Are you using a multi-server setup, or just a single server?

Multi

[jhuckaby]

Thank you for reporting this. I'll fix it ASAP.

[cliffshaw]

With pleasure... brilliant creation and thank you for your efforts on this.

[jhuckaby]

Fixed in v0.9.57.

I just want to point out that if you encounter a situation where the Cronicle PID file is hanging around on your filesystem, and it exists when it tries to start back up, this is VERY BAD. It's a situation which should "never" occur during normal operations, because Cronicle ALWAYS deletes its PID file on normal shutdown. So a stale PID file can only occur is if Cronicle crashed, or if the server / VM / container died while it was running. This is a terrible situation, because Cronicle's internal database can become corrupted. (

(Note that it ATTEMPTS to rollback the database on the next startup after an unclean shutdown, but it's generally a situation you don't want to encounter.)

So while the "bug" is now fixed in v0.9.57 and Cronicle will now correctly sniff the PID process and detect if it is really Cronicle or some other process, the fact that you have a stale PID file that now points to your [nfsd] daemon means that your Cronicle crashed horribly, or your server powered off unexpectedly, or you had a kernel panic. Either way, this makes Cronicle very sad, similar to powering off a database while it's writing records.

Just wanted to point that out. Thanks again for the issue report!

[cliffshaw]

Thanks for the speedy fix on this.

Just to make you aware, this was a Worker machine and not the Cronicle server itself. Unexpected reboots are part of life so I'm glad to hear that you've got some protections in place if the server were to unexpectedly reboot.

[jhuckaby]

Ah good, I'm glad it was just a worker. Those don't write any data to the internal database, so you're good.

Yup, unexpected reboots do happen 😊