Removing Cron Jobs from 3rd-party Add-Ons

[Martinski4GitHub]

Added code to remove cron jobs from 3rd-party Add-Ons right before ejecting the USB-attached drive and starting the actual F/W Update process. This is an effort to eliminate the possibility of a cron job starting to execute and accessing the USB drive filesystem.

[Martinski4GitHub]

@ExtremeFiretop,

Whenever you get a chance, please take a look at this PR and see how the new code works.

The variable debugAddOnCronJobsList can be set to true for testing/debugging purposes. When set, the script "/jffs/addons/MerlinAU.d/restoreAddOnCronJobs.sh" will be automatically created which will have all the cron jobs that were removed and can be used to restore them. I used this myself for testing & validation. For production releases, this "test/debug code" would be eventually removed.

[ExtremeFiretop]

@ExtremeFiretop,

Whenever you get a chance, please take a look at this PR and see how the new code works.

The variable debugAddOnCronJobsList can be set to true for testing/debugging purposes. When set, the script "/jffs/addons/MerlinAU.d/restoreAddOnCronJobs.sh" will be automatically created which will have all the cron jobs that were removed and can be used to restore them. I used this myself for testing & validation. For production releases, this "test/debug code" would be eventually removed.

Hey buddy, sorry about the delay...

I honestly have been so swapped I've been passing out right after work... I'm currently acting IT-04 (Manager) for the workstation engineering group for a bit and my stress levels are high.

Usually as soon as 4pm rolls around I'm passed out... I wake up at 8pm and work on dinner and picking up the girlfriend and then pass out again shortly again; that is if I haven't been working on work things.

The list is finally slowly getting smaller so all that to say I should have time tomorrow or this weekend to review this with you and discuss.

I'm still a little confused as to how the crons come back after a reboot if we remove them. It was on my to-do list to test that and also get an understanding of how that worked instead of just asking dumb questions I can figure out myself.

I also wanted to test to see if this change stopped the message from showing up on my router but I haven't had the chance to actually test on my primary router since I've been so busy, and I don't think my nodes are running much to ever show the message.

Hope you've been doing good, It's 12:30am and I'm just watching some Ancient Aliens on Plex to try and unwind lol and this is the latest I've stayed up all week! 😀

[ExtremeFiretop]

I took a second to look, and to what I understand cron jobs aren't persistent across reboots but I do see the cru commands for some things under the system startup scripts.

I'm assuming on reboot it re-adds the crons using these startup scripts like services-start

[Martinski4GitHub]

@ExtremeFiretop,
Whenever you get a chance, please take a look at this PR and see how the new code works.
The variable debugAddOnCronJobsList can be set to true for testing/debugging purposes. When set, the script "/jffs/addons/MerlinAU.d/restoreAddOnCronJobs.sh" will be automatically created which will have all the cron jobs that were removed and can be used to restore them. I used this myself for testing & validation. For production releases, this "test/debug code" would be eventually removed.

Hey buddy, sorry about the delay...

I honestly have been so swapped I've been passing out right after work... I'm currently acting IT-04 (Manager) for the workstation engineering group for a bit and my stress levels are high.

I know the feeling, bud!! There are some days, especially at the beginning & during the peaks of s/w development cycles, when the workload increases quite a bit, along with the pressure to perform & deliver, so at the end of the day I'm very tired and just want to rest & decompress. Often, a short nap before dinner is needed to get my "second wind." My wife takes short naps as well so we usually have late dinners, and by then we are a bit more relaxed & less prone to irritability from the stress of the job.

Anyway, no worries. Take your time. There's nothing urgent at this point.

Usually as soon as 4pm rolls around I'm passed out... I wake up at 8pm and work on dinner and picking up the girlfriend and then pass out again shortly again; that is if I haven't been working on work things.

The list is finally slowly getting smaller so all that to say I should have time tomorrow or this weekend to review this with you and discuss.

I'm still a little confused as to how the crons come back after a reboot if we remove them. It was on my to-do list to test that and also get an understanding of how that worked instead of just asking dumb questions I can figure out myself.

Remember that these routers don't have a full Linux distro. Their embedded OS is meant to run with very limited resources & under many constraints so a lot of the standard system files & configs that are always persistent on a full Linux distro are not found on the routers. This is the case with the system crontab file (e.g. /etc/crontab). On ASUS routers, the crontab file is not natively present so it must be generated on a virtual filesystem (i.e. RAM drive) during every reboot because it's not persistent across reboots. That's why all 3rd-party add-ons that want to create cron jobs must do so via the startup script hooks that RMerlin has provided (recall that we do this for our MerlinAU script). This is what regenerates the cron jobs at every reboot.

The bottom line is that when removing a cron job via the cru command only removes it temporarily and only for the current OS execution. On the next reboot, the cron job is regenerated via the startup script hook (as long as the script and the cron job hook have not been deleted).

I also wanted to test to see if this change stopped the message from showing up on my router but I haven't had the chance to actually test on my primary router since I've been so busy, and I don't think my nodes are running much to ever show the message.

It might not stop all ejusb messages because the new code prevents "future" cron job executions, but if one is already running at the same time that MerlinAU is also executing (say, both cron jobs happen to be scheduled to run at exactly the same day & time), the running process may still be accessing the USB drive filesystem when we're about to eject the drive. But I think the odds of that happening would be very small.

Hope you've been doing good, It's 12:30am and I'm just watching some Ancient Aliens on Plex to try and unwind lol and this is the latest I've stayed up all week! 😀

Have a good night, bud & take care.

[Martinski4GitHub]

I took a second to look, and to what I understand cron jobs aren't persistent across reboots but I do see the cru commands for some things under the system startup scripts.

I'm assuming on reboot it re-adds the crons using these startup scripts like services-start

Exactly!!
There are 2 startup scripts that are used (and recommended) to create cron jobs: services-start & post-mount

[ExtremeFiretop]

I know the feeling, bud!! There are some days, especially at the beginning & during the peaks of s/w development cycles, when the workload increases quite a bit, along with the pressure to perform & deliver, so at the end of the day I'm very tired and just want to rest & decompress. Often, a short nap before dinner is needed to get my "second wind." My wife takes short naps as well so we usually have late dinners, and by then we are a bit more relaxed & less prone to irritability from the stress of the job.

Anyway, no worries. Take your time. There's nothing urgent at this point.

Fake it till you make it buddy! I honestly feel a bit of "imposture syndrome". I just became IT-03 not long ago, and as you know I'm just in my late 20s. So I feel some pressure for sure, just trying to "Ride out the storm" and keep my head above water while I'm acting...

The main thing is that work has been "Gogogogo" right now that I've been passing out right after work, and that's the only reason I haven't tested this yet. I have to get myself informed on a bunch of stuff I wasn't previously informed on. It's hard to make decisions without the background knowledge and understanding so for a bunch of projects the team is handling I got to "get up to speed" so to speak and inform myself before I can make a decision.

Actually; similar to how in this PR I didn't fully understand at first so I just waited until I had the time to be informed. I'm not scared to admit I don't know everything and when I have a gap in the knowledge, but I also know I'm capable of digging up answers, it just takes time sometimes lol. Look who I'm telling, you probably know this all too well, you've always displayed the capability of digging up answers.

Remember that these routers don't have a full Linux distro. Their embedded OS is meant to run with very limited resources & under many constraints so a lot of the standard system files & configs that are always persistent on a full Linux distro are not found on the routers. This is the case with the system crontab file (e.g. /etc/crontab). On ASUS routers, the crontab file is not natively present so it must be generated on a virtual filesystem (i.e. RAM drive) during every reboot because it's not persistent across reboots. That's why all 3rd-party add-ons that want to create cron jobs must do so via the startup script hooks that RMerlin has provided (recall that we do this for our MerlinAU script). This is what regenerates the cron jobs at every reboot.

The bottom line is that when removing a cron job via the cru command only removes it temporarily and only for the current OS execution. On the next reboot, the cron job is regenerated via the startup script hook (as long as the script and the cron job hook have not been deleted).

See this is one of the many reasons your great Martinski, not only are you an amazingly impressive coder.
But your communication ability is really great. I know we joke lots about trying to make our messages clear to one another, but the truth is lots of people I know are extremely smart, but a bit confused when it comes to day to day interactions.

You just displayed a really amazing skill i wished more people had.. You heard my limitation of understanding, and you already knew where to start, you didn't need to start at absolute zero like I'm an baby, but you knew where the confusion came from. You then compared it to standard linux distro and pointed out the key differences that would trip someone like me up. Being able to identify this, explain it simply and clearly is a true skill; and I'm so happy to have someone like you to work with on this :)

It might not stop all ejusb messages because the new code prevents "future" cron job executions, but if one is already running at the same time that MerlinAU is also executing (say, both cron jobs happen to be scheduled to run at exactly the same day & time), the running process may still be accessing the USB drive filesystem when we're about to eject the drive. But I think the odds of that happening would be very small.

I really am liking this solution more and more as I get a better grasp of the theory, let me give it a test soon and then I'll either merge it provide additional feedback based on the test results.

[ExtremeFiretop]

Merged!

[Martinski4GitHub]

I know the feeling, bud!! There are some days, especially at the beginning & during the peaks of s/w development cycles, when the workload increases quite a bit, along with the pressure to perform & deliver, so at the end of the day I'm very tired and just want to rest & decompress. Often, a short nap before dinner is needed to get my "second wind." My wife takes short naps as well so we usually have late dinners, and by then we are a bit more relaxed & less prone to irritability from the stress of the job.
Anyway, no worries. Take your time. There's nothing urgent at this point.

Fake it till you make it buddy! I honestly feel a bit of "imposture syndrome". I just became IT-03 not long ago, and as you know I'm just in my late 20s. So I feel some pressure for sure, just trying to "Ride out the storm" and keep my head above water while I'm acting...

The main thing is that work has been "Gogogogo" right now that I've been passing out right after work, and that's the only reason I haven't tested this yet. I have to get myself informed on a bunch of stuff I wasn't previously informed on. It's hard to make decisions without the background knowledge and understanding so for a bunch of projects the team is handling I got to "get up to speed" so to speak and inform myself before I can make a decision.

Actually; similar to how in this PR I didn't fully understand at first so I just waited until I had the time to be informed. I'm not scared to admit I don't know everything and when I have a gap in the knowledge, but I also know I'm capable of digging up answers, it just takes time sometimes lol. Look who I'm telling, you probably know this all too well, you've always displayed the capability of digging up answers.

Remember that these routers don't have a full Linux distro. Their embedded OS is meant to run with very limited resources & under many constraints so a lot of the standard system files & configs that are always persistent on a full Linux distro are not found on the routers. This is the case with the system crontab file (e.g. /etc/crontab). On ASUS routers, the crontab file is not natively present so it must be generated on a virtual filesystem (i.e. RAM drive) during every reboot because it's not persistent across reboots. That's why all 3rd-party add-ons that want to create cron jobs must do so via the startup script hooks that RMerlin has provided (recall that we do this for our MerlinAU script). This is what regenerates the cron jobs at every reboot.
The bottom line is that when removing a cron job via the cru command only removes it temporarily and only for the current OS execution. On the next reboot, the cron job is regenerated via the startup script hook (as long as the script and the cron job hook have not been deleted).

See this is one of the many reasons your great Martinski, not only are you an amazingly impressive coder. But your communication ability is really great. I know we joke lots about trying to make our messages clear to one another, but the truth is lots of people I know are extremely smart, but a bit confused when it comes to day to day interactions.

You just displayed a really amazing skill i wished more people had.. You heard my limitation of understanding, and you already knew where to start, you didn't need to start at absolute zero like I'm an baby, but you knew where the confusion came from. You then compared it to standard linux distro and pointed out the key differences that would trip someone like me up. Being able to identify this, explain it simply and clearly is a true skill; and I'm so happy to have someone like you to work with on this :)

Thanks for the praise and the kind words.

It might not stop all ejusb messages because the new code prevents "future" cron job executions, but if one is already running at the same time that MerlinAU is also executing (say, both cron jobs happen to be scheduled to run at exactly the same day & time), the running process may still be accessing the USB drive filesystem when we're about to eject the drive. But I think the odds of that happening would be very small.

I really am liking this solution more and more as I get a better grasp of the theory, let me give it a test soon and then I'll either merge it provide additional feedback based on the test results.

I hope the solution works for you as well as it did for me.