Adjust timing of daily apt tasks

[rmol]

Status

Work in progress (timing needs to honor sdconfig options)

Description of Changes

Fixes #5851.

Adjusts the timing of the apt-daily systemd timer so that updates are downloaded about every three hours.

Adjusts the timing of the apt-daily-upgrade timer so that the updates are applied at a random time between 0400 and 0500.

Testing

• make build-debs-focal && make staging-focal
• On each server, check that:
  • An override.conf file is present in both of:
    • /etc/systemd/system/apt-daily.timer.d
    • /etc/systemd/system/apt-daily-upgrade.timer.d
  • The CalendarOn and RandomizedDelaySec entries in those files match what's reported by:
    • systemctl show apt-daily.timer
    • systemctl show apt-daily-upgrade.timer

Deployment

This should ensure that updates are more reliably noticed and applied every day.

Checklist

If you made changes to the system configuration:

• Configuration tests pass

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

Choose one of the following:

• I have opened a PR in the docs repo for these changes, or will do so later
• I would appreciate help with the documentation
• These changes do not require documentation

[zenmonkeykstop]

I'm trying this out applied manually as part of RC3 testing (don't wanna wait overnight to see if it's applied) and it does not appear to trigger updates or upgrades. I think /etc/apt/apt.conf.d/20auto-upgrades would have to change as well to change the APT::Periodic::* values to "always".

[rmol]

That's surprising. All it does is change the schedule of the timers. I don't know why it would require other configuration changes.

[zenmonkeykstop]

https://wiki.debian.org/UnattendedUpgrades describes the timer format in a bit of detail but it's describing a case where you want to move the daily run from 6,1800 with a randomized delay of 12 hours to 1am precisely, and what actually happens is that the timer triggers apt-daily.service, which checks the delta between now and its last runtime against the interval defined in the APT::Periodic::* values above, and only actually does something if said delta is bigger than the interval.

So! in order to be able to trigger updates and upgrades (which are 2 separate timers) arbitrarily, you'd need to set those APT::Periodic:: values to "always", which appears to be a keyword meaning "always run when asked", which the developers probably had to add because "0" can't mean "0 interval" as it already means "don't ever run".

[conorsch]

you'd need to set those APT::Periodic:: values to "always", which appears to be a keyword meaning "always run when asked"

@zenmonkeykstop Where are you seeing that "always" is a permissible value? For APT::Periodic::Unattended-Upgrade my understanding is that the value must be an integer, where 0 is disabled, and any positive int is the frequency, in days, that the update should be run.

[zenmonkeykstop]

Docs are a little sparse, it's true. The apt.conf man page will redirect you to /usr/lib//apt/apt.systemd.dailywhich documents the N for N days or 0 for off behaviour. But if you grep for "always" in said script you will find the check_stamp function that does the actual interval checking, with logic to short-circuit the check if the interval is set to "always".

Most apt.systemd.daily operations will only run if the interval since they last ran is greater than the defined interval in the APT::Periodic:: settings, so if you want to run them arbitrarily (say for testing purposes) you need to override that behaviour.

[zenmonkeykstop]

On further review/discussion - we can get the desired test behaviour (rerunning apt.systemd.daily on demand) by removing timestamp files in /var/lib/apt/periodic - so actually yes, we don't need the changes in /etc/apt/apt.conf.d/, just the timer settings above. That'll get us a reliable run - my preference would be (reboot time - 1hr with a 30min fuzz) for the update and (reboot time - 30min with a 30min fuzz) for the upgrade, but do what works!

[rmol]

closing in favor of #5855