Upgrading Guide

Day-to-day updates

MailCleaner machines run the following cron task as root:

0,15,30,45 * * * *  /usr/mailcleaner/scripts/cron/mailcleaner_cron.pl > /dev/null

Data feeds

For Enterprise Edition, this script has the following child scripts:

• /usr/mailcleaner/bin/check_update.pl - looks for out-of-schedule updates via exec.sh scripts (Enterprise Edition only). This allows us to force service upgrades or take other critical actions as soon as solutions are available.
• /usr/mailcleaner/bin/fetch_clamspam.sh - fetches the latest clamspam signatures.
• /usr/mailcleaner/bin/fetch_rules.sh - fetches the latest spamc and newsl rules.
• /usr/mailcleaner/scripts/cron/update_antivirus.sh - fetches clamd signature updates, including third-party signatures.

For Community edition, the freshclam downloads only the default clamspam and clamav signatures, and spamc and newsl rules are downloaded nightly as part of the git pull update.

There is a dedicated script to fetch bayes data. Enterprise Edition will see updates around a couple of times per day. Community Edition will get a monthly snapshot.

Nightly updates

There is also a nightly cron task:

30 22 * * *  /root/Updater4MC/updater4mc.sh

which primarily does 2 things:

• Performs a git pull on /usr/mailcleaner/ to apply the latest patches from this repository. This will deploy almost all new features which don't require changes to the database or running of system commands. The latest patch hash is shown after the '-' in the 'patch level', eg: '91-3645b23'
• Runs scripts from it's own repository (after pulling) in /root/Updater4MC/updates/. These scripts will perform actions which cannot reasonably be done by git, including updating packages, applying patches to files outside the git repository, etc. For significant update scripts, the version will be bumped to that update's publication date. Regardless, the latest applied update script number will appear before the '-' in the patch level, eg: '91-3645b23'

You can manually run this script to have it run out-of-schedule, or modify the cron task to always run at at time other than 22:30. However, you should make sure that it does not conflict with the Configuration->General settings->Periodic tasks->Daily task run at time. The updater script will stop and start all services if there were any changes in git or any update scripts, so this would interrupt the sending of quarantine reports. The updates typically take less than 10 minutes. Summary reports usually only take a couple of minutes, but could potentially take more than hour on a machine with 10's of thousands of reports. Since the default report time is 00:00 and this is rarely changed to be earlier in the evening, 22:30 was selected as a safe default time.

Major version updagrades

WARNING: As noted elsewhere, MailCleaner-Next is still too early in developement to use in production. Migration procedure is also not fully tested and documented. If you are going to Alpha test MailCleaner-Next, we would ask you to work from a new installation, without migrating your configuration, until this procedure is verified.

Upgrading from MailCleaner (Jessie)

Note: MailCleaner Next is still in Alpha status, it should not be used for production.

In order to have the best roll-out possible for new appliances, we are seeking Alpha testers for the Community release.

Why Upgrade?

The current release is based on Debian Jessie, which is now EOL. By upgrading, you will gain access to new, supported packages as well as the following benefitial changes (checks indicate completed changes):

• Upgrades to core package versions:
  • Apache2 => 2.4.57
  • Exim => 4.98 (recompiled Debian package with extra features)
  • PHP => 8.2
  • Perl => 5.36
  • MariaDB (replaces MySQL) => 10.11.3
  • SpamAssassin (spamd) => 4.0.0
  • ClamAV => 1.0.3
  • Fail2Ban => v1.0.2
  • Many others from upstream Debian Bookworm repositories
• Adoption of mostly upstream Debian versions of packages to provide easier upgrades and better security guarantees going forward.
• Adoption of systemd for service management (mostly complete). Will provide faster startup times and better reactivity to service failures.
• Changing to distinct user account for each service to isolate permissions and limit damage in the event of an exploit.
• Migration from openssl to gnutls inside exim for better cipher compatibility and additional features (eg. DANE)
• Migration from iptables to nftables (with ufw front-end) for improved firewall usability and performance
• Complete overhaul of WebUI libraries for new PHP version and Zend framework
• Lightly updated WebUI template with backwards-compatibility for custom templates.
• Simplified first-start installation/configuration script.
• Improved back-end image building tools for continuous development. Will allow for faster initialization of new machines. Will potentially allow for additional image formats and installation methods (TBD).

Other upgrades which will be rolled out down the road to the Next release only:

• (Optional) New OCR module using TessoractOCR
• (Enterprise) Enhancements to ESET Anti-Virus integration
• (Enterprise) New MachineLearning module

How to Upgrade

Upon the official release, there will be a simplified, (semi-)automated migration process. Until then, you will need to run the steps below manually. Even when the official release is complete installing a new appliance will always be necesasry. There will be no in-place upgrades of an existing appliance. The new machine will be referred to as new-server below.

Ensure that your existing appliance is up-to-date by checking that /root/Updater4MC/updater4mc.sh completes without errors. This will be referred to as old-server below.

We are working on a single-command migration script to ingest and upgrade your existing data, but for now it is a manual process:

• Stop MailCleaner on old_server:

/etc/init.d/mailcleaner stop

• and new_server:

systemctl stop mailcleaner

• Copy to the database and all quarantines from old_server to new_server:

rsync -av --delete --exclude 'spool/tmp' --exclude 'run' /var/mailcleaner/* root@new_server:/var/mailcleaner

• Copy config file /etc/mailcleaner.conf from old_server to new_server

rsync -av --delete /etc/mailcleaner.conf root@new_server:/etc/mailcleaner.conf

• Copy your network configuration

rsync -av --delete /etc/network root@new_server:/etc/network

• Take old_server offline

• Restart networking on new_server

systemctl restart networking

• Now restart MailCleaner on new_server. Beware of IP address and network configuration.

systemctl start mailcleaner

• Repair and resync databases:

/usr/mailcleaner/bin/check_db.pl -m --myrepair
/usr/mailcleaner/bin/check_db.pl -s --myrepair
/usr/mailcleaner/bin/resync_db.sh

• Verify that everything is working by accessing the WebUI at the address of old_server (which now belongs to new_server), test sending/receiving emails, etc. In the event of a failure, you can take down new_server and then bring old_server back up so that it reclaims it's IP.'

Other considerations

There may be some additional files that you need to copy if you have customized your configuration. Specifically, you may have a custom set of template files in /usr/mailcleaner/www/user/htdocs/templates and /usr/mailcleaner/templates/*.