CHANGELOG

v1.5 | Sep 19 2015:
[New] added -f|--file-list CLI option to allow user supplied run-time file list for scanning
[New] added -i|--include-regex CLI option for run-time path/file inclusion based on posix-egrep regular expressions
[New] added -x|--exclude-regex CLI option for run-time path/file exclusion based on posix-egrep regular expressions
[New] added support for custom md5/hex signatures with preservation across signature and version updates, files located at:
      sigs/custom.md5.dat
      sigs/custom.hex.dat
[New] custom signatures perform run-time conversion into clamscan compatible format on systems that use clamscan engine
[New] new md5 signature format (md5v2) now includes file size that an md5 hash was derived from in format of:
      hash:filesize:signame
[New] added support for custom cleaner rules to be executed on clean events, file name format of
      "clean/custom.signame"; rules are preserved across signature and version updates
[New] added support for clam(d) engine when running in inotify monitoring mode
[New] added URL import feature for global configuration overrides using import_config_url variable in conf.maldet
[New] added URL import feature for user custom signatures using import_sigs_md5_url & import_sigs_hex_url variables in conf.maldet
[New] added set of defined exit codes for errored exits(1), successful runs with hits(2), successful runs with no hits(0)
[New] added uninstall.sh script to maldetect installation path
[New] added md5 hash verification of signature and version update downloads
[New] added scan_cpunice option to control CPU priorty value of all scan operations such as find, clamscan etc.. (default 19)
[New] added scan_ionice option to control IO priority value of all scan operations such as find, clamscan etc.. (default 6)
[New] added autoupdate_signatures/autoupdate_version options to control daily cron based signature/version updates
[New] added autoupdate_version_hashed option to control validating hash of maldet executable against upstream version
[New] added support for virtualmin to cron.daily scans
[New] added support for ispmanager to cron.daily scans
[New] added support/detection of clamdscan to leverage memory preloaded signatures and multi-threaded scanning
[New] added scan_find_timeout option which controls the maximum execution time, in seconds, for the find command to generate a file list
[New] added scan_ignore_root option to exclude root owned files from scans
[New] added scan_ignore_user and scan_ignore_group options which allow for the exclusion of specified user and group names from scans
[New] added scan_export_filelist option allowing for daily scan of recent added/modified files to be exported to a static path
[New] added sourcing of of conf.maldet.cron into the cron.daily task which allows for cron specific configurations
[New] added inotify_reloadtime which controls the time at which inotify watcher will reload LMD configuration data
[New] added support for comma space (,) path list on CLI
[New] added reload option for monitor mode to invoke forced configuration reload (-m|--monitor reload)
[New] added maldet init script for monitor mode with $default_monitor_mode conf.maldet variable
[New] added usage of /etc/sysconfig/maldet for configuration of monitor mode init/systemd options, overrides conf.maldet
[New] added support for 'cpulimit' usage, when installed, through scan_cpulimit and inotify_cpulimit configuration variables
[Change] increased randomness of quarantine temporary file names
[Change] added atime,mtime,ctime since epoch values into quarantine info files
[Change] monitor mode now supports all existing ignore options as well as enforcing minimum/maximum file sizes
[Change] monitor mode now supports hot configuration reloads by touching reload_monitor under installation path (e.g: touch /usr/local/maldetect/monitor_reload)
[Change] modsec.sh has been renamed to hookscan.sh and more generic hook based scanning conventions set
[Change] hookscan.sh will now autodetect if clamdscan is running and perform scans through clamd when appropriate
[Change] hookscan.sh will now provide more verbose output on malware hit events
[Change] hookscan.sh now explicitly disables scanning of temporary paths, ensuring only requested file/paths are scanned
[Change] install.sh now gracefully handles upgrades when monitoring mode is enabled by restarting monitor mode
[Change] improved handling of single file scans which should now behave as expected
[Change] explicitly removed the inclusion of tmpdir paths during single file scans
[Change] automagically remove empty lines from ignore files
[Change] reordered configuration file, expanded on variable descriptions, overall attempt to simplify/streamline conf.maldet
[Change] installer symlinks LMD signatures into known/existing ClamAV paths to ensure signatures are loaded into memory by clamd
[Change] installer issues SIGUSR2 to any running clamd processes to force reload of signature databases
[Change] cron.daily signature updates issue SIGUSR2 to any running clamd processes to force reload of siganture databases
[Change] cron.daily signature/version updates sleep random interval 1-999 secs before contacting upstream rfxn.com servers to reduce cdn load
[Change] modified clamscan database path checks to support cPanel >=11.40 RPM clamAV connector RPM's
[Change] modified location of statistical data files from tmpdir to sessdir making tmpdir a stateless path that can be purged at anytime
[Change] when clamscan engine is enabled scan_max_filesize value is now set dynamically based on the largest known file in the md5v2 signature set
[Change] modified e-mail based alerts to source from an e-mail template file at .email.template
[Change] clamscan execution command logged to logs/clamscan_log to make debugging clamscan errors easier
[Change] clamscan stderr/stdout output now pipes to logs/clamscan_log and if clamscan returns an error code (2), flag with an appropriate
         error message to check the clamscan_log file for more details
[Change] ambiguous variables renamed for better consistency and more logical naming conventions, documented in CHANGELOG.VARIABLES
[Change] modified sessionid values to derive from YYMMDD instead of MMDDYY and adjusted human readable report START/END date to include year value
[Change] modified view_report output to sort output on unix time of scan start times
[Change] signature updates now download as a single file tgz to reduce bandwidth usage and request load on upstream cdn
[Change] modified signature update function for additional error checking and better handling of zero sized signature downloads
[Change] modified version update function for additional error checking and better handling of zero sized file downloads
[Change] modified '-e|--report list' output include total files scanned, hits and cleaned results, reversed output order and
         consistent column spacing (column -t)
[Change] moved tlog executable out of inotify/ path, changed inotify_log path to logs/, removed inotify directory
[Change] created logs/ path, moved event_log path to logs/
[Change] modified previous wget timeout values of 3s timeout & 3 retries to 5s timeout & 3 retries
[Change] wget timeout and retry attempts are now configurable through internals.conf wget_timeout & wget_retries variables
[Change] removed file type check on native LMD stage2 hex scanner which was part of legacy code and no longer needed
[Change] removed verbose progressive scan output for native LMD scanner as performance penalty was unreasonable
[Change] replaced usage of tmpwatch with find in cron.daily for temporary path pruning
[Change] removed internals.conf from version check hashing for installation version updates (-d|--update-ver)
[Change] cron.daily now tests for directadmin and scans appropriate user domain paths
[Change] directory checkout uploads limited to maximum of 50 files
[Change] added tmpdir_paths option to explicitly scan known temporary (world-write) paths on all scan types
[Change] updated example ModSecurity rule in README file for compatibility with ModSec 2.7 which now requires
         every rule, even hooks, to have a rule ID
[Change] -r|--recent scan now uses mtime and ctime, instead of just mtime, to find recently changed/modified files
[Change] LMD v1.4.2+ will now use the new md5 v2 signature format and make direct requests on signature
         updates to the appropriate upstream file (md5v2.dat); old format, md5 v1, preserved in signature
         releases for compatibility of pre-1.4.2 releases
[Change] modified hexfifo.pl & hexstring.pl to accept user supplied value for path to hex signature file
[Change] install.sh now deletes LMD backup installation copies older than 30days
[Change] references to www.rfxn.com for remote signature and version updates now query cdn.rfxn.com
[Change] cleaner rules are now executable scripts in which infected files are passed as an argument ($1)
         allowing for a diverse set of cleaner rule options apart from the previous sed only setup
[Change] converted current cleaner rules to new executable scripts format
[Change] checkout uploads now store malware in the filename format of (hostid is an anonymous md5 identifier):
         $hostid.$RANDOM.$filename.[ascii|bin]
[Change] inotifywait from inotify-tools is no longer packaged with LMD, it should be downloaded in binary or
         source form from:
         https://github.com/rvoicilas/inotify-tools/wiki/
         binary versions are also available from dag repo at:
         http://pkgs.repoforge.org/inotify-tools/
[Change] internals.conf will now attempt to detect the path to inotifywait from $PATH
[Change] inotify max_user_watches was static set to 128, now configurable with inotify_user_watches
[Change] inotify values for max_user_instances|watches will first be checked and only modified if the existing
         values are lower than what maldet requires
[Change] modified error output for missing inotifywait to display URL to inotify-tools github page
[Change] modified default scan_hexdepth value to 65k as a result of improved scan efficiency in native scanner engine
[Change] added backwards compatibility for all pre-v1.5 configuration options however they should be considered deprecated and will be removed in the future
[Change] expanded on EICAR test signature support for native LMD scanner engine to better facilitate testing of functional installed signature set
[Change] added scan/find elapsed execution time values to scan report and cli output
[Change] relocated internal files into $inspath/internals/
[Change] created generic clean_exit() function to handle file cleanups on all fatal exist and replaced many random rm -f calls with it
[Change] moved all pre/post actions into a prerun() and postrun() functions
[Change] moved statistical logging to record_hits() function
[Change] quarantine() function borrows stat file data from record_hits to reduce calls to stat
[Change] more extensible cleaner rules with additional input arguments:
         $1 file path, $2 signame, $3 owner.group, $4 file_chmod, $5 file_size, $6 file_md5
[Change] added additional fields file_size and file_md5 to quarantine info file
[Change] added caching support for import_config_url with import_config_expire to control expiry interval
[Change] stricter handling of variable definitions which contain dynamic variable values
[Change] modified daily cron recent range from 2 to 1 as mtime/ctime values are n*24h, as such value of 1 is equal to two days
[Change] modified daily cron to use comma spaced path lists instead of multiple maldet executions
[Change] changed quarantine malware cleaning default value to 0
[Change] use of clamav engine output statement now more verbose
[Change] previously LMD only linked clamav signatures into clamav data paths on install, this is now done after each signature update
[Change] maldet.sh init script exites code 1 on status check when maldet monitor mode is not found running
[Change] monitor mode now invokes every 15 seconds, legacy installations will preserve 30 second cycle timing
[Change] modified shebang to use env bash for portability
[Fix] when clamdscan was running as a non-root user, would generate lstat errors for all file find results leading
      to potential false positive hits/quarantines
[Fix] the permissions of the $tmpdir path can cause clamd when running as a non-root user to fail on startup due
      as a result of lstat errors on the custom user signature files stored under $tmpdir
[Fix] clamd.conf configurations containing FollowDirectorySymlinks/FollowFileSymlinks set to false results in the
      rfxn.* and lmd.user.* links causing clamd startup failures; corrected by updating clamav_linksigs() to copy
      signatures into clamav data paths instead of linking them
[Fix] inotify monitor execution now properly passes ionice configuration value
[Fix] monitor_paths was not being preserved on version updates
[Fix] record_hit() was not being invoked outside of clamscan based events
[Fix] monitor.pid file would potentially have an incorrect pid written to it on each execution of monitor_check()
[Fix] quote syntax error in scan.etpl
[Fix] help output for -k|--kill-monitor incorrectly referred to --kill instead of --kill-monitor
[Fix] inotify_user_instances was defined in internals.conf incorrectly as inotify_user_watches
[Fix] tlog executable was not being set +x during installation
[Fix] install.sh was attempt to create default event_log while the parent directory did not yet exist
[Fix] invalid find expression was causing find to return directory paths on recent scans
[Fix] OSTYPE env checking was not properly matching on all FreeBSD versions
[Fix] renamed alert() to genalert() to avoid builtin function conflict on Ubuntu
[Fix] corrected -r|--recent scan mode trap on SIGINT (CTRL+C) not calling trap_exit() for cleanup actions
[Fix] modified native LMD scanner to better leverage bash internal field separator for handling of paths with spaces
[Fix] modified all calls to system executables to use paths derived from $PATH
[Fix] suppressed ignore signature count being displayed when calling with --modsec
[Fix] set modsec.sh to use /bin/bash as interpreter instead of /bin/sh for compatibility
[Fix] removed MAILTO & SHELL variables from crons which were causing crond 'bad minute' errors on some systems
[Fix] quoted extension values from ignore_file_ext input to provide consistent behavior
[Fix] added trailing slash '/' to all cron.daily scan (find) paths to properly handle symlinked paths
[Fix] install.sh now links LMD clamav signatures into all clamav data paths it finds instead of just the first
[Fix] clean() function was improperly exiting after first file hit clean attempt and ignoring all other hits
[Fix] set interpreter in uninstall.sh to /bin/bash instead of /bin/sh for better compatibility
[Fix] modified psa scan paths to pull in top level and sub domain content
[Fix] corrected invalid matching against clamdscan binary when clamd was not available

v1.4.2 | Feb 25th 2013:
[New] detection and alerting of libkeyutils root compromised libraries
[Change] cron.daily now tests for directadmin and scans appropriate user domain paths
[Change] removed temporary paths /var/tmp, /tmp, and /dev/shm from cron.daily which are
         now added explicitly to all scanning paths / modes

v1.4.1 | Nov 20th 2011:
[Change] rfxn.com ftp server moved and anonymous FTP checkout uploads changed
[Change] modsec.sh force sets clamav_scan=0 as native LMD scanner engine is faster on
         single / small file sets
[Fix] correct plesk if statement added to to daily scan cronjob
[New] added -U|--user to force execution under defined user, ideal for restoring user
      quarantined data or viewing user reports
      e.g: maldet --user nobody --report
      e.g: maldet --user nobody --restore 050910-1534.21135
[New] added public_scan variable to conf.maldet to control enabling of public mode
      scanning, disabled by default
[New] added cron.d/maldet_pub cronjob to populate public user paths when public mode
      scanning is enabled; does nothing when disabled
[Change] README file updated, had fallen behind on CLI usage help details
[New] added -co|--config-option for defining conf.maldet options on the CLI
[Fix] README, COPYING.GPL and CHANGELOG are now properly copied into the installation path
[Fix] version header in config import template was incorrect
[Fix] value of email_ignore_clean is now properly preserved on version upgrades
[New] added modsec.sh to allow for easy calls from mod_security2 inspectFile hook
[Change] autodetect executing uid and define public mode scanning variables
[New] added public mode scanning which redefines tmpdir, sessdir, quardir to pub/username/
      directory tree for user initiated (non-root) scans
[Change] installation permissions changed to 644/755 for public mode support
[Change] revised (gz)base64 rules to be more specific thus reducing false positives
[Fix] tlog was set to use /bin/sh which breaks usage on systems with default shells other
      than bash

v1.4.0 | Apr 17th 2011:
[Change] default editor now inherited from $EDITOR
[New] clamav signatures update through sigup(), -u|--update
[New] cleaner rules update through sigup(), -u|--update
[Change] added error checking for missing or corrupted signature files
[Fix] monitor_cycle() now properly trims inotify_log
[Fix] version dates in CHANGELOG for 1.3.8 -> current had 2010 instead of 2012
[New] added -b|--background flag to execute scans in background
[Change] cron.daily now uses the -b flag for background scanning
[Change] wget calls now use the --referer option to broadcast local LMD version
[Fix] replaced stray references of absolute install path with the install path variable
[New] stage2 (HEX) scanner now supports use of named pipe (FIFO) for passing file hex contents,
      enabled by default, provides better performance with larger depth anlaysis of files
[New] added hex_fifo_scan & hex_fifo_depth variables to conf.maldet for fifo hex scanning
[Change] -c|--checkout now supports directory paths
[Change] -r|--scan-recent and -a|--scan-all now supports single file scans
[Fix] replaced absolute path to nice on inotifywait exec to which located variable value
[Change] added error checking for all internally required binaries e.g: wget, find, od etc...
[New] detection of ClamAV clamscan binary and usage as default scanner engine; when detected,
      clamscan is executed on scan file lists using rfxn.com LMD clamav-compat sigs
[Change] added OSTYPE check for differentiating md5 sum binaries on linux and FreeBSD
[Change] added OSTYPE check on monitor mode to disable on FreeBSD, pending kqueue alternative
         to inotifywait
[Fix] revised od flags for FreeBSD support
[Fix] ignore_inotify now properly interprets extended posix regexp as ignore parameters
[Change] added sample ignore values into ignore_inotify along with sane defaults to
         ignore common noisy files
[New] added statistical analysis for string length to identify threats based on the longest
      uninterrupted string within files, common of obfuscated code (e.g: base64, gzip etc...)
[New] added string_length_scan & string_length variables to conf.maldet for strlength scanning
[Fix] ignore_file_ext has been readded and now correctly ignores files based on extension
[Fix] replaced absolute path to mail with which located variable value
[Fix] lmdup() now properly errors out when rfxn.com web server is offline
[New] added clamav_scan variable to conf.maldet to toggle clamscan detection
[New] Full compatibility under the following distros has been verified :)
        - FreeBSD 9.0-CURRENT
        - RHEL/CentOS 5.6
        - RHEL 6
        - Fedora Core 14
        - OpenSuse 11.4
        - Suse Linux Enterprise Server 11 SP1
        - Ubuntu Desktop/Server 10.10
        - Debian 6.0.1a
[Change] updated README file for new features & vars, sample ignore usage, revised features
         and updated cymru hash statistics
[Fix] relaxed grep for inotify sysfunctions to just inotify_ on System.map file
[New] can now pass list to -e|--report to view all available scan reports
      e.g: maldet --report list
[New] can now pass an e-mail address to -e|--report to email a specific report
      e.g: maldet --report SCANID user@domain.com
[New] added email_ignore_clean variable to suppress alerts where all hits are cleaned

v1.3.9 | Mar 16th 2011:
[Fix] ignore files are now properly imported on version updates
[Change] cron.daily now checks for version updates
[Fix] hexdepth greater than 65Kb caused an 'argument list too long' error with hexstring.pl
      which would fail-clean any malware on hex checks
[Change] default hex depth increased to 61440 as there was an increasing margin of error on
         missing threats due to them falling outside the default hexdepth; will add offset
         option to signatures in the near future
[Change] updated cymru hash statistics in README file

v1.3.8 | Jan 30th 2011:
[Fix] revised inotify tracking log file to properly rotate instead of growing indefinitely

v1.3.7 | Nov 27th 2010:
[Fix] package ownership at some point got set to uid 501 instead of root
[Fix] daily cronjob now checks ps output for inotifywait proc instead of pidof
[Fix] monitor mode users would exit prematurely if a user home path did not exist
[Fix] a file hijacking race condition existed with quarantine mode restore function
[Fix] inotify max_user_instances value was being set to a value that would cause inotifywait
      to fail

v1.3.6 | May 21st 2010:
[Fix] restore option will now handle session based restores for quarantines that
      were manually invoked with -q|--quar SCANID
[Fix] session data gets recreated if it disappears during scan

v1.3.5 | May 18th 2010:
[Fix] tlog now handles data that logged between 0bytes and first wake cycle
[Fix] monitor_check now properly handles CREATE,ISDIR events
[Change] --alert-daily|weekly alerts have been changed similar to manual alerts
[Fix] cleaner was not properly running on monitor_check calls to scan files
[Fix] quar_suspend was not properly running on monitor_check calls to quar()
[Change] monitor tracker files now pass through trim_log to avoid oversizing
[Fix] monitor_check now properly handles path names with spaces
[Fix] monitor_check was throwing nx file/directory error for monitor.pid
[Fix] older bash versions were having trouble with the [[ =~ ]] regexp search
[Change] set all script files from shebang/bin/sh to shebang/bin/bash
[Change] --alert-daily|weekly will now only send alerts if hits were found
[New] -d|--update-ver now compares file hashes to determine update status
[Fix] suspend events were not properly being added to monitor alerts
[Change] all alerts have had spacing changes to make them more readable
[Fix] signature names now properly list for daily|weekly alerts hit list
[Fix] monitor_check will now recursive monitor newly created directories
[New] monitor daily|weekly alerts now save as a pseudo scan report with SCANID
[Fix] monitor reports now generate properly when quar_hits=0

v1.3.4 | May 16th 2010:
[Fix] cleaner function was not properly executing under certain conditions
[Change] additional error checking/output added to the cleaner function
[Change] default status output of scans changed for better performance
[New] added ignore_intofiy for ignoring paths from the monitor service
[Change] updated ignore section of README
[Fix] backreference errors kicking from scan_stage1 function
[New] -d|--update-ver option added to update installed version from rfxn.com
[Change] updated short and long usage output for update-ver usage
[Fix] -k|--kill-monitor now properly kills only the inotifywait/monitor pid's
[Fix] monitor_cycle function now correctly stores its pid in the pidfile
[Fix] files with multiple events in the same waking cycle are only scanned once
[Change] install.sh now symlinks maldet executable to /usr/local/sbin/lmd

v1.3.3 | May 15th 2010:
[Fix] quarantined files were not properly dropping owner
[New] signature based, rule driven, cleaner component added
[New] base64.inject cleaner rule
[New] gzbase64.inject cleaner rule
[New] -n|--clean SCANID option added to batch clean scan all files from a scan
[Fix] made default install file/path permissions more strict (750/640)
[New] install.sh now preserves conf.maldet settings
[New] install.sh now links backups of old installation to INSTALL_PATH.last
[Fix] install.sh now properly imports session data from previous install
[New] -s|--restore can now take a SCANID to batch restore all files from a scan
[Change] improved the layout of conf.maldet; more scan options and commenting
[New] added quar_susp_minuid option for suspend user minimum user id
[Fix] inotify monitor now properly acts on MODIFY,MOVE_TO,MOVE_FROM states
[Change] inotify monitor now can take a list of paths or file for path input
[Change] inotify monitor now has no default use, must specifiy USER|FILE|PATHS
[Change] revised short and long usage output for new options/usage changes
[Change] inotify monitor now spawns only one process for all monitored paths
[Change] inotify monitor sets max_user_instances to processors*2
[Change] inotify monitor sets max_user_watches to inotify_base_watches*users
[Change] migrated all inotify options from internals.conf to conf.maldet
[New] added inotify_base_watches to conf.maldet for max file wathces multiplier
[New] added inotify_nice to conf.maldet for run-time prio of inotifywait
[New] added inotify_webdir to conf.maldet for html/web root only monitoring
[Change] extensive format change to README
[Change] rewrote inotify section of README to reflect the many changes
[New] added cleaner section to README
[Change] -q|--quarantine now calls cleaner if quar_clean=1
[Change] -n|--clean can now do in place cleaning without quarantine
[Fix] cleaner function was not properly executing under certain conditions

v1.3.2 | May 13th 2010:
[New] added ignore files: ignore_paths , ignore_sigs 
[Change] ignore_sigs is processed as a pre-scan component before all scans
[Change] revised README file to include details on new ignore options
[Change] signature counts now displayed pre-scan and post-update
[Change] install.sh now runs --update after installation
[Fix] -p|--purge now properly clears session state data
[New] added [ SIGNATURE UPDATES ] section to README file
[Fix] some functions were referencing full paths instead of the variable equivs

v1.3.1 | May 12th 2010:
[Fix] typo in report command eout()
[Fix] cron.daily tmpwatch on invalid path
[Change] redirect stdout to /dev/null on tmpwatch calls in cron.daily
[Change] better commented cron.daily actions
[Change] cron.daily scans will now hit /home*/*/public_html on non-ensim systems
[Change] inotify monitor now properly handles any user homedir paths
[Fix] sigup will now download full signature set when no sigs are found local
[Fix] rewrote 17 signatures that would never match due to hexdepth restrictions
[Fix] removed some HEX signatures derived from ClamAV that would never hit
[Change] files must now be >32bytes to be included in search results
[Change] search results default to a max directory depth of 15
[New] added vars for minfilesize and maxdepth scan options
[Change] updated inotifywait to v1.3.6, statically linked binary
[Info] signature RSS and XML data sources added, see:
http://www.rfxn.com/signature-updates-rss-feed/
[Info] LMD now has a homepage on rfxn.com:
http://www.rfxn.com/projects/linux-malware-detect/
[New] adopted new versioning scheme 
      [MAJOR].[MINOR].[REV]
      1       3       1

v1.3 | May 11th 2010:
- First public release

v1.1 - v1.2 | Mar. 2010 - May 2010:
- Internal releases

v0.5 - v1.0 | Nov. 2009 - Feb. 2010:
- Closed beta

v0.4< | Oct. 2009:
- Internal releases