diff --git a/README.md b/README.md index a26c5e9a..8c24d304 100644 --- a/README.md +++ b/README.md @@ -117,7 +117,7 @@ by `myselector`, 3. add a DNS TXT record to `mydomain.example.com`, as described in `/var/lib/rspamd/dkim/myselector.txt` -## Configuration override for ClamAV unofficial signatures +## Configuration override for ClamAV unofficial signatures Changes to `clamav-unofficial-sigs` configuration are volatile. When the `clamav` container is stopped, local configuration changes are lost. @@ -330,10 +330,47 @@ Migration notes: `rspamd/RecipientWhiteList`. 1. New Full Text Search (FTS) engine Flatcurve. Old FTS indexes are - excluded from the migration. To massively rebuild the search indexes - run the following command during system idle time: + excluded from the migration. Refer to **Full Text Searche (FTS)** + section below. - podman exec dovecot nice doveadm index -A -q '*' +## Full Text Search (FTS) + +The default configuration enables full-text search over mail headers and +body. To keep the FTS index size small, only exact-word tokens are indexed +by default. + +To enable complete substring searches, the indexes must be rebuilt, which +significantly increases disk space usage. + +### Enable FTS substring search + +It is advisable to run this procedure outside of working hours because +mailbox reindexing has high system resource demands. + +Edit the module's `state/environment` file by adding the following line: + + DOVECOT_FLATCURVE_SUBSTRING_SEARCH=yes + +Reload the Dovecot container: + + systemctl --user reload dovecot + +Open a shell in the Dovecot container: + + podman exec -ti dovecot ash -l + +Calculate the size of the current FTS indexes (in MB): + + find . -type d -name fts-flatcurve -print0 | xargs -r -0 -- du -m | awk '{ tot += $1 } ; END { print tot }' + +Remove existing `fts-flatcurve` directories: + + find . -type d -name fts-flatcurve -exec rm -rf '{}' \; + +To rebuild the search indexes in bulk, run the following command during +system idle time: + + nice doveadm index -A -q '*' ## UI translation diff --git a/dovecot/README.md b/dovecot/README.md index 5ddcb97e..ed22a42e 100644 --- a/dovecot/README.md +++ b/dovecot/README.md @@ -61,8 +61,16 @@ Private TCP ports: documentation](https://doc.dovecot.org/settings/core/#core_setting-default_process_limit). The `default_client_limit` parameter is proportionally set to 10 times this value. - `DOVECOT_SHAREDSEEN`, default empty. If set, the SEEN flag of shared folders is stored in a `dovecot.pvt*` index file, for every user -- `DOVECOT_FLATCURVE_SUBSTRING_SEARCH`, default `no` (`yes` to enable). If enabled, allows substring searches (RFC 3501 compliant). However, this requires significant additional storage space. Many users today expect "Google-like" behavior, which is prefix searching, so substring searching is arguably not the modern expected behavior anyway. Therefore, even though it is not strictly RFC compliant, prefix (non-substring) searching is enabled by default. - +- `DOVECOT_FLATCURVE_SUBSTRING_SEARCH`, default `no` (`yes` to enable). + Many users today expect "Google-like" behavior, which is prefix + searching, so substring searching is arguably not the modern expected + behavior anyway. Therefore, even though it is not strictly RFC + compliant, prefix (non-substring) searching is enabled by default. + + If enabled, allows substring searches (RFC 3501 compliant). However, + this requires rebuilding the existing fts-flatcurve indexes with + significant additional storage space. + ## Logs The container sends log messages to `syslog`. Increase log verbosity by