Core

intelmq.lib.utils.drop_privileges: When IntelMQ is called as root and dropping the privileges to user intelmq, also set the non-primary groups associated with the intelmq user. Makes the behaviour of running intelmqctl as root closer to the behaviour of sudo -u intelmq ... (PR#2507 by Mikk Margus Möll).
intelmq.lib.utils.unzip: Ignore directories themselves when extracting data to prevent the extraction of empty data for a directory entries (PR#2512 by Kamil Mankowski).

Bots

intelmq.bots.collectors.shadowserver.collector_reports_api.py:
- Added support for the types parameter to be either a string or a list (PR#2495 by elsif2).
- Refactored to utilize the type field returned by the API to match the requested types instead of a sub-string match on the filename.
- Fixed timezone issue for collecting reports (PR#2506 by elsif2).
- Fixed behaviour if parameter reports value is empty string, behave the same way as not set, not like no report (PR#2523 by Sebastian Wagner).
intelmq.bots.collectors.shodan.collector_stream (PR#2492 by Mikk Margus Möll):
- Add alert parameter to Shodan stream collector to allow fetching streams by configured alert ID
intelmq.bots.collectors.mail._lib: Remove deprecated parameter attach_unzip from default parameters (PR#2511 by Sebastian Wagner).

intelmq.bots.parsers.shadowserver._config:
- Fetch schema before first run (PR#2482 by elsif2, fixes #2480).
intelmq.bots.parsers.dataplane.parser: Use | as field delimiter, fix parsing of AS names including | (PR#2488 by DigitalTrustCenter).
all parsers: add copy_collector_provided_fields parameter allowing copying additional fields from the report, e.g. extra.file_name.
(PR#2513 by Kamil Mankowski).

intelmq.bots.experts.sieve.expert:
- For :contains, =~ and !~, convert the value to string before matching avoiding an exception. If the value is a dict, convert the value to JSON (PR#2500 by Sebastian Wagner).
- Add support for variables in Sieve scripts (PR#2514 by Mikk Margus Möll, fixes #2486).
intelmq.bots.experts.filter.expert:
- Treat value false for parameter filter_regex as false (PR#2499 by Sebastian Wagner).

intelmq.bots.outputs.misp.output_feed: Handle failures if saved current event wasn't saved or is incorrect (PR by Kamil Mankowski).
intelmq.bots.outputs.smtp_batch.output: Documentation on multiple recipients added (PR#2501 by Edvard Rejthar).

Bots: Clarify some section of Mail collectors and the Generic CSV Parser (PR#2510 by Sebastian Wagner).

This is short list of the most important known issues. The full list can be retrieved from GitHub.

intelmq.parsers.html_table may not process invalid URLs in patched Python version due to changes in urllib (#2382).
Breaking changes in 'rt' 3.0 library (#2367).
Type error with SQL output bot's prepare_values returning list instead of tuple (#2255).
intelmq_psql_initdb does not work for SQLite (#2202).
intelmqsetup: should install a default state file (#2175).
Misp Expert - Crash if misp event already exist (#2170).
Spamhaus CERT parser uses wrong field (#2165).
Custom headers ignored in HTTPCollectorBot (#2150).
intelmqctl log: parsing syslog does not work (#2097).
Bash completion scripts depend on old JSON-based configuration files (#2094).
Bots started with IntelMQ-API/Manager stop when the webserver is restarted (#952).
Corrupt dump files when interrupted during writing (#870).