Add RFC 5424 extractors, update README #11
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
OPNsense is capable of sending RFC 5424-compliant syslog messages when configured to do so. This ensures timestamps are sent with timezone info attached, enabling correct timestamps in Graylog. However, since Graylog will then strip out the "filterlog" header into the "application_name" field, we must inspect "full_message" instead of "message". The regex also changes, since the compliant syslogs look different than normal syslogs (as they now include meta information). The IPv6 ICMP extractor also experienced a slight formatting change in an OPNsense update.
Added expanded usage and version information
|
The RFC5424 extractors added by @subract are working for me (thanks!). |
|
Working for me as well! |
|
Oh nice @subract, thanks for your work. Why is it not merged already... |
IRQ10
approved these changes
Sep 26, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When I tried using these extractors in my environment, the timestamps were incorrect (OPNsense was sending timestamps in its local time zone). When I configured OPNsense to send RFC 5424-compliant logs instead, the timestamp issue was fixed, but the extractors could no longer parse the logs, particularly because
filterlogwas now being parsed out into theapplication_namefield (and therefore was no longer present in themessagefield). This necessitates instructing Graylog to save the full message, as detailed in the updated README.I updated the regexes to match the different RFC-compliant format, which looks something like this (example is an IPv6 ICMP message). Note the inclusion of the
metastructured data ID - this meant changing the regex to match anything in brackets, not just digits:Finally, it appears that OPNsense has switched from sending
ipv6-icmpto justicmp, so I fixed that up as well.Let me know if there's anything I should address, and thanks for developing these extractors!