Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove JSON parsing as graylog now supports it fully #10

Closed
SjonHortensius opened this issue Jun 30, 2020 · 2 comments
Closed

remove JSON parsing as graylog now supports it fully #10

SjonHortensius opened this issue Jun 30, 2020 · 2 comments

Comments

@SjonHortensius
Copy link
Member

SjonHortensius commented Jun 30, 2020

remove json parsing from SystemdJournal2Gelf - let users configure this themselves as a pipeline. Steps to recreate this as a custom pipeline:

  • Go to /system/pipelines in your graylog install and create a new pipeline

  • Edit the new pipeline and add a stage:

  • In the first stage, add a rule to interpret the message as json:

rule "interpret message as JSON"
when
    has_field("message") && starts_with(to_string($message.message), "{\"")
then
    let json = parse_json(to_string($message.message));
    let map = to_map(json);
    set_fields(map);

    rename_field("Message", "message");
    rename_field("FullMessage", "full_message");
end
  • you can prefix additional stages to cleanup your message eg. when receiving messages from php-fpm:
rule "strip fpm pool prefix"
when
  has_field("message") && starts_with(to_string($message.message), "pool ")
then
  set_field("message", regex_replace("^pool [a-zA-Z_\\[\\d\\]]+: ", to_string($message.message), ""));
end
  • attach the pipeline to the appropriate stream(s). Make sure the Pipeline Processor is configured after Message Filter Chain in system/configurations
@hmmmsausages
Copy link

Hi @SjonHortensius just found this by accident.
I think the README.md requires updating, so people don't expect this JSON parsing to happen automatically.

@SjonHortensius
Copy link
Member Author

@hmmmsausages you're right - I've updated the README.MD

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants