Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add checks to rename processor for the message field for all Observability Integrations #7822

Closed
34 of 35 tasks
ishleenk17 opened this issue Sep 14, 2023 · 2 comments
Closed
34 of 35 tasks
Assignees

Comments

@ishleenk17
Copy link
Contributor

ishleenk17 commented Sep 14, 2023

This ticket is created as a followup of #3451
The rename processor (renaming message to event.original) in the ingest pipelines for log datastreams don't have an if/ignore_missing check

Current state:

  • rename:
    field: message
    target_field: event.original

Expected State:

  • rename:
    field: message
    target_field: event.original
    ignore_missing: true
    if: 'ctx.event?.original == null'
    description: 'Renames the original message field to event.original to store a copy of the original message. The event.original field is not touched if the document already has one; it may happen when Logstash sends the document.'

Below are the Observability Integrations which need this change

Phase1 updates:

Cloud Monitoring:

Cloud Native:

Phase2 updates:

@lalit-satapathy
Copy link
Collaborator

Adding @ritalwar for phase 2.

@ritalwar
Copy link
Contributor

ritalwar commented Dec 5, 2023

Implemented null checks for all the mentioned packages of Phase2 in this PR.

Due to the upcoming deprecation of Tomcat Netwitness logs, with a recommendation to switch to Apache Tomcat, we won't include these checks there.

image

As of now, there is no existing rename or set processor to manage event.original in the MySQL package. This will be taken care in a separate issue. Therefore, I'm closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants