Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Update Grok pattern for UDM logs #25616

Merged
merged 2 commits into from
May 14, 2021

Conversation

legoguy1000
Copy link
Contributor

@legoguy1000 legoguy1000 commented May 7, 2021

What does this PR do?

Updates the IPtables ingest pipeline with an additional grok pattern.

Why is it important?

The module currently isn't able to parse some log entries created by the UDM pro

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 7, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 7, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25616 updated

  • Start Time: 2021-05-13T11:30:56.320+0000

  • Duration: 108 min 33 sec

  • Commit: bd5a220

Test stats 🧪

Test Results
Failed 0
Passed 7031
Skipped 1193
Total 8224

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 7031
Skipped 1193
Total 8224

@legoguy1000 legoguy1000 marked this pull request as ready for review May 9, 2021 12:20
@legoguy1000
Copy link
Contributor Author

@JAndritsch Let me know what you think for these changes. Also if you find out anything on the weird format of these logs, let me know.

@JAndritsch
Copy link

@JAndritsch Let me know what you think for these changes. Also if you find out anything on the weird format of these logs, let me know.

I'll try to test them out today. I just posted an issue on the Ubiquiti forums, so hopefully that'll help shed some light on things.

https://community.ui.com/questions/Strange-UDM-Pro-log-formatting/27969dc6-78eb-4d17-9417-38d252c62ce9

@JAndritsch
Copy link

JAndritsch commented May 10, 2021

I just had a chance to test these changes. It looks like the sample messages I gave you now parse properly. I was hoping to hear back from the UI forums before this gets merged, but it's quiet over there. I still feel like there's got to be something wrong with the log format from those UDM logs.

@P1llus
Copy link
Member

P1llus commented May 11, 2021

If it is working as you would expected we can go ahead and merge it, worst case scenario we can apply more changes later on if needed.

@P1llus
Copy link
Member

P1llus commented May 11, 2021

run tests

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 11, 2021
@P1llus P1llus self-assigned this May 11, 2021
Copy link
Member

@P1llus P1llus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, waiting for CI to pass

@andrewkroh andrewkroh added the needs_integration_sync Changes in this PR need synced to elastic/integrations. label May 11, 2021
@mergify
Copy link
Contributor

mergify bot commented May 12, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25615-udm-logs upstream/25615-udm-logs
git merge upstream/master
git push upstream 25615-udm-logs

@P1llus
Copy link
Member

P1llus commented May 12, 2021

run tests

@mergify
Copy link
Contributor

mergify bot commented May 13, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25615-udm-logs upstream/25615-udm-logs
git merge upstream/master
git push upstream 25615-udm-logs

@P1llus P1llus merged commit fcf19c3 into elastic:master May 14, 2021
@P1llus P1llus added the backport-v7.14.0 Automated backport with mergify label May 14, 2021
mergify bot pushed a commit that referenced this pull request May 14, 2021
* Update Grok pattern for UDM logs

* update changelog

(cherry picked from commit fcf19c3)
cachedout pushed a commit that referenced this pull request May 18, 2021
* Update Grok pattern for UDM logs

* update changelog

(cherry picked from commit fcf19c3)
P1llus added a commit that referenced this pull request Jun 28, 2021
…25721)

* [Filebeat] Update Grok pattern for UDM logs (#25616)

* Update Grok pattern for UDM logs

* update changelog

(cherry picked from commit fcf19c3)

* trying to regenerate golden files222222

* fixing tags

* add options from master

Co-authored-by: Alex Resnick <adr8292@gmail.com>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Co-authored-by: Marius Iversen <marius.iversen@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.14.0 Automated backport with mergify enhancement Filebeat Filebeat needs_integration_sync Changes in this PR need synced to elastic/integrations.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Iptables module unable to parse some logs from UDM-Pro device
5 participants