Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement: add hostname to Microsoft 365 defender ingest pipeline #29863

Merged

Conversation

hinchliff
Copy link
Contributor

@hinchliff hinchliff commented Jan 14, 2022

  • Enhancement

What does this PR do?

The events from Defender ATP include the fields host.name and host.hostname, but these fields were missing from the 365 Defender events.

This PR adds parsing/extraction of host.name and host.hostname field from the 365 Defender events, using the json.alerts.devices.deviceDnsName from the source.

Why is it important?

In order to use Elastic Security, host.name is a required field

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 14, 2022
@mergify
Copy link
Contributor

mergify bot commented Jan 14, 2022

This pull request does not have a backport label. Could you fix it @hinchliff? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Jan 14, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 14, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview [preview](https://ci-stats.elastic.co/app/apm/services/beats-ci/transactions/view?rangeFrom=2022-03-08T00:41:32.346Z&rangeTo=2022-03-08T01:01:32.346Z&transactionName=BUILD Beats/beats/PR-{number}&transactionType=job&latencyAggregationType=avg&traceId=2ed22f4fef372c83ead1b27cc6df51e9&transactionId=88dc971b7beb86cd)

Expand to view the summary

Build stats

  • Start Time: 2022-03-08T00:51:32.346+0000

  • Duration: 97 min 53 sec

Test stats 🧪

Test Results
Failed 0
Passed 2419
Skipped 159
Total 2578

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 17, 2022
@efd6
Copy link
Contributor

efd6 commented Jan 18, 2022

/test

@efd6
Copy link
Contributor

efd6 commented Jan 19, 2022

I have some better advice; the hosts can be appended only into the related.hosts rather than into host.hostname and host.name as well. Then the httpjson split change can be dropped. Apologies for leading you astray.

My understanding is that there is no mapping for host.name/host.hostname for m365 defender. The MS reference for the fields is available here.

@mergify
Copy link
Contributor

mergify bot commented Feb 1, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 29859-enhancement-hostname-ms-365-defender upstream/29859-enhancement-hostname-ms-365-defender
git merge upstream/master
git push upstream 29859-enhancement-hostname-ms-365-defender

@mergify
Copy link
Contributor

mergify bot commented Feb 23, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 29859-enhancement-hostname-ms-365-defender upstream/29859-enhancement-hostname-ms-365-defender
git merge upstream/main
git push upstream 29859-enhancement-hostname-ms-365-defender

@mergify
Copy link
Contributor

mergify bot commented Mar 7, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 29859-enhancement-hostname-ms-365-defender upstream/29859-enhancement-hostname-ms-365-defender
git merge upstream/main
git push upstream 29859-enhancement-hostname-ms-365-defender

@efd6
Copy link
Contributor

efd6 commented Mar 8, 2022

/test

Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your patience.

@efd6 efd6 merged commit 45c09c6 into elastic:main Mar 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.2-candidate backport-skip Skip notification from the automated backport with mergify enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The Ingest Pipeline for Microsoft 365 Defender should populate hostname
4 participants