Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Anomali ThreatStream support to threatintel module #26350

Merged
merged 9 commits into from
Jun 24, 2021
Merged

Add Anomali ThreatStream support to threatintel module #26350

merged 9 commits into from
Jun 24, 2021

Conversation

adriansr
Copy link
Contributor

@adriansr adriansr commented Jun 16, 2021 •
edited
Loading

What does this PR do?

This adds a new dataset, anomalithreatstream to the threatintel module.

Why is it important?

Allows to ingest indicators from Anomali ThreatStream Integrator via a custom SDK output.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Screenshots

Docs:

image

@adriansr
Add Anomali ThreatStream support to threatintel module
f07ec94 
This adds a new dataset, `anomali_threatstream` to the threatintel
module. It allows to ingest indicators from Anomali ThreatStream
Integrator via a custom SDK output.
@adriansr
Update docs
3c47f30
@adriansr
Fix manifest defaults
0b77fda
@adriansr
Changelog entry
d3c9557
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 16, 2021
@mergify
Copy link
Contributor

mergify bot commented Jun 16, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b anomali_threatstream upstream/anomali_threatstream
git merge upstream/master
git push upstream anomali_threatstream

@adriansr adriansr requested a review from P1llus June 16, 2021 17:19
@adriansr
Overwrite updated documents
3dc700b 
This sets the op_type metadata flag to index so that documents with
duplicate `id` field are overwritten instead of discarded.
@adriansr
Merge branch 'master' into anomali_threatstream
1fce7f2
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 17, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #26350 updated

  • Start Time: 2021-06-23T15:32:59.192+0000

  • Duration: 115 min 13 sec

  • Commit: 1cc2713

Test stats 🧪

Test Results
Failed 0
Passed 14112
Skipped 2311
Total 16423

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 14112
Skipped 2311
Total 16423

P1llus
P1llus approved these changes Jun 17, 2021
View reviewed changes
Copy link
Member

@P1llus P1llus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great and the TI Mapping seems to be on point as well! Asked the CTI team to also confirm, but if there is no feedback from them then this is all good to go 👍

Awesome work!

adriansr
adriansr commented Jun 18, 2021
View reviewed changes
Copy link
Contributor Author

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to fix

x-pack/filebeat/module/threatintel/anomali_threatstream/config/config.yml Outdated Show resolved Hide resolved
andrewkroh
andrewkroh approved these changes Jun 21, 2021
View reviewed changes
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice docs.

Should this have its own dashboard?

@adriansr
Copy link
Contributor Author

Renamed from anomali_threatstream to anomalithreatstream to prevent some issues with the new Threat Intel Dashboard.

@adriansr adriansr added the backport-v7.14.0 Automated backport with mergify label Jun 24, 2021
@adriansr adriansr merged commit a6d8cdb into elastic:master Jun 24, 2021
mergify bot pushed a commit that referenced this pull request Jun 24, 2021
@adriansr
Add Anomali ThreatStream support to threatintel module (#26350)
ea252df 
* Add Anomali ThreatStream support to threatintel module

This adds a new dataset, `anomalithreatstream` to the threatintel
module. It allows to ingest indicators from Anomali ThreatStream
Integrator via a custom SDK output.

(cherry picked from commit a6d8cdb)
@mergify mergify bot mentioned this pull request Jun 24, 2021
Merged
adriansr added a commit that referenced this pull request Jun 24, 2021
@mergify @adriansr
Add Anomali ThreatStream support to threatintel module (#26350) (#26472)
0334abb 
This adds a new dataset, `anomalithreatstream` to the threatintel
module. It allows to ingest indicators from Anomali ThreatStream
Integrator via a custom SDK output.

(cherry picked from commit a6d8cdb)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Jun 28, 2021
@mdelapenya
Merge branch 'master' into archive-system-tests
b6dbec3 
* master: (32 commits)
  [Metricbeat] Change Account ID to Project ID in `gcp.billing` module (elastic#26412)
  update libbeat fields.ecs.yml file and ecsVersion to 1.10.0 (elastic#26121)
  [Filebeat] Update AWS ELB ingest pipeline (elastic#26441)
  [FIlebeat] add strict_date_optional_time_nanos date format to PanOS module (elastic#26158)
  Fix the irregular and typo on prometheus module. (elastic#25726)
  [Filebeat] Parse additonal debug data fields for Okta module (elastic#25818)
  fix: update MSSQL Server linux image's Docker registry (elastic#26440)
  Update indexing.go godocs (elastic#26408)
  Do not close filestream harvester if an unexpected error is returned when close.on_state_change.* is enabled (elastic#26411)
  Add support for copytruncate method when rotating input logs with an external tool in `filestream` input (elastic#23457)
  Allow fields with ip_range datatype (elastic#26444)
  Add Anomali ThreatStream support to threatintel module (elastic#26350)
  fix: use the right param type (elastic#26469)
  [Automation] Update elastic stack version to 8.0.0-7640093f for testing (elastic#26460)
  Set SM Filebeat modules as GA (elastic#26226)
  Fix rfc5464 date parsing in the syslog input (elastic#26419)
  Add linked account information into billing metricset (elastic#26285)
  [Filebeat] Update HA Proxy log grok patterns (elastic#25835)
  disable metricbeat logstash test_node_stats (elastic#26436)
  chore: pass BEAT_VERSION when running E2E tests (elastic#26291)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@P1llus P1llus P1llus approved these changes

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees
No one assigned
Labels
backport-v7.14.0 Automated backport with mergify enhancement review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@adriansr @elasticmachine @andrewkroh @P1llus