From 2b13aa004b4d0ed0506eed7b6d2046e264914fde Mon Sep 17 00:00:00 2001 From: noaco <31587493+noaco@users.noreply.github.com> Date: Tue, 21 Aug 2018 18:36:36 +0300 Subject: [PATCH] FireEye ETP integration (#1735) * add etp integration * add outputs description * print response text and fix milliseconds when fetching * Fireeye etp integration fixes (#1853) * Clear release notes (#1733) * clear rn * git hash * fix cylance test (#1740) - make the test run in sequance instead of parallel - command was failing because didnt found some threat * Farsight DNSDB - Added handling for 404 and 400 responses (#1675) * Added handling for 404 and 400 responses graceful handling of no inforamaion found condition Also improved humanreadable for results * fix commonfields fix commonfields * Added test plybook for DNSDB Farsight DNSDB test playbook * Remove special support for EWS (#1736) * Remove special support for EWS Remove special playbook for custom fields (Use "setIncident" instead) * Add release notes * CR fixes * move qualys test to nightly - it requires only single run of build in parallel (#1697) * add connections to canvas suggestions (#1729) * add connections to canvas suggestions * fix schema validation * Add vt connections (#1742) add VT connections * Ews untitled/empty attachments (#1728) * Fixed handling untitled/empty attachments * Added null checks for on attachment content. * Implemented workaround for exchangelib not handling empty file attachments (zero bytes) * Added empty attachment test playbook * Support for RedLock alerts (#1721) (#1738) * Support for RedLock alerts (#1721) * Support for RedLock alerts * Fixes issue with EWS Search and Delete (#1696) * Fixes issue with EWS Search and Delete * CR fixes * Fix typo and releaseNotes * Add Test playbook * Remove forward/ replay prefixes only from beginning of Subject * Update "Detonate File - Generic" (#1722) * Update "Detonate File - Generic" * Improve documentation * Add supported file types * Add support for updated Falcon integration * Add outputs description * Add output description * Remove auto-log from QRadarFullSearch (#1715) * Remove auto-log from QRadarFullSearch Was automatically printing logs to the war-room * Fix CR * CrowdStrike Falcon Sandbox enhancements and fixes (#1635) * Netskope integration test fix * CrowdStrike Falcon Sandbox enhancements and fixes * Made requested changes * add context canvas connections (#1718) * add context canvas connections * add schema validation * updated argument types (#1725) * Update playbook-RedLockTest.yml Fixed and rephrased some task names. * Added fetch-incidents * Splunkpy search (#1717) * add enhancement script for splunk search * add to indicator types SplunkSearchPy * use cmd only in depnds on * Reverted addition of threat-grid-detonate-file and threat-grid-url-to-file commands (#1726) * Update Intezer integration (#1727) * Update Intezer integration - Malicious should be added only for bad reputation hashes. * Update outputs * change the url for 'GET request test' task (#1731) * add command line output to cb defense (#1730) * remove minemeld for now (#1732) * remove minemeld for now * skip minemeld test * Fixed comments from code review * Update playbook-RedLockTest.yml Removed old command arguments from playbook tasks * Added RedLock test * Unskip Cybereason test (#1746) * remove releasenotes for SEPM14 (#1622) * remove releasenotes * Update integration-SymantecEndpointProtection.yml * ipinfo.io - Added support to use API token for paid plans (#1673) * Added support to use API token for paid plans * Add token to ipinfo_field command * token * only send token parameter if token is set * ' * Remove "command-timeout" command argument for 3.6.1 (#1749) * Removed argument "command-timeout" as this is replaced wtih the global argument with the same name and (effective) functionality. * Removed depracated argument * Renamed problematic argument instead of removing it. * Renamed problematic argument instead of removing it (in test playbook). * Rename integration-Carbon_Black_Enterprise_Live_Response_old.yml to integration-Carbon_Black_Enterprise_Live_Response_3.6.0.yml * Handle tanium/vmware timeout on errors issue (#1751) * handle errors from tanium integration * handle errors from vmware integration * unskip tests * vmware - verify logout is done even if there is an error * vmware - verify logout is done even if there is an error * added release notes * skip vmware test * Script helper python arg order fix (#1754) * fix arg order of args in pythoncommonserver doc * set automationOnly tag for commonServer commands using executeCommand * fix null argument descriptions in script helper * releasenotes * Fix circleci2 (#1759) * try fix curl bad characters * try fix curl bad characters * fix space * skip anomali test (#1763) * Hybrid Analysis Integration (#1745) * Hybrid Analysis Integration * fixed fromversion * Made requested changes * Single-setup adjustments (#1752) * Wildfire getReport bug fix (#1753) * getReport bug fix getReport bug fix * Added empty RN * Improved implementation * Cylance Protect v2 device data context path fix (#1661) * Cylance Protect v2 device data context path fix * Made requested changes * Made requested changes * Fixed test according to context changes * use the added command (#1761) * use the added command * deprecate * Postgres fix error (#1765) * fix error on no rows returned * test playbook * fix exception * Fixed move-between-mailboxes using impersonation (#1766) * Archer add fields checks and full results for get-records-by-report (#1744) * Archer add fields checks and full results for get-records-by-report * CR fixes * Access Investigation - Generic (#1760) * Access Investigation - Generic New playbooks: * Access Investigation - Generic * Access Investigation - QRadar Updated playbooks: * IP Enrichment - Generic New script: * IPToHost Updated script: * EmailAskUser * Add description * add description * Update task scheme * CR fixes * add systemAssociatedTypes (#1758) * Vulnerability Management - Nexpose: (#1762) * Vulnerability Management - Nexpose: New playbooks: * Vulnerability Handling - Nexpose * Vulnerability Management - Nexpose (Job) Updated playbooks: * Calculate Severity - Generic * Calculate Severity - 3rd-party integrations New scriptL * NexposeCreateIncidentsFromAssets * CR fixes * Add description * bug fix * Email sender in Python with embedded images (#1671) * Email sender in Python with embedded images * Added template variables in ugly way * Changed default value for sender address at email sender integration * removed empty lines at EOF * added Mail Sender (New) integration & playbook * added Mail Sender (New) integration & playbook * removed old integration file (difference only in name of integration) * changed email sender python (new) ID * fixed playbook trying to activate script by old name and failing * changed deletecontext script back to original * added google apps integration for mail sender (new) * Added newline support for base64 images in html * Fix missing release notes (#1767) * print commands outputs * check if files exist * refactor * print files * add prints * check if file is empty * grep error * update git hash * add missing rn * revert config.yml * remove prints * add missing space * removed palo alto from conf.json (#1771) * removed palo alto from conf.json * add running-playbooks widget (#1755) * add running-playbooks widget * Update widget-RunningPlaybooks.json * Crowdstrike falcon intel v2 support (#1768) * added crowdstrike intel test playbook + v2 indicator integration * fixed format (whitespace missing) * extended playbook cs-indicators * added test-module by version, more documentation * added releaseNotes to crowdstrike falcon intel * remove approve action from tanium playbook (#1769) * TruSTAR integration enhancements (#1772) * Enhanced Trustar integration (#1706) * Enhanced trustar integration * Enhanced trustar integration * Enhanced trustar integration * Revert "Enhanced trustar integration" This reverts commit c7aa5c9149c5b17bbe46904ef90cd030d6df25ec. * Enhanced trustar integration * Incorporated review comments for trustar integration * Incorporated review comment - added priority level in entry context * Added priority level to software indicator & in output parameter * Priority level key error handled for treding and search indicators command * Added RN * new widget should be predfined (#1773) * Recorded Future integration (#1764) * Recorded Future integration * Made requested changes * Skip Intezer test (#1777) * Add delay to intezer test playbook * Skip Intezer test * avoid error in domain format script (#1774) * AWS ec2 (#1770) * AWS EC2 Integration * add get-latest-ami outputs * added aws connection function * add test playbooks * fix test playbook location * Fix describe instances context issue * fix #12097 & describe instances tags output * fix #12097 for all aws integrations * Added new commands * added release notes * Nexpose enhancements (#1714) * paste * python * add commands * fix char * reports * scans, fixes * outputs, login, scan wait * test playbook * image * fixes #1 * rn, fixed playbook test * add report formats * fix test playbook * fix test playbook * fix test playbook * merge * add cve output, add raw outputs, search by multiple hosts & ips * Removed start-scan commands * Clear release notes (#1780) * Clear release notes * Update git hash * Add fromversion field to relevant playbooks * Added empty RN * Enable Intezer test (#1779) * spelling fixes (#1781) * fix wether to whether * relase notes * Moved qualys test to skipped due to expired account issues (#1783) currently fails content build nightly * Demisto REST API - new commands to upload and download files (#1748) * added multipart and download commands * added multipart and download commands * added multipart and download commands * added multipart and download commands * added multipart and download commands * added scripts to download logs bundle, and upload files to war room * Convert Incident fields to array (#1784) * convert to list * skip validate * add import json * use seek and truncate * revert config yml * Fix domain rep (#1785) * domain fix reputation * fix RN * fix RN * Crowdstrike falcon intel (#1790) * crowdstrike falcon intel change report id to retrieve due to size * updating default value of API version to 2.0 (#1782) * updating default value of API version to 2.0 1.6 is no longer available, 2.0 is the default version in the hosted environment * Clear release notes (#1789) * Clear release notes * Update git hash * Add fromversion field to relevant playbooks * Added empty RN * clear release notes after 18.7.1 release * update git hash * Replace demisto lock logo (#1792) * Updated integration name source (#1775) * Ews readable errors (#1788) * Changed default authentication method to "Basic" in accordance with instance defaults for office365. * Beautified error messages in test_module * Handled case where no error message is set * secureworks add default url -https://api.secureworks.com (#1798) * secureworks add default url -https://api.secureworks.com - fixes https://github.com/demisto/etc/issues/12378 * Update integration-SecureWorks.yml * fix ArcSight ESM addEntries (#1797) * fix ArcSight ESM addEntries - if entries had passed from context as JSON then we got exception * Update integration-ArcSightESM.yml * Added eventType fetch filter (#1796) * remove `runonce: true` from phish.ai (#1799) * remove `runonce: true` from phish.ai * add rn * scripts - deprecate checkwhitelist + add filterbywhite lists (#1708) * scripts - deprecate checkwhitelist + add filterbywhite lists * Change wording * add support for array input + change whitelist ot list * malicious ration reputation script (#1778) * malicious ration reputation script * change script logic to return score as reputation script & DBot score * disable TE test playbook (#1802) * disable TE test playbook * ignore right test * Top malicious ratio indicators (#1750) * Top malicious ratio indicators * Fix script schema validation * remove script schema validatiom * fix CR * add widget to display script results * add fromversion filter 0 malicious ratio * add widget from version * fix file format * Fix desc build (#1808) * adding RN * add desc * Replace integrations logos (#1807) * Replace integrations logos * Add release notes * Add release notes * Fixed logos * fix widget should be isPredefined (#1818) * fix widget should be isPredefined * Update widget-TopMaliciousRationIndicators.json * Validate widget isPredefined property is true (#1819) Output in case some widget has `isPredefined: false` ```bash Starting validate Widgets... Failed: Widgets/widget-IncidentInErrorNumber.json failed Finished validate Widgets validate_files_structure.sh exiting with error ``` * Added traceback import (#1806) * Added traceback import * Moved redlock test to nightly (#1804) * check proxy parameter before client.connect() (#1824) * check proxy parameter before client.connect() Attempting to connect to the splunk server before checking for the proxy parameter causes a connection timeout if the splunk server is not accessible without using a proxy. * add release notes * Change the term investigation to incident in the layouts. (#1825) * FireEye URL submissions (#1743) (#1820) * FireEye URL submissions (#1743) * FireEye URL submissions Added functionality to submit URLs to FireEye and retrieve their status. Functions created are fe-submit-url and fe-submit-url-status * Modify integration description Modified integration description to aligh with naming standards and help user understand how certain parameters should be passed. * Reverted fe-submit and fe-submit-status back to original name Reverted fe-submit and fe-submit-status back to original name from fe-submit-file and fe-submit-status * deleting file that is not part of integration * Update integration-fireeye.yml * added predefined parameters for commands added predefined parameters for commands * Added release notes * Fixed Twilio test function (#1826) * Fixed Teilio test function Fixes https://github.com/demisto/etc/issues/12214 * CR fixes * Feature/widgets for engine and workers (#1689) * Widgets for engine and workers * Fixed unnamed attachments bug (#1822) * Fixed unnamed attachments bug. * Handled possible case where attachment name is not a string. * Corrected comparison method according to PEP-8 recommendation. * enable TE again (#1828) * add note to schema (#1830) * Passive Total - added proxy and insecure parameters (#1814) * added insecure and proxy settings * proxy defaults to true and insecure defaults to false * Fixed URL command, added IP and Domain * Added to release notes * add widget description (#1823) * Recorded Future bug fix (#1832) * Vulnerability Management issue fix (#1815) * RTIR integration (#1833) * RTIR Integration * add return_error function * added docstring * Updated the regex (#1801) (#1834) * Updated the regex (#1801) Updated the regex to properly pull the detection ID. Sometimes the detection ID changes in length, but it's always a number from 0-9 * add release notes * RTIR Spanish support (#1835) * RTIR integration spanish support * RTIR integration spanish support * prettify common server doc python error (#1836) * limit fetch incidents from netwitness (#1800) - fixes https://github.com/demisto/etc/issues/12195 * Update process email + phishing layout (#1813) * Update process email + phishing layout * And `HTML Rendered Image` MD field to phishing layout * Add Base64 output to the `rasterize-email` command * Update rasterized image to the Phishing summary page TODO: * Remove the HTML field mapping in the relevant integrations * Update incidentfields.json Change field name * Update layout-details-Phishing.json Change field name * Update playbook-Process_Email_-_Generic.yml change field name * Update playbook-Process_Email_-_Generic.yml big scheme issue * Update playbook-Process_Email_-_Generic.yml really fixing it * Update incidentfields.json typo fix * Add scheme * add missing tag * Add release notes * move note up (#1838) * Repopulate files (#1839) * Repopulate files adds the File context based on file entries * Removed some lines * Skipped redlock test (#1840) * Skipped redlock test * Removed duplicate test * Zoom support within Demisto (#1757) * Zoom support within Demisto * fix typo * - added test playbook to test zoom commands - added automation script to generate a random email * fixed 2 bugs in the zoom-fetch-recording: 1. Fetch recording didn't work because the wrong arg key was used (id instead of meeting_id) 2. Recording delete didn't work because params and headers weren't passed to the delete request * Updated zoom test playbook * changes requested in code review * changes requested in code review * Removed obsolete file that made tests fail * Added description to zoom integration * Handle tanium/vmware timeout on errors issue (#1751) * handle errors from tanium integration * handle errors from vmware integration * unskip tests * vmware - verify logout is done even if there is an error * vmware - verify logout is done even if there is an error * added release notes * skip vmware test * Script helper python arg order fix (#1754) * fix arg order of args in pythoncommonserver doc * set automationOnly tag for commonServer commands using executeCommand * fix null argument descriptions in script helper * releasenotes * Fix circleci2 (#1759) * try fix curl bad characters * try fix curl bad characters * fix space * skip anomali test (#1763) * Hybrid Analysis Integration (#1745) * Hybrid Analysis Integration * fixed fromversion * Made requested changes * Single-setup adjustments (#1752) * Wildfire getReport bug fix (#1753) * getReport bug fix getReport bug fix * Added empty RN * Improved implementation * Cylance Protect v2 device data context path fix (#1661) * Cylance Protect v2 device data context path fix * Made requested changes * Made requested changes * Fixed test according to context changes * use the added command (#1761) * use the added command * deprecate * Postgres fix error (#1765) * fix error on no rows returned * test playbook * fix exception * Fixed move-between-mailboxes using impersonation (#1766) * Archer add fields checks and full results for get-records-by-report (#1744) * Archer add fields checks and full results for get-records-by-report * CR fixes * Access Investigation - Generic (#1760) * Access Investigation - Generic New playbooks: * Access Investigation - Generic * Access Investigation - QRadar Updated playbooks: * IP Enrichment - Generic New script: * IPToHost Updated script: * EmailAskUser * Add description * add description * Update task scheme * CR fixes * add systemAssociatedTypes (#1758) * Vulnerability Management - Nexpose: (#1762) * Vulnerability Management - Nexpose: New playbooks: * Vulnerability Handling - Nexpose * Vulnerability Management - Nexpose (Job) Updated playbooks: * Calculate Severity - Generic * Calculate Severity - 3rd-party integrations New scriptL * NexposeCreateIncidentsFromAssets * CR fixes * Add description * bug fix * Email sender in Python with embedded images (#1671) * Email sender in Python with embedded images * Added template variables in ugly way * Changed default value for sender address at email sender integration * removed empty lines at EOF * added Mail Sender (New) integration & playbook * added Mail Sender (New) integration & playbook * removed old integration file (difference only in name of integration) * changed email sender python (new) ID * fixed playbook trying to activate script by old name and failing * changed deletecontext script back to original * added google apps integration for mail sender (new) * Added newline support for base64 images in html * Fix missing release notes (#1767) * print commands outputs * check if files exist * refactor * print files * add prints * check if file is empty * grep error * update git hash * add missing rn * revert config.yml * remove prints * add missing space * removed palo alto from conf.json (#1771) * removed palo alto from conf.json * add running-playbooks widget (#1755) * add running-playbooks widget * Update widget-RunningPlaybooks.json * Crowdstrike falcon intel v2 support (#1768) * added crowdstrike intel test playbook + v2 indicator integration * fixed format (whitespace missing) * extended playbook cs-indicators * added test-module by version, more documentation * added releaseNotes to crowdstrike falcon intel * remove approve action from tanium playbook (#1769) * TruSTAR integration enhancements (#1772) * Enhanced Trustar integration (#1706) * Enhanced trustar integration * Enhanced trustar integration * Enhanced trustar integration * Revert "Enhanced trustar integration" This reverts commit c7aa5c9149c5b17bbe46904ef90cd030d6df25ec. * Enhanced trustar integration * Incorporated review comments for trustar integration * Incorporated review comment - added priority level in entry context * Added priority level to software indicator & in output parameter * Priority level key error handled for treding and search indicators command * Added RN * new widget should be predfined (#1773) * Recorded Future integration (#1764) * Recorded Future integration * Made requested changes * Skip Intezer test (#1777) * Add delay to intezer test playbook * Skip Intezer test * avoid error in domain format script (#1774) * AWS ec2 (#1770) * AWS EC2 Integration * add get-latest-ami outputs * added aws connection function * add test playbooks * fix test playbook location * Fix describe instances context issue * fix #12097 & describe instances tags output * fix #12097 for all aws integrations * Added new commands * added release notes * Nexpose enhancements (#1714) * paste * python * add commands * fix char * reports * scans, fixes * outputs, login, scan wait * test playbook * image * fixes #1 * rn, fixed playbook test * add report formats * fix test playbook * fix test playbook * fix test playbook * merge * add cve output, add raw outputs, search by multiple hosts & ips * Removed start-scan commands * Clear release notes (#1780) * Clear release notes * Update git hash * Add fromversion field to relevant playbooks * Added empty RN * Enable Intezer test (#1779) * spelling fixes (#1781) * fix wether to whether * relase notes * Moved qualys test to skipped due to expired account issues (#1783) currently fails content build nightly * Demisto REST API - new commands to upload and download files (#1748) * added multipart and download commands * added multipart and download commands * added multipart and download commands * added multipart and download commands * added multipart and download commands * added scripts to download logs bundle, and upload files to war room * Convert Incident fields to array (#1784) * convert to list * skip validate * add import json * use seek and truncate * revert config yml * Fix domain rep (#1785) * domain fix reputation * fix RN * fix RN * Crowdstrike falcon intel (#1790) * crowdstrike falcon intel change report id to retrieve due to size * updating default value of API version to 2.0 (#1782) * updating default value of API version to 2.0 1.6 is no longer available, 2.0 is the default version in the hosted environment * Clear release notes (#1789) * Clear release notes * Update git hash * Add fromversion field to relevant playbooks * Added empty RN * clear release notes after 18.7.1 release * update git hash * Replace demisto lock logo (#1792) * Updated integration name source (#1775) * Ews readable errors (#1788) * Changed default authentication method to "Basic" in accordance with instance defaults for office365. * Beautified error messages in test_module * Handled case where no error message is set * secureworks add default url -https://api.secureworks.com (#1798) * secureworks add default url -https://api.secureworks.com - fixes https://github.com/demisto/etc/issues/12378 * Update integration-SecureWorks.yml * fix ArcSight ESM addEntries (#1797) * fix ArcSight ESM addEntries - if entries had passed from context as JSON then we got exception * Update integration-ArcSightESM.yml * Added eventType fetch filter (#1796) * remove `runonce: true` from phish.ai (#1799) * remove `runonce: true` from phish.ai * add rn * scripts - deprecate checkwhitelist + add filterbywhite lists (#1708) * scripts - deprecate checkwhitelist + add filterbywhite lists * Change wording * add support for array input + change whitelist ot list * malicious ration reputation script (#1778) * malicious ration reputation script * change script logic to return score as reputation script & DBot score * disable TE test playbook (#1802) * disable TE test playbook * ignore right test * Top malicious ratio indicators (#1750) * Top malicious ratio indicators * Fix script schema validation * remove script schema validatiom * fix CR * add widget to display script results * add fromversion filter 0 malicious ratio * add widget from version * fix file format * Fix desc build (#1808) * adding RN * add desc * Replace integrations logos (#1807) * Replace integrations logos * Add release notes * Add release notes * Fixed logos * fix widget should be isPredefined (#1818) * fix widget should be isPredefined * Update widget-TopMaliciousRationIndicators.json * Validate widget isPredefined property is true (#1819) Output in case some widget has `isPredefined: false` ```bash Starting validate Widgets... Failed: Widgets/widget-IncidentInErrorNumber.json failed Finished validate Widgets validate_files_structure.sh exiting with error ``` * Added traceback import (#1806) * Added traceback import * Moved redlock test to nightly (#1804) * check proxy parameter before client.connect() (#1824) * check proxy parameter before client.connect() Attempting to connect to the splunk server before checking for the proxy parameter causes a connection timeout if the splunk server is not accessible without using a proxy. * add release notes * Change the term investigation to incident in the layouts. (#1825) * FireEye URL submissions (#1743) (#1820) * FireEye URL submissions (#1743) * FireEye URL submissions Added functionality to submit URLs to FireEye and retrieve their status. Functions created are fe-submit-url and fe-submit-url-status * Modify integration description Modified integration description to aligh with naming standards and help user understand how certain parameters should be passed. * Reverted fe-submit and fe-submit-status back to original name Reverted fe-submit and fe-submit-status back to original name from fe-submit-file and fe-submit-status * deleting file that is not part of integration * Update integration-fireeye.yml * added predefined parameters for commands added predefined parameters for commands * Added release notes * Fixed Twilio test function (#1826) * Fixed Teilio test function Fixes https://github.com/demisto/etc/issues/12214 * CR fixes * Feature/widgets for engine and workers (#1689) * Widgets for engine and workers * Fixed unnamed attachments bug (#1822) * Fixed unnamed attachments bug. * Handled possible case where attachment name is not a string. * Corrected comparison method according to PEP-8 recommendation. * enable TE again (#1828) * add note to schema (#1830) * Passive Total - added proxy and insecure parameters (#1814) * added insecure and proxy settings * proxy defaults to true and insecure defaults to false * Fixed URL command, added IP and Domain * Added to release notes * add widget description (#1823) * Recorded Future bug fix (#1832) * Vulnerability Management issue fix (#1815) * RTIR integration (#1833) * RTIR Integration * add return_error function * added docstring * Updated the regex (#1801) (#1834) * Updated the regex (#1801) Updated the regex to properly pull the detection ID. Sometimes the detection ID changes in length, but it's always a number from 0-9 * add release notes * RTIR Spanish support (#1835) * RTIR integration spanish support * RTIR integration spanish support * prettify common server doc python error (#1836) * limit fetch incidents from netwitness (#1800) - fixes https://github.com/demisto/etc/issues/12195 * Update process email + phishing layout (#1813) * Update process email + phishing layout * And `HTML Rendered Image` MD field to phishing layout * Add Base64 output to the `rasterize-email` command * Update rasterized image to the Phishing summary page TODO: * Remove the HTML field mapping in the relevant integrations * Update incidentfields.json Change field name * Update layout-details-Phishing.json Change field name * Update playbook-Process_Email_-_Generic.yml change field name * Update playbook-Process_Email_-_Generic.yml big scheme issue * Update playbook-Process_Email_-_Generic.yml really fixing it * Update incidentfields.json typo fix * Add scheme * add missing tag * Add release notes * move note up (#1838) * Repopulate files (#1839) * Repopulate files adds the File context based on file entries * Removed some lines * Skipped redlock test (#1840) * Skipped redlock test * Removed duplicate test * rebased master * Bug fix - Detonate playbooks (#1846) * Alien Vault OTX DBot Score removal (#1844) * Alien Vault OTX DBot Score removal * Removed AlienVault instance from tests and added VirusTotal * Parse email files enhancements (#1843) * Added support for "SMTP mail text, ASCII text" files. Fixed bug in email address extraction. * Added test case for multiline address * Fixed release note format * Fixed release note format * Created playbook-TestQradar (#1842) * Created playbook-TestQradar * 1. Updated Test playbooks id and version 2. Added QRadar to conf.json * Clear release notes (#1847) * Clear release notes * Update git hash * Add fromversion field to relevant playbooks * Added empty RN * clear release notes after 18.7.1 release * update git hash * 18.7.2 clear rn * 18.7.2 changed git hash * DeleteContext - added the ability to provide keys to keep (#1787) * added the ability to provide keys to keep * improving argument description * improving argument description * Demisto lock description fix and increase default timeout (#1849) * fixed description of param and argument * default timeout changed to 600 second (10 min) * default timeout changed to 600 second (10 min) * default timeout changed to 600 second (10 min) * enhance ExportToCSV script (#1669) * - add option to add csv headers as script argument - add parsing in case of string input * remove runonce * add newline at the end of file * add releaseNotes * add newline at the end of the file * handle array of strings * handle boolean and number values * modify to except more input types as valid inputs for csvArray * add release notes * add test playbook * fix scriptName reference * add test playbook * rn * versions * ES6 to ES5 * added usage of return_error_and_exit in http_request * Update integration-FireEye_ETP.yml * fixed time formatting for last_run * change fetch incidents to poll alerts * Update integration-SplunkPy.yml * fix diff * add last alert creation time stamp to last run, to prevent duplicate incidents --- Integrations/integration-FireEye_ETP.yml | 918 +++++++++++++++++++++++ 1 file changed, 918 insertions(+) create mode 100644 Integrations/integration-FireEye_ETP.yml diff --git a/Integrations/integration-FireEye_ETP.yml b/Integrations/integration-FireEye_ETP.yml new file mode 100644 index 000000000000..05500b2f5970 --- /dev/null +++ b/Integrations/integration-FireEye_ETP.yml @@ -0,0 +1,918 @@ +commonfields: + id: FireEye ETP + version: -1 +name: FireEye ETP +display: FireEye ETP +category: Email Gateway +image: data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHMAAAAhCAYAAADqBZQaAAAGqUlEQVR4Ae3ZA3AcfR/A8V/Y6GI711xSK8ldH9u2bdu2bdsoHte2bdt2+3u/M+9mZmdnb7N1b2Y782mM/X/zx+3Jnvg3qbIuFSGIZ//ZU98oGm+hFBLJvJjAJXgCEvm8mI3wN1pDIpkXE/gN70IinxezA7pCIpsXMwbj8AEksnkx67ACbXEd2kIikxfzRbyOIqzEoZDI48WMRQcU4Wgo7oJEHi9mBd6B4GIo5sEPiSBezOmBYJpxBygOT+I2jMdwZEMighcTWFgdqptbFSqfEwhWQHhfayzHVFRCIoAX00EQS7EA7SEOopGARi4kIAqCHJQhEbK3iWRHQQ5Mu/aFUUhDAYpFsniZ6YNYGTN0qaENJIzmGI1xmOhgKgYjH4IOUJy8D0KehtEYjGENGI+jD9SYCTgSL6EH5mA5VmAlFqELYa/Kjc2LHu6vLZlTFToP5iV3HkogNtphGxSrHazHHBRB0A2KM/dBzMuhhq3Y5kBxJmRviorNi0I0RPjPEV8QwNMYAw1jG2ZgPKYxI/883VcSmBEIPjW1Mvg8xJg5il8gNtpCMQuNkYMCB9EQtMHxyNkHMS+GYiiKUYbyMPxI2gchT8ZtaO4UsQRvYK0l2jKoyQAcyoyMhyAGh4mknntfduPERdXtJ0ysrLsUwoC8CcWJDjFnIg3ikg9ZiIMYElCKPAjScApuRR3EJAVBXI6bcC78iHaI2QfiUiZKkQ1xkItypEJM8nAULsWFqEWyETMRN+AZHGEXMQ3PWSLOxvsI4TGoYQJyIVY1CUVRkFmB4F2T/7/PRSMPW/BpAzMzA+JSR5s983jswJ9ogmlQw2cQw6WYAbXYjh+QGyZmP4hLJ2Iz5qAMYqMFlhpaQpCB97EOajENZxlB89EaPnPEGFyO6VCswdc4B6kQw+dQw/WQeguqQjHEu35eVegQCITHoOcZA3QwBF0xFtFhYk6HD+LSP1CcCjGcBMUkTMZqdEBHPATBPVBswze4CufiHoyFYjDXkwGBOWZfiEvR6AvFExAb70DRwRSyHxRz8Cyuwg34GtuhuNp6AGqEELpAsRQvohRi42sodqA9pB4X/wCehdSbyEywHFCewSIk2cTcji0YhH7ob2MUHmsg5glQQ18UQkzaYBMUV0IsktENyqryHAQiklUfcy0GY2gYI9EVORBcDcVsm+W2GEuhOAGC96EYhQKIxSVQLEaxOeZlWGX6YBOIgwehhhMtMXvhNIjhVii2oBUE92EOEmxiboW68IXLmKtQCbF4ymbJtToIihmzq4KpEJHMi12eZhWLTCFSMA6KayAmt0DRDYIsLMZ2BCFh/ATFneaYgoPQDYou8EHCqMZKKL6G1GNJ/YqZuM6YjQOghvcghtfQx2HPnIs2qEDARjXyXcYcBrHRAYqx6IJeFj3RHVugLLEhmGfmUJSgHBU2/ChFDMRwMxT9EA1BIkZDcR4EIezAenRAR3Sy+BljoPgZpphANO7BFgxDISSMp6HYgbMhkFH+2pzF1e07QQmrRF3DYLxvWVKH4KE9eJp1ijkYYuM/KOZgMqbZmIrRmEDIgywx+0B2Ui4WQnESBBdAMRkpEBwDxSbMxDwb8zEDs/GFNaZZBf7BCpwHCeMlKLbgYZagZMgTOdWxb+Q39Xcta1MzLRAsMN12E1yMQUjeg6dZp5hDIDY+hOJhCBrZiIYgnpAxlpj9ILvgOSg6QfA3FLdDDFXYjEXIRxySLRIQhRgkOMQEcBGGogNqITZOQAcsxwgu9mOC3iviK4GYMSgX4j80huznmOe5eJruQvTEsxCY98zekF1QhmVYhfOwHnORDTHEYTAUT0Fs+PA6fkVNwzGBRNyOwfgWZyAnzDdvhiqRjFQIZAZ/0QxCDV7GK8iE7OGYf+9CzDh0hWIcjoEPCcjGLdgA5RpOhsD80GQizsTZOCeMC9ECYvERFKugeB1icTbU8AwKEIt4BPAbFH3sZ6azeByLt9EZn+ImHI8qJELMuCfbgj3zTgbkbjSHNKAGivnIgrjU1ebe7GlQjICEkY3/oIaVmI1NUCzHOZabBldBd8IbEIsabDYFDUBsXI2NUGzCZMzCdii6IHd3nzVJQluci3twKdqJZPkgkJmBYBwXX450iEv5eAA3IAHi0rl4FJUQgx8P41KIg3icjY8wEGPxLx6CH2KJ2QrP4FE81oBncQzERkco3oM4aIan0B3TMAk/4WIkQKwxPftWAeZBcTDEpSiDAPBi7nfvm0+0O8+Lub/dgCEYBcVa1HoxI9NtWI7ZGIzTIHva/wAx/Zd/Mj383wAAAABJRU5ErkJggg== +description: 'FireEye Email Threat Prevention (ETP Cloud) is a cloud-based platform + that protects against advanced email attacks. ' +configuration: +- display: 'Server URL. Valid values: https://etp.us.fireeye.com, https://etp.eu.fireeye.com, + https://etp.us.fireeyegov.com ' + name: server + defaultvalue: https://etp.us.fireeye.com + type: 0 + required: true +- display: API key + name: api_key + defaultvalue: "" + type: 4 + required: true +- display: Trust any certificate (unsecure) + name: unsecure + defaultvalue: "true" + type: 8 + required: false +- display: Use system proxy settings + name: proxy + defaultvalue: "" + type: 0 + required: false +- display: Fetch incidents + name: isFetch + defaultvalue: "" + type: 8 + required: false +- display: Incident type + name: incidentType + defaultvalue: "" + type: 13 + required: false +- display: Messages status. All messages with the status specified will be imported + as incidents. + name: message_status + defaultvalue: delivered (retroactive) + type: 0 + required: false +script: + script: |- + from datetime import timedelta, datetime + import requests + import os + import re + import copy + import json + # disable insecure warnings + from requests.packages.urllib3.exceptions import InsecureRequestWarning + requests.packages.urllib3.disable_warnings(InsecureRequestWarning) + + def set_proxies(): + if demisto.params()['proxy']: + http = os.environ['http_proxy'] or os.environ['HTTP_PROXY'] + https = os.environ['https_proxy'] or os.environ['HTTPS_PROXY'] + proxies = { + 'http': http, + 'https': https + } + return proxies + return None + + + ''' + GLOBAL VARS + ''' + + API_KEY = demisto.params().get('api_key') + PROXIES = set_proxies() + BASE_PATH = '{}/api/v1'.format(demisto.params().get('server')) + HTTP_HEADERS = { + 'Content-Type': 'application/json' + } + USE_SSL = not demisto.params().get('unsecure') + MESSAGE_STATUS = demisto.params().get('message_status') + + ''' + SAERCH ATTRIBUTES VALID VALUES + ''' + + REJECTION_REASONS = [ + 'ETP102', 'ETP103', 'ETP104', 'ETP200', 'ETP201', 'ETP203','ETP204', 'ETP205', + 'ETP300', 'ETP301', 'ETP302', 'ETP401','ETP402', 'ETP403', 'ETP404', 'ETP405'] + + STATUS_VALUES = ["accepted", "deleted", "delivered", "delivered (retroactive)", "dropped", + "dropped oob", "dropped (oob retroactive)", "permanent failure", "processing", "quarantined", "rejected", "temporary failure"] + + ''' + BASIC FUNCTIONS + ''' + def listify(comma_separated_list): + + if isinstance(comma_separated_list, list): + return comma_separated_list + return comma_separated_list.split(',') + + def http_request(method, url, body=None, headers={}, url_params=None): + ''' + returns the http response + ''' + + #add API key to headers + headers['x-fireeye-api-key'] = API_KEY + + request_kwargs = { + 'headers': headers, + 'verify': USE_SSL, + 'proxies': PROXIES + } + + #add optional arguments if specified + if body is not None: + request_kwargs['data'] = json.dumps(body) + if url_params is not None: + request_kwargs['params'] = json.dumps(url_params) + + LOG('attempting {} request sent to {} with body:\n{}'.format(method, url, json.dumps(body, indent=4))) + response = requests.request( + method, + url, + **request_kwargs + ) + #handle request failure + if response.status_code not in range(200,205): + raise ValueError('Request failed with status code {}\n{}'.format(response.status_code, response.text)) + return response.json() + + def return_error_entry(message): + entry = { + 'Type': entryTypes['error'], + 'Contents': e.message, + 'ContentsFormat': formats['text'], + } + demisto.results(entry) + + def to_search_attribute_object(value, filter=None, is_list=False ,valid_values=None): + + values = listify(value) if is_list else value + if valid_values: + for val in values: + if val not in valid_values: + raise ValueError('{} is not a valid value'.format(val)) + + attribute = { + 'value': values + } + if filter: + attribute['filter'] = filter + return attribute + + + def format_search_attributes(from_email=None, from_email_not_in=None, recipients=None, + recipients_not_in=None, subject=None, from_accepted_date_time=None, to_accepted_date_time=None, + rejection_reason=None, sender_ip=None, status=None, status_not_in=None, last_modified_date_time=None, domains=None): + + search_attributes = {} + + #handle from_email attribute + if from_email and from_email_not_in: + raise ValueError('Only one of the followings can be specified: from_email, from_email_not_in') + if from_email: + search_attributes['fromEmail'] = to_search_attribute_object(from_email, filter='in', is_list=True) + elif from_email_not_in: + search_attributes['fromEmail'] = to_search_attribute_object(from_email_not_in, filter='not in', is_list=True) + + #handle recipients attributes + if recipients and recipients_not_in: + raise ValueError('Only one of the followings can be specified: recipients, recipients_not_in') + if recipients: + search_attributes['recipients'] = to_search_attribute_object(recipients, filter='in', is_list=True) + elif recipients_not_in: + search_attributes['recipients'] = to_search_attribute_object(recipients_not_in, filter='not in', is_list=True) + + #handle status attributes + if status and status_not_in: + raise ValueError('Only one of the followings can be specified: status, status_not_in') + if status: + search_attributes['status'] = to_search_attribute_object(status, filter='in', is_list=True, valid_values=STATUS_VALUES) + elif status_not_in: + search_attributes['status'] = to_search_attribute_object(status, filter='in', is_list=True, valid_values=STATUS_VALUES) + + if subject: + search_attributes['subject'] = to_search_attribute_object(subject, filter='in', is_list=True) + if rejection_reason: + search_attributes['rejectionReason'] = to_search_attribute_object(rejection_reason, is_list=True, valid_values=REJECTION_REASONS) + if sender_ip: + search_attributes['senderIP'] = to_search_attribute_object(sender_ip, filter='in', is_list=True) + if domains: + search_attributes['domains'] = to_search_attribute_object(domains, is_list=True) + if from_accepted_date_time and to_accepted_date_time: + search_attribute['period'] = { + 'range': { + 'fromAcceptedDateTime': from_accepted_date_time, + 'toAcceptedDateTime': to_accepted_date_time + } + } + if last_modified_date_time: + # try to parse '>timestamp' | '>=timestamp' | '', context_data['senderHeader'].replace('\\"','')) + context_data['from'] = match.group() if match else context_data['senderHeader'] + + if context_data.get('recipientHeader') is None: + context_data['recipients'] = [] + return context_data + + recipients = [] + for recipient_header in context_data.get('recipientHeader', []): + match = re.search('<(.*)>', recipient_header) + recipient_address = match.group() if match else recipient_header + recipients.append(recipient_address) + context_data['recipients'] = ','.join(recipients) + + return context_data + + def search_messages_request(attributes={}, has_attachments=None, max_message_size=None): + + url = '{}/messages/trace'.format(BASE_PATH) + body = { + 'attributes': attributes, + 'type': 'MessageAttributes', + 'size': max_message_size or 20 + } + if has_attachments is not None: + body['hasAttachments'] = has_attachments + response = http_request( + 'POST', + url, + body=body, + headers=HTTP_HEADERS + ) + # no results + if response['meta']['total'] == 0: + return [] + return response['data'] + + def search_messages_command(): + + args = demisto.args() + if args.has_key('size'): + # parse to int + args['size'] = int(args['size']) + if args.get('has_attachments') is not None: + # parse to boolean + args['hasAttachments'] = args['hasAttachments'] == 'true' + + search_attributes = format_search_attributes( + from_email=args.get('from_email'), + from_email_not_in=args.get('from_email_not_in'), + recipients=args.get('recipients'), + recipients_not_in=args.get('recipients_not_in'), + subject=args.get('subject'), + from_accepted_date_time=args.get('from_accepted_date_time'), + to_accepted_date_time=args.get('to_accepted_date_time'), + rejection_reason=args.get('rejection_reason'), + sender_ip=args.get('sender_ip'), + status=args.get('status'), + status_not_in=args.get('status_not_in'), + last_modified_date_time=args.get('last_modified_date_time'), + domains=args.get('domains') + ) + + #raw data + messages_raw = search_messages_request(search_attributes, args.get('hasAttachments'), args.get('size')) + + # create context data + messages_context = [message_context_data(message) for message in messages_raw] + + # create readable data + messages_readable_data = [readable_message_data(message) for message in messages_context] + messages_md_headers = [ + 'Message ID', + 'Accepted Time', + 'From', + 'Recipients', + 'Subject', + 'Message Status' + ] + md_table = tableToMarkdown( + 'FireEye ETP - Search Messages', + messages_readable_data, + headers=messages_md_headers + ) + + entry = { + 'Type': entryTypes['note'], + 'Contents': messages_raw, + 'ContentsFormat': formats['json'], + 'ReadableContentsFormat': formats['markdown'], + 'HumanReadable': md_table, + 'EntryContext': { + "FireEyeETP.Messages(obj.id==val.id)": messages_context + } + } + demisto.results(entry) + + def get_message_request(message_id): + + url = '{}/messages/{}'.format(BASE_PATH, message_id) + response = http_request( + 'GET', + url + ) + if response['meta']['total'] == 0: + return {} + return response['data'][0] + + def get_message_command(): + + # get raw data + raw_message = get_message_request(demisto.args()['message_id']) + + if raw_message: + # create context data + context_data = message_context_data(raw_message) + + # create readable data + message_readable_data = readable_message_data(context_data) + messages_md_headers = [ + 'Message ID', + 'Accepted Time', + 'From', + 'Recipients', + 'Subject', + 'Message Status' + ] + md_table = tableToMarkdown( + 'FireEye ETP - Get Message', + message_readable_data, + headers=messages_md_headers + ) + + entry = { + 'Type': entryTypes['note'], + 'Contents': raw_message, + 'ContentsFormat': formats['json'], + 'ReadableContentsFormat': formats['markdown'], + 'HumanReadable': md_table, + 'EntryContext': { + "FireEyeETP.Messages(obj.id==val.id)": context_data + } + } + demisto.results(entry) + # no results + else: + entry = { + 'Type': entryTypes['note'], + 'Contents': {}, + 'ContentsFormat': formats['text'], + 'ReadableContentsFormat': formats['markdown'], + 'HumanReadable': '### FireEye ETP - Get Message \n no results' + } + demisto.results(entry) + + def alert_readable_data_summery(alert): + return { + 'Alert ID': alert['id'], + 'Alert Timestamp': alert['alert']['timestamp'], + 'From': alert['email']['headers']['from'], + 'Recipients': '{}|{}'.format(alert['email']['headers']['to'], alert['email']['headers']['cc']), + 'Subject': alert['email']['headers']['subject'], + 'MD5': alert['alert'].get('malware_md5'), + 'URL/Attachment': alert['email']['attachment'], + 'Email Status': alert['email']['status'], + 'Email Accepted': alert['email']['timestamp']['accepted'], + 'Threat Intel': alert['ati'] + } + + def alert_readable_data(alert): + return { + 'Alert ID': alert['id'], + 'Alert Timestamp': alert['alert']['timestamp'], + 'From': alert['email']['headers']['from'], + 'Recipients': '{}|{}'.format(alert['email']['headers']['to'], alert['email']['headers']['cc']), + 'Subject': alert['email']['headers']['subject'], + 'MD5': alert['alert'].get('malware_md5'), + 'URL/Attachment': alert['email']['attachment'], + 'Email Status': alert['email']['status'], + 'Email Accepted': alert['email']['timestamp']['accepted'], + 'Sevirity': alert['alert']['severity'] + } + + def malware_readable_data(malware): + return { + 'Name': malware['name'], + 'Domain': malware.get('domain'), + 'Downloaded At': malware['downloaded_at'], + 'Executed At': malware['executed_at'], + 'Type': malware['stype'], + 'Submitted At': malware['submitted_at'], + 'SID': malware['sid'] + } + + def alert_context_data(alert): + context_data = copy.deepcopy(alert) + # remove 'attributes' level + context_data.update(context_data.pop('attributes', {})) + return context_data + + def get_alerts_request(legacy_id=None, from_last_modified_on=None, etp_message_id=None, size=None, raw_response=False): + + url = '{}/alerts'.format(BASE_PATH) + + # constract the body for the request + body = {} + attributes = {} + if legacy_id: + attributes['legacy_id'] = legacy_id + if etp_message_id: + attributes['etp_message_id'] = etp_message_id + if attributes: + body['attribute'] = attributes + if size: + body['size'] = size + if from_last_modified_on: + body['fromLastModifiedOn'] = from_last_modified_on + + response = http_request( + 'POST', + url, + body=body, + headers=HTTP_HEADERS + ) + if raw_response: + return response + if response['meta']['total'] == 0: + return [] + return response['data'] + + def get_alerts_command(): + + args = demisto.args() + + if args.has_key('size'): + args['size'] = int(args['size']) + + if args.has_key('legacy_id'): + args['legacy_id'] = int(args['legacy_id']) + + # get raw data + alerts_raw = get_alerts_request( + legacy_id=args.get('legacy_id'), + from_last_modified_on=args.get('from_last_modified_on'), + etp_message_id=args.get('etp_message_id'), + size=args.get('size') + ) + + # create context data + alerts_context = [alert_context_data(alert) for alert in alerts_raw] + + #create readable data + alerts_readable_data = [alert_readable_data_summery(alert) for alert in alerts_context] + alerts_summery_headers = [ + 'Alert ID', + 'Alert Timestamp', + 'Email Accepted', + 'From', + 'Recipients', + 'Subject', + 'MD5', + 'URL/Attachment', + 'Email Status', + 'Threat Intel' + ] + md_table = tableToMarkdown( + 'FireEye ETP - Get Alerts', + alerts_readable_data, + headers=alerts_summery_headers + ) + entry = { + 'Type': entryTypes['note'], + 'Contents': alerts_raw, + 'ContentsFormat': formats['json'], + 'ReadableContentsFormat': formats['markdown'], + 'HumanReadable': md_table, + 'EntryContext': { + "FireEyeETP.Alerts(obj.id==val.id)": alerts_context + } + } + demisto.results(entry) + + def get_alert_request(alert_id): + url = '{}/alerts/{}'.format(BASE_PATH, alert_id) + response = http_request( + 'GET', + url + ) + if response['meta']['total'] == 0: + return {} + return response['data'][0] + + def get_alert_command(): + + # get raw data + alert_raw = get_alert_request(demisto.args()['alert_id']) + + if alert_raw: + # create context data + alert_context = alert_context_data(alert_raw) + + # create readable data + readable_data = alert_readable_data(alert_context) + alert_md_table = tableToMarkdown( + 'Alert Details', + readable_data + ) + malware_data = [malware_readable_data(malware) for malware in alert_context['alert']['explanation']['malware_detected']['malware']] + malware_md_table = tableToMarkdown( + 'Malware Details', + malware_data + ) + + entry = { + 'Type': entryTypes['note'], + 'Contents': alert_raw, + 'ContentsFormat': formats['json'], + 'ReadableContentsFormat': formats['markdown'], + 'HumanReadable': '## FireEye ETP - Get Alert\n{}\n{}'.format(alert_md_table, malware_md_table), + 'EntryContext': { + "FireEyeETP.Alerts(obj.id==val.id)": alert_context + } + } + demisto.results(entry) + # no results + else: + entry = { + 'Type': entryTypes['note'], + 'Contents': {}, + 'ContentsFormat': formats['json'], + 'ReadableContentsFormat': formats['markdown'], + 'HumanReadable': '### FireEye ETP - Get Alert\nno results', + + } + demisto.results(entry) + + + def parse_string_in_iso_format_to_datetime(iso_format_string): + alert_last_modified = None + try: + alert_last_modified = datetime.strptime(iso_format_string, "%Y-%m-%dT%H:%M:%S.%f") + except ValueError: + try: + alert_last_modified = datetime.strptime(iso_format_string, "%Y-%m-%dT%H:%M:%S") + except ValueError: + alert_last_modified = datetime.strptime(iso_format_string, "%Y-%m-%dT%H:%M") + return alert_last_modified + + def parse_alert_to_incident(alert): + + context_data = alert_context_data(alert) + incident = { + 'name': context_data['email']['headers']['subject'], + 'rawJSON': json.dumps(context_data) + } + return incident + + def fetch_incidents(): + + last_run = demisto.getLastRun() + week_ago = datetime.now() - timedelta(days=7) + iso_format = "%Y-%m-%dT%H:%M:%S.%f" + + if not last_run.has_key('last_modified'): + # parse datetime to iso format string yyy-mm-ddThh:mm:ss.fff + last_run['last_modified'] = week_ago.strftime(iso_format)[:-3] + if not last_run.has_key('last_created'): + last_run['last_created'] = week_ago.strftime(iso_format) + + alerts_raw_response = get_alerts_request( + from_last_modified_on = last_run['last_modified'], + size = 100, + raw_response=True + ) + # end if no results returned + if not alerts_raw_response or not alerts_raw_response.has_key('data'): + return + + alerts = alerts_raw_response['data'] + last_alert_created = parse_string_in_iso_format_to_datetime(last_run['last_created']) + alert_creation_limit = parse_string_in_iso_format_to_datetime(last_run['last_created']) + incidents = [] + + for alert in alerts: + # filter by message status if specified + if MESSAGE_STATUS and alert['attributes']['email']['status'] != MESSAGE_STATUS: + continue + # filter alerts created before 'last_created' + current_alert_created = parse_string_in_iso_format_to_datetime(alert['attributes']['alert']['timestamp']) + if current_alert_created < alert_creation_limit: + continue + # append alert to incident + incidents.append(parse_alert_to_incident(alert)) + # set last created + if current_alert_created > last_alert_created: + last_alert_created = current_alert_created + + last_run['last_modified'] = alerts_raw_response['meta']['fromLastModifiedOn']['end'] + last_run['last_created'] = last_alert_created.strftime(iso_format) + + demisto.incidents(incidents) + demisto.setLastRun(last_run) + + ''' + EXECUTION + ''' + + try: + if demisto.command() == 'test-module': + alerts = get_alerts_request(size=1) + # request was succesful + demisto.results('ok') + if demisto.command() == 'fetch-incidents': + fetch_incidents() + if demisto.command() == 'fireeye-etp-search-messages': + search_messages_command() + if demisto.command() == 'fireeye-etp-get-message': + get_message_command() + if demisto.command() == 'fireeye-etp-get-alerts': + get_alerts_command() + if demisto.command() == 'fireeye-etp-get-alert': + get_alert_command() + except ValueError as e: + LOG(e.message) + LOG.print_log() + return_error_entry(e.message) + type: python + commands: + - name: fireeye-etp-search-messages + arguments: + - name: from_email + description: 'List of ''From'' email-addresses, max limit of entries is 10. ' + - name: from_email_not_in + description: 'List of ''From'' email-addresses not to be included, max limit + of entries is 10. ' + - name: recipients + description: List of 'To'/'Cc' email-addresses, max limit of entries is 10. + - name: recipients_not_in + description: 'list of ''To''/''Cc'' email-addresses not to be included, max + limit of entries is 10. ' + - name: subject + description: List of strings, max limit of entries is 10. + - name: from_accepted_date_time + description: ' The time stamp of the email-accepted date to specify the beginning + of the date range to search, e.g. 2017-10- 24T10:48:51.000Z . Specify ''to_accepted_date_time'' as + well to set the complete date range for the search.' + - name: to_accepted_date_time + description: ' The time stamp of the email-accepted date to specify the end + of the date range to search, e.g. 2017-10- 24T10:48:51.000Z . Specify ''from_accepted_date_time'' as + well to set the complete date range for the search.' + - name: rejection_reason + description: 'list of ETP rejection reason codes ( "ETP102", "ETP103", "ETP104", + "ETP200", "ETP201", "ETP203", "ETP204", "ETP205", "ETP300", "ETP301", "ETP302", + "ETP401", "ETP402", "ETP403", "ETP404", "ETP405") ' + - name: sender_ip + description: List of sender IP addresses, max limit of entries is 10. + - name: status + description: List of email status values( "accepted", "deleted", "delivered", + "delivered (retroactive)", "dropped", "dropped oob", "dropped (oob retroactive)", + "permanent failure", "processing", "quarantined", "rejected", "temporary failure"). + - name: status_not_in + description: List of email status values not to include( "accepted", "deleted", + "delivered", "delivered (retroactive)", "dropped", "dropped oob", "dropped + (oob retroactive)", "permanent failure", "processing", "quarantined", "rejected", + "temporary failure"). + - name: last_modified_date_time + description: 'Date corresponding to last modified date, along with one of the + following operators: ">", "<", ">=", "<=". E.g. use value "<2017-10-24T18:00:00.000Z" + to search for messages that were last modified after the specified time stamp.' + - name: domain + description: List of domain names. + - name: has_attachments + auto: PREDEFINED + predefined: + - "true" + - "false" + description: Boolean value to indicate if the message has attachments. + - name: max_message_size + description: The default value is 20kb and maximum value is 100kb. + outputs: + - contextPath: FireEyeETP.Message.acceptedDateTime + description: Message accepted date. + - contextPath: FireEyeETP.Message.countryCode + description: Sender country code. + - contextPath: FireEyeETP.Message.domain + description: Domain. + - contextPath: FireEyeETP.Message.emailSize + description: Email size in kb. + - contextPath: FireEyeETP.Message.lastModifiedDateTime + description: Message last modified date. + - contextPath: FireEyeETP.Message.recipientHeader + description: List of message recipients header (includes the display name of + the user). + - contextPath: FireEyeETP.Message.recipients + description: List of message recipients. + - contextPath: FireEyeETP.Message.senderHeader + description: Message sender header (includes the display name of the user). + - contextPath: FireEyeETP.Message.sender + description: Message sender address. + - contextPath: FireEyeETP.Message.senderSMTP + description: Message sender SMTP. + - contextPath: FireEyeETP.Message.senderIP + description: Message sender IP. + - contextPath: FireEyeETP.Message.status + description: Message status. + - contextPath: FireEyeETP.Message.subject + description: Message subject + - contextPath: FireEyeETP.Message.verdicts.AS + description: pass/fail verdict for AS. + - contextPath: FireEyeETP.Message.verdicts.AV + description: pass/fail verdict for AV + - contextPath: FireEyeETP.Message.verdicts.AT + description: pass/fail verdict for AT + - contextPath: FireEyeETP.Message.verdicts.PV + description: pass/fail verdict for PV + - contextPath: FireEyeETP.Message.id + description: Message ID. + description: Search for messages that include specified message attributes that + are accessible in he ETP portal. + - name: fireeye-etp-get-message + arguments: + - name: message_id + required: true + description: The message ID. + outputs: + - contextPath: FireEyeETP.Message.acceptedDateTime + description: Message accepted date. + - contextPath: FireEyeETP.Message.countryCode + description: Sender country code. + - contextPath: FireEyeETP.Message.domain + description: Domain. + - contextPath: FireEyeETP.Message.emailSize + description: Email size in kb. + - contextPath: FireEyeETP.Message.lastModifiedDateTime + description: Message last modified date. + - contextPath: FireEyeETP.Message.recipientHeader + description: List of message recipients header (includes the display name of + the user). + - contextPath: FireEyeETP.Message.recipients + description: List of message recipients. + - contextPath: FireEyeETP.Message.senderHeader + description: Message sender header (includes the display name of the user). + - contextPath: FireEyeETP.Message.sender + description: Message sender address. + - contextPath: FireEyeETP.Message.senderSMTP + description: Message sender SMTP. + - contextPath: FireEyeETP.Message.senderIP + description: Message sender IP. + - contextPath: FireEyeETP.Message.status + description: Message status. + - contextPath: FireEyeETP.Message.subject + description: Message subject + - contextPath: FireEyeETP.Message.verdicts.AS + description: pass/fail verdict for AS. + - contextPath: FireEyeETP.Message.verdicts.AV + description: pass/fail verdict for AV + - contextPath: FireEyeETP.Message.verdicts.AT + description: pass/fail verdict for AT + - contextPath: FireEyeETP.Message.verdicts.PV + description: pass/fail verdict for PV + - contextPath: FireEyeETP.Message.id + description: Message ID. + description: Get the data of a specific message. + - name: fireeye-etp-get-alerts + arguments: + - name: legacy_id + description: Alert ID as shown in ETP Web Portal. + - name: from_last_modified_on + description: Datetime in yyy-mm-ddThh:mm:ss.fff format. Default last 90 days. + - name: etp_message_id + description: Email message id. + - name: size + description: Number of alerts intended in response. Default 20. Valid range + 1-100 . + outputs: + - contextPath: FireEyeETP.Alerts.meta.read + description: Email read flag. + - contextPath: FireEyeETP.Alerts.meta.last_modified_on + description: Last modified timestamp. + - contextPath: FireEyeETP.Alerts.meta.legacy_id + description: 'Alert ID as shown in ETP Web Portal ' + - contextPath: FireEyeETP.Alerts.alert.product + description: Product alerted + - contextPath: FireEyeETP.Alerts.alert.timestamp + description: Alert timestamp + - contextPath: FireEyeETP.Alerts.alert.malware_md5 + description: md5 of file attached + - contextPath: FireEyeETP.Alerts.email.status + description: The email status. + - contextPath: FireEyeETP.Alerts.email.source_ip + description: Email source IP. + - contextPath: FireEyeETP.Alerts.email.smtp.rcpt_to + description: Recipient SMTP. + - contextPath: FireEyeETP.Alerts.email.smtp.mail_from + description: Sender SMTP. + - contextPath: FireEyeETP.Alerts.email.etp_message_id + description: The message ID. + - contextPath: FireEyeETP.Alerts.email.headers.cc + description: Email 'cc' recipients. + - contextPath: FireEyeETP.Alerts.email.headers.to + description: Email recipients. + - contextPath: FireEyeETP.Alerts.email.headers.from + description: Email sender. + - contextPath: FireEyeETP.Alerts.email.headers.subject + description: Email subject. + - contextPath: FireEyeETP.Alerts.email.attachment + description: File name or URL pointing to file. + - contextPath: FireEyeETP.Alerts.email.timestamp.accepted + description: Email accepted time. + - contextPath: FireEyeETP.Alerts.id + description: The alert ID. + description: Get summary format information about the alerts. + - name: fireeye-etp-get-alert + arguments: + - name: alert_id + required: true + description: The alert ID. + outputs: + - contextPath: FireEyeETP.Alerts.meta.read + description: Email read flag. + - contextPath: FireEyeETP.Alerts.meta.last_modified_on + description: Last modified timestamp. + - contextPath: FireEyeETP.Alerts.meta.legacy_id + description: 'Alert ID as shown in ETP Web Portal ' + - contextPath: FireEyeETP.Alerts.meta.acknowledged + description: Acknowledged + - contextPath: FireEyeETP.Alerts.alert.product + description: Product generate the alert. + - contextPath: FireEyeETP.Alerts.alert.alert_type + description: Alert type code. + - contextPath: FireEyeETP.Alerts.alert.severity + description: Severity code. + - contextPath: FireEyeETP.Alerts.alert.explanation.analysis + description: Analysis + - contextPath: FireEyeETP.Alerts.alert.explanation.anomaly + description: Anomaly + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.domain + description: Malware domain + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.downloaded_at + description: Malware downloaded at timestamp + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.executed_at + description: Malware executed at timestamp + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.name + description: Malware name + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.sid + description: Malware sid + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.stype + description: Malware type + - contextPath: FireEyeETP.Alerts.alert.explanation.malware_detected.malware.submitted_at + description: Malware submitted at + - contextPath: FireEyeETP.Alerts.alert.explanation.protocol + description: Protocol + - contextPath: FireEyeETP.Alerts.alert.explanation.timestamp + description: Explanation timestamp + - contextPath: FireEyeETP.Alerts.alert.timestamp + description: Alert timestamp. + - contextPath: FireEyeETP.Alerts.alert.action + description: Alert acrion + - contextPath: FireEyeETP.Alerts.alert.name + description: Alert name. + - contextPath: FireEyeETP.Alerts.email.status + description: The email status. + - contextPath: FireEyeETP.Alerts.email.source_ip + description: Email source IP. + - contextPath: FireEyeETP.Alerts.email.smtp.rcpt_to + description: Recipient SMTP. + - contextPath: FireEyeETP.Alerts.email.smtp.mail_from + description: Sender SMTP. + - contextPath: FireEyeETP.Alerts.email.etp_message_id + description: FE ETP unique message ID. + - contextPath: FireEyeETP.Alerts.email.headers.cc + description: Email cc recipients. + - contextPath: FireEyeETP.Alerts.email.headers.to + description: Email recipients. + - contextPath: FireEyeETP.Alerts.email.headers.from + description: 'Email sender ' + - contextPath: FireEyeETP.Alerts.email.headers.subject + description: Email subject + - contextPath: FireEyeETP.Alerts.email.attachment + description: File name or URL pointing to file + - contextPath: FireEyeETP.Alerts.email.timestamp.accepted + description: Email eccepted time + - contextPath: FireEyeETP.Alerts.id + description: The alert unique ID + description: Detailed information from any particular alert. Alerts more than + 90 days old are not available. + isfetch: true + runonce: false