Skip to content

Latest commit

 

History

History
103 lines (93 loc) · 6.26 KB

manual_readme_content.md

File metadata and controls

103 lines (93 loc) · 6.26 KB
Raw

Preprocess Script

The user can add a script file in the configuration parameter Script with functions to preprocess containers and artifacts . The script must contain a function with the name preprocess_container (to pre-process the containers and the artifacts) or else, it will throw an error.

App ID

  • Optionally, you can specify an App ID to be used with the Streaming API. If one isn't set, it will default to the asset id.
  • It is recommended to have a unique App ID for each connection to the Streaming API. That is to say, if you are planning on having multiple assets using the Streaming API at once, you should give them unique App IDs.

On Poll

  • Common points for both manual and scheduled interval polling
    • Default parameters of the On Poll action are ignored in the app. i.e. start_time, end_time, container_count, artifact_count
    • The app will fetch all the events based on the value specified in the configuration parameters [Maximum events to get while POLL NOW] (default 2000 if not specified) and [Maximum events to get while scheduled and interval polling] (default 10,000 if not specified). For ingestion, the events are fetched after filtering them based on the event type - DetectionSummaryEvent . The app will exit from the polling cycle in the below-mentioned 2 cases whichever is earlier.
      • If the total DetectionSummaryEvents fetched equals the value provided in the [Maximum events to get while POLL NOW] (for manual polling) or [Maximum events to get while scheduled and interval polling] (for scheduled | interval polling) parameters
      • If the total number of continuous blank lines encountered while streaming the data equals the value provided in the [Maximum allowed continuous blank lines] (default 50 if not specified) asset configuration parameter
    • The default behavior of the app is that each event will be placed in its container. By checking the configuration parameter [Merge containers for Hostname and Eventname] as well as specifying an interval in the configuration parameter [Merge same containers within specified seconds], all events which are of the same type and on the same host will be put into one container, as long as the time between those two events is less than the interval.
    • The [Maximum allowed continuous blank lines] parameter will be used to indicate the allowed number of continuous blank lines while fetching DetectionSummaryEvents . For example, of the entire data of the DetectionSummaryEvents, some of the 'DetectionSummaryEvents' exists after 100 continuous blank lines and if you've set the [Maximum allowed continues blank lines] parameter value to 500, it will keep on ingesting all the 'DetectionSummaryEvents' until the code gets 500 continuous blank lines and hence, it will be able to cover the DetectionSummaryEvents successfully even after the 100 blank lines. If you set it to 50, it will break after the 50th blank line is encountered. Hence, it won't be able to ingest the events which exist after the 100 continuous blank lines because the code considers that after the configured value in the [Maximum allowed continuous blank lines] configuration parameter (here 50), there is no data available for the 'DetectionSummaryEvents'.
  • Manual Polling
    • During manual poll now, the app starts from the 1st event that it can query up to the value configured in the configuration parameter [Maximum events to get while POLL NOW] and creates artifacts for all the fetched DetectionSummaryEvents. The last queried event's offset ID will not be remembered in Manual POLL NOW and it fetches everything every time from the beginning.
  • Scheduled | Interval Polling
    • During scheduled | interval polling, the app starts from the 1st event that it can query up to the value configured in the configuration parameter [Maximum events to get while scheduled and interval polling] and creates artifacts for all the fetched DetectionSummaryEvents. Then, it remembers the last event's offset ID and stores in the state file against the key [last_offset_id]. In the next scheduled poll run, it will start from the stored offset ID in the state file and will fetch the maximum events as configured in the [Maximum events to get while scheduled and interval polling] parameter.

The DetectionSummaryEvent is parsed to extract the following values into an Artifact.

Artifact Field Event Field
cef.sourceUserName UserName
cef.fileName FileName
cef.filePath FilePath
cef.sourceHostName ComputerName
cef.sourceNtDomain MachineDomain
cef.hash MD5String
cef.hash SHA1String
cef.hash SHA256STring
cef.cs1 cmdLine

The App also parses the following sub-events into their own Artifacts.

  • Documents Accessed
  • Executables Written
  • Network Access
  • Scan Result
  • Quarantine Files
  • DNS Requests

Each of the sub-events has a CEF key called parentSdi that stands for Parent Source Data Identifier. This is the value of the SDI of the main event that the sub-events were generated from.