Merge pull request #41 from DomainTools/1.5.0

Merged 1.5.0 splunk SOAR to next
DomainTools · Feb 19, 2024 · 452513a · 452513a
2 parents cb9c257 + 65fd55b
commit 452513a
Show file tree

Hide file tree

Showing 48 changed files with 534 additions and 93 deletions.
diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
@@ -1,7 +1,7 @@
 name: Linting
 on: [push, pull_request]
 jobs:
-  lint: 
+  lint:
     # Run per push for internal contributers. This isn't possible for forked pull requests,
     # so we'll need to run on PR events for external contributers.
     # String comparison below is case insensitive.

diff --git a/LICENSE b/LICENSE
@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright (c) 2019-2023 DomainTools, LLC
+   Copyright (c) 2019-2024 DomainTools, LLC
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
@@ -198,4 +198,4 @@
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
-   limitations under the License.
+   limitations under the License.
diff --git a/NOTICE b/NOTICE
@@ -1,5 +1,5 @@
 Splunk SOAR DomainTools Iris Investigate
-Copyright (c) 2019-2023 DomainTools, LLC
+Copyright (c) 2019-2024 DomainTools, LLC
 
 Third-party Software Attributions:
 

diff --git a/README.md b/README.md
@@ -2,16 +2,16 @@
 # DomainTools Iris Investigate
 
 Publisher: DomainTools  
-Connector Version: 1.4.1  
+Connector Version: 1.5.0  
 Product Vendor: DomainTools  
 Product Name: DomainTools Iris Investigate  
 Product Version Supported (regex): ".\*"  
-Minimum Product Version: 5.5.0  
+Minimum Product Version: 6.1.1  
 
 This app supports investigative actions to profile domain names, get risk scores, and find connected domains that share the same Whois details, web hosting profiles, SSL certificates, and more on DomainTools Iris Investigate
 
 [comment]: # " File: README.md"
-[comment]: # "  Copyright (c) 2019-2023 DomainTools, LLC"
+[comment]: # "  Copyright (c) 2019-2024 DomainTools, LLC"
 [comment]: # ""
 [comment]: # "Licensed under the Apache License, Version 2.0 (the 'License');"
 [comment]: # "you may not use this file except in compliance with the License."
@@ -24,8 +24,55 @@ This app supports investigative actions to profile domain names, get risk scores
 [comment]: # "either express or implied. See the License for the specific language governing permissions"
 [comment]: # "and limitations under the License."
 [comment]: # ""
-**Note:** For the playbooks on the domain tools data, visit
-[this](https://github.com/DomainTools/playbooks/tree/master/Splunk%20Phantom) Github repository.
+
+[comment]: # "Monitoring/Scheduling Playbook(s) feature"
+## DomainTools Iris Investigate Monitoring Playbook Feature
+This feature allows the user to schedule playbooks to run on an specified interval and run it on a specific container/event ID you provided on each row. Coupled with our reference playbooks, linked below, this can be a powerful tool to notify you of domain infrastructure changes, or when newly created domains match specific infrastructure you're monitoring. See the individual playbooks for more information. This readme covers how to set up Iris Monitoring for those playbooks.
+
+### Configuration
+This feature depends on the 1 asset configuration fields that are **required** when using this feature.
+| **Name** | **Description** | **Default Value** | **Required** |
+| --- | --- | --- | --- |
+| Splunk SOAR HTTPS port (default: 8443) | Splunk SOAR HTTP port if your instance uses one other than the default, 8443 | 8443 | Yes |
+
+To configure this, you need to:
+1. Go to **Apps**
+2. Select **DomainTools Iris Investigate**
+3. Select a configured asset or create one if you don't have any.
+4. Go to **Asset Settings**
+5. Look for `Splunk SOAR HTTPS port (default: 8443)` field. By default it contains `8443` value.
+
+
+### Prerequisites
+This feature uses a custom list named `domaintools_scheduled_playbooks`. <br>
+To generate the custom list, you need to:
+1. Go to **Apps**
+2. Select **DomainTools Iris Investigate**
+3, Select a configured asset or create one if you don't have any.
+4. Go to **Actions** dropdown then;
+5. Select '`configure scheduled playbooks`' action, then;
+6. Hit `Test Action`.
+
+If you go back to custom list page. you should have the `domaintools_scheduled_playbooks` generated for you.
+
+**Note:** The values of this list has 6 columns and the header should not be altered. The last 3 columns are intentionally left blank and used by the playbook scheduler.<br>
+**Sample domaintools_scheduled_playbooks table:**
+| **repo/playbook_name** | **event_id** | **interval (mins)** | **last_run (server time)** | **last_run_status** | **remarks** |
+| --- | --- | --- | --- | --- | --- |
+| `local/DomainTools Monitor Domain Risk Score`| `<your_event_id>` | 1440 | | | |
+| `local/DomainTools Monitor Domain Infrastructure`| `<your_event_id>` | 1440 | | | |
+| `local/DomainTools Monitor Search Hash`| `<your_event_id>` | 1440 | | | |
+In this example, we've specified to run three separate monitoring playbooks on daily schedules. Note that each scheduled lookup will consume Iris Investigate queries, depending how many domains or Iris search hashes are being monitored.<br>
+
+### How to use monitoring/scheduling feature in DomainTools Iris Investigate App
+1. Under **Apps** > **DomainTools Iris Investigate** > **Asset Settings** > **Ingest Settings** > **Label**, specify or select a label to apply to objects from this source. <br>
+**Recommended:** Use a custom label rather using a predefined label like `events`.
+2. Specify a polling interval to check if playbooks need to be run. Note that this is separate from the playbook run interval specified in step 4. We **recommend** running **every minute** for the most accurate scheduling.
+3. Under Custom Lists > `domaintools_scheduled_playbooks` input your desired playbook schedule following the example in the Configuration Section<br>
+**Note:** Make sure the label of the **playbook** and **event_id** you inputted shares the label that you selected in *Step 1*. The `domaintools_scheduled_playbooks` custom list should have been created when you updated our installed the DomainTools app, but if you don't see it, you can generate it by following the **Prerequisites** section of this page.
+
+**Note:** For the DomainTools reference playbooks, see
+[this](https://github.com/DomainTools/playbooks/tree/main/Splunk%20SOAR) Github repository.
 
 
 ### Configuration Variables
@@ -44,6 +91,7 @@ VARIABLE | REQUIRED | TYPE | DESCRIPTION
 **custom_ssl_certificate** |  optional  | boolean | Use Custom SSL Certificate
 **ssl** |  optional  | boolean | Use SSL
 **custom_ssl_certificate_path** |  optional  | string | Custom SSL Certificate Path
+**http_port** |  optional  | string | Splunk SOAR HTTPS port (default: 8443)
 
 ### Supported Actions  
 [test connectivity](#action-test-connectivity) - Validate the asset configuration for connectivity  
@@ -55,6 +103,8 @@ VARIABLE | REQUIRED | TYPE | DESCRIPTION
 [reverse email](#action-reverse-email) - Find domains with email in Whois, DNS SOA or SSL certificate  
 [lookup domain](#action-lookup-domain) - Get all Iris Investigate data for a domain using the Iris Investigate API endpoint (required)  
 [enrich domain](#action-enrich-domain) - Get all Iris Investigate data for a domain except counts using the high volume Iris Enrich API endpoint (if provisioned)  
+[configure scheduled playbooks](#action-configure-scheduled-playbooks) - Run on initial setup to configure the optional monitoring playbooks. This action creates a custom list to manage the playbook scheduling and run status  
+[on poll](#action-on-poll) - Execute scheduled playbooks based on the set interval(mins) in 'domaintools_scheduled_playbooks' custom list. Smaller intervals will result in more accurate schedules  
 
 ## action: 'test connectivity'
 Validate the asset configuration for connectivity
@@ -554,4 +604,35 @@ action_result.data.\*.website_title.value | string |  |
 action_result.summary | string |  |  
 action_result.message | string |  |  
 summary.total_objects | numeric |  |   1 
-summary.total_objects_successful | numeric |  |   1 
+summary.total_objects_successful | numeric |  |   1   
+
+## action: 'configure scheduled playbooks'
+Run on initial setup to configure the optional monitoring playbooks. This action creates a custom list to manage the playbook scheduling and run status
+
+Type: **investigate**  
+Read only: **True**
+
+#### Action Parameters
+No parameters are required for this action
+
+#### Action Output
+DATA PATH | TYPE | CONTAINS | EXAMPLE VALUES
+--------- | ---- | -------- | --------------
+action_result.status | string |  |   failed  success 
+action_result.data.\* | string |  |  
+action_result.summary | string |  |  
+action_result.message | string |  |  
+summary.total_objects | numeric |  |   1 
+summary.total_objects_successful | numeric |  |   1   
+
+## action: 'on poll'
+Execute scheduled playbooks based on the set interval(mins) in 'domaintools_scheduled_playbooks' custom list. Smaller intervals will result in more accurate schedules
+
+Type: **ingest**  
+Read only: **True**
+
+#### Action Parameters
+No parameters are required for this action
+
+#### Action Output
+No Output
diff --git a/__init__.py b/__init__.py
@@ -1,6 +1,6 @@
 # --
 # File: __init__.py
 #
-# Copyright (c) 2019-2023 DomainTools, LLC
+# Copyright (c) 2019-2024 DomainTools, LLC
 #
 # --
diff --git a/domaintools_iris.json b/domaintools_iris.json
@@ -5,14 +5,14 @@
     "publisher": "DomainTools",
     "package_name": "phantom_domaintools_iris",
     "type": "information",
-    "license": "Copyright (c) 2019-2023 DomainTools, LLC",
+    "license": "Copyright (c) 2019-2024 DomainTools, LLC",
     "main_module": "domaintools_iris_connector.py",
-    "app_version": "1.4.1",
-    "utctime_updated": "2023-07-14T15:44:30.000000Z",
+    "app_version": "1.5.0",
+    "utctime_updated": "2023-10-25T15:44:30.000000Z",
     "product_vendor": "DomainTools",
     "product_name": "DomainTools Iris Investigate",
     "product_version_regex": ".*",
-    "min_phantom_version": "5.5.0",
+    "min_phantom_version": "6.1.1",
     "python_version": "3",
     "logo": "logo_domaintools_iris.svg",
     "logo_dark": "logo_domaintools_iris_dark.svg",
@@ -91,6 +91,12 @@
             "description": "Custom SSL Certificate Path",
             "data_type": "string",
             "order": 10
+        },
+        "http_port": {
+            "description": "Splunk SOAR HTTPS port (default: 8443)",
+            "data_type": "string",
+            "default": "8443",
+            "order": 11
         }
     },
     "actions": [
@@ -249,7 +255,9 @@
                         "google_analytics",
                         "adsense",
                         "asn",
-                        "isp_name"
+                        "isp_name",
+                        "tagged_with_any",
+                        "tagged_with_all"
                     ],
                     "required": true,
                     "order": 0
@@ -2039,95 +2047,134 @@
                 }
             ],
             "versions": "EQ(*)"
+        },
+        {
+            "action": "configure scheduled playbooks",
+            "description": "Run on initial setup to configure the optional monitoring playbooks. This action creates a custom list to manage the playbook scheduling and run status",
+            "type": "investigate",
+            "identifier": "configure_monitoring_scheduled_playbooks",
+            "read_only": true,
+            "parameters": {},
+            "output": [
+                {
+                    "data_path": "action_result.status",
+                    "data_type": "string",
+                    "example_values": [
+                        "failed",
+                        "success"
+                    ]
+                },
+                {
+                    "data_path": "action_result.data.*",
+                    "data_type": "string"
+                },
+                {
+                    "data_path": "action_result.summary",
+                    "data_type": "string"
+                },
+                {
+                    "data_path": "action_result.message",
+                    "data_type": "string"
+                },
+                {
+                    "data_path": "summary.total_objects",
+                    "data_type": "numeric",
+                    "example_values": [
+                        1
+                    ]
+                },
+                {
+                    "data_path": "summary.total_objects_successful",
+                    "data_type": "numeric",
+                    "example_values": [
+                        1
+                    ]
+                }
+            ],
+            "versions": "EQ(*)"
+        },
+        {
+            "action": "on poll",
+            "description": "Execute scheduled playbooks based on the set interval(mins) in 'domaintools_scheduled_playbooks' custom list. Smaller intervals will result in more accurate schedules",
+            "type": "ingest",
+            "identifier": "on_poll",
+            "read_only": true,
+            "parameters": {},
+            "output": [],
+            "versions": "EQ(*)"
         }
     ],
     "pip39_dependencies": {
-        "pypi": [
-            {
-                "module": "regex"
-            }
-        ],
         "wheel": [
             {
                 "module": "anyio",
-                "input_file": "wheels/anyio-3.7.1-py3-none-any.whl"
-            },
-            {
-                "module": "certifi",
-                "input_file": "wheels/certifi-2023.5.7-py3-none-any.whl"
-            },
-            {
-                "module": "charset-normalizer",
-                "input_file": "wheels/charset_normalizer-3.1.0-py3-none-any.whl"
+                "input_file": "wheels/py3/anyio-4.2.0-py3-none-any.whl"
             },
             {
                 "module": "dateparser",
-                "input_file": "wheels/dateparser-1.1.8-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/dateparser-1.2.0-py2.py3-none-any.whl"
             },
             {
-                "module": "domaintools-api",
-                "input_file": "wheels/domaintools_api-1.0.1-py2.py3-none-any.whl"
+                "module": "domaintools_api",
+                "input_file": "wheels/shared/domaintools_api-1.0.1-py2.py3-none-any.whl"
             },
             {
                 "module": "exceptiongroup",
-                "input_file": "wheels/exceptiongroup-1.1.2-py3-none-any.whl"
+                "input_file": "wheels/py3/exceptiongroup-1.2.0-py3-none-any.whl"
             },
             {
                 "module": "filelock",
-                "input_file": "wheels/filelock-3.12.2-py3-none-any.whl"
+                "input_file": "wheels/py3/filelock-3.13.1-py3-none-any.whl"
             },
             {
                 "module": "h11",
-                "input_file": "wheels/h11-0.14.0-py3-none-any.whl"
+                "input_file": "wheels/py3/h11-0.14.0-py3-none-any.whl"
             },
             {
                 "module": "httpcore",
-                "input_file": "wheels/httpcore-0.17.3-py3-none-any.whl"
+                "input_file": "wheels/py3/httpcore-1.0.2-py3-none-any.whl"
             },
             {
                 "module": "httpx",
-                "input_file": "wheels/httpx-0.24.1-py3-none-any.whl"
-            },
-            {
-                "module": "idna",
-                "input_file": "wheels/idna-3.4-py3-none-any.whl"
+                "input_file": "wheels/py3/httpx-0.26.0-py3-none-any.whl"
             },
             {
                 "module": "python_dateutil",
-                "input_file": "wheels/python_dateutil-2.8.2-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/python_dateutil-2.8.2-py2.py3-none-any.whl"
             },
             {
                 "module": "pytz",
-                "input_file": "wheels/pytz-2023.3-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/pytz-2024.1-py2.py3-none-any.whl"
             },
             {
-                "module": "requests-file",
-                "input_file": "wheels/requests_file-1.5.1-py2.py3-none-any.whl"
+                "module": "regex",
+                "input_file": "wheels/py39/regex-2023.12.25-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
             },
             {
-                "module": "requests",
-                "input_file": "wheels/requests-2.31.0-py3-none-any.whl"
+                "module": "requests_file",
+                "input_file": "wheels/shared/requests_file-2.0.0-py2.py3-none-any.whl"
             },
             {
                 "module": "six",
-                "input_file": "wheels/six-1.16.0-py2.py3-none-any.whl"
+                "input_file": "wheels/shared/six-1.16.0-py2.py3-none-any.whl"
             },
             {
                 "module": "sniffio",
-                "input_file": "wheels/sniffio-1.3.0-py3-none-any.whl"
+                "input_file": "wheels/py3/sniffio-1.3.0-py3-none-any.whl"
             },
             {
                 "module": "tldextract",
-                "input_file": "wheels/tldextract-3.4.4-py3-none-any.whl"
+                "input_file": "wheels/py3/tldextract-3.4.4-py3-none-any.whl"
             },
             {
-                "module": "tzlocal",
-                "input_file": "wheels/tzlocal-5.0.1-py3-none-any.whl"
+                "module": "typing_extensions",
+                "input_file": "wheels/py3/typing_extensions-4.9.0-py3-none-any.whl"
             },
             {
-                "module": "urllib3",
-                "input_file": "wheels/urllib3-2.0.3-py3-none-any.whl"
+                "module": "tzlocal",
+                "input_file": "wheels/py3/tzlocal-5.2-py3-none-any.whl"
             }
         ]
-    }
-}
+    },
+    "executable": "spawn3"
+}