[CTI] adds Risky Host Overview Card

[ecezalp]

Summary

Risky Hosts Overview Card

Inspect Query

Sorted by Current Risk

Updates to Threat Intel Card to match the new designs

When there are no events in the selected time range

notes to design

• "view source" links were designed to be visible on hover, for now left visible at all times to match with the Threat Intel card.
• "No items found" text is visible when there are no items found, this was default EUI behavior, when I replaced with emptyness the border-bottom appeared awkwardly as a line about 10pxs under the column title. I could conditionally hide it with CSS but that felt a bit hacky, so I left the default behavior in for now.

testing scenario

0. Log into the security app and visit the overview page, and confirm that there is no Risky Hosts card. Confirm that there is an updated view for the Threat Intel card. Add xpack.securitySolution.enableExperimental: ['riskyHostsEnabled'] to kibana.dev.yml

1. Log into the security app and navigate to the overview page. Confirm the presence of the Risky Hosts card with a warning message and no data. Clicking the button takes the user to a documentation file on github.

2. if you are using a custom .siem-signals index as per your kibana.dev.yml, run the following reindex query to obtain a .siem-signals-default-* index
   note: this step is required for properly loading Drilldown of Host Risk Score dashboard, as it looks for the index pattern .siem-signals-default-*

Create default .siem-signals index

POST _reindex
{
  "source": {
    "index": "my-custom-siem-signals-index-name"
  },
  "dest": {
    "index": ".siem-signals-default-test"
  }
}

3. make the following requests in Kibana Dev Tools

Create ml_host_risk_score_latest index

PUT ml_host_risk_score_latest_default
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "host": {
        "properties": {
          "name": {
            "type": "keyword"
           }
         }
       },
      "ingest_timestamp": {
        "type": "date"
      },
      "risk": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "risk_score": {
        "type": "float"
      }
    }
  }
}

Add data to ml_host_risk_score_latest index

POST _bulk
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-02-23T23:00:00.000Z","risk_score":21,"host":{"name":"ip-10-10-10-121"},"rules":{"Unusual Linux Username":{"average_risk":21,"rule_count":2,"rule_risk":42}},"ingest_timestamp":"2021-07-27T18:02:08.319296053Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-02-23T23:00:00.000Z","risk_score":21,"host":{"name":"ip-10-10-10-53"},"rules":{"Unusual Linux Username":{"average_risk":21,"rule_count":2,"rule_risk":42}},"ingest_timestamp":"2021-07-27T18:02:08.319484548Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2020-08-21T21:00:00.000Z","risk_score":21,"host":{"name":"ip-172-31-44-235"},"rules":{"AWS Access Secret in Secrets Manager":{"average_risk":21,"rule_count":1,"rule_risk":21}},"ingest_timestamp":"2021-07-27T18:02:08.318446811Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-09T00:00:00.000Z","risk_score":21,"host":{"name":"EC2AMAZ-855H8KG"},"rules":{"Unusual Windows Path Activity":{"average_risk":21,"rule_count":1,"rule_risk":21}},"ingest_timestamp":"2021-08-09T01:02:09.209326766Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-11T19:00:00.000Z","risk_score":21,"host":{"name":"skynet-2.skynet.local"},"rules":{"Anomalous Windows Process Creation":{"average_risk":21,"rule_count":1,"rule_risk":21},"Unusual Windows Network Activity":{"average_risk":21,"rule_count":1,"rule_risk":21}},"ingest_timestamp":"2021-08-11T20:02:08.497398786Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-11T19:00:00.000Z","risk_score":21,"host":{"name":"skynet.skynet.local"},"rules":{"Anomalous Windows Process Creation":{"average_risk":21,"rule_count":2,"rule_risk":42}},"ingest_timestamp":"2021-08-11T20:02:08.497783581Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-11T19:00:00.000Z","risk_score":21,"host":{"name":"windows-native.skynet.local"},"rules":{"Anomalous Windows Process Creation":{"average_risk":21,"rule_count":11,"rule_risk":231},"Unusual Windows Network Activity":{"average_risk":21,"rule_count":3,"rule_risk":63}},"ingest_timestamp":"2021-08-11T20:02:08.497874368Z","risk":"Low"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-13T01:00:00.000Z","risk_score":47,"host":{"name":"linux-auditbeat-2"},"rules":{"Unusual Process Execution - Temp":{"average_risk":47,"rule_count":127,"rule_risk":5969}},"ingest_timestamp":"2021-08-13T02:02:08.431822027Z","risk":"Moderate"}
{"index":{"_index":"ml_host_risk_score_latest_default"}}
{"@timestamp":"2021-08-13T14:00:00.000Z","risk_score":47,"host":{"name":"linux-multibeat"},"rules":{"RDP (Remote Desktop Protocol) from the Internet":{"average_risk":47,"rule_count":4,"rule_risk":188}},"ingest_timestamp":"2021-08-13T15:02:08.536780945Z","risk":"Moderate"}

Create ml_host_risk_score index

POST _reindex
{
  "source": {
    "index": "ml_host_risk_score_latest_default"
  },
  "dest": {
    "index": "ml_host_risk_score_default"
  }
}

4. Create an index pattern in Kibana / Stack Management / Index Patterns. The name of the id pattern should be ml-host-risk-score-index-pattern, and it should point to ml_host_risk_score

5. Navigate to Overview page and observe that the Risky Hosts data warning message has changed (it is a new message specific to data missing in current time range) dashboard buttons and source links should still be disabled at this stage.

6. update time range to 1 year to see the Risky Hosts card populated with data, click on column headers and arrow for sorting

7. Upload the Host Risk Score ndjson file into Kibana using Stack Management / Saved Objects / Import

8. Observe that the View Dashboard button is now active, and clicking the button takes the user to Current Risk Score for Hosts dashboard with the correct time range

9. Observe that the individual links for hosts are now active, and clicking the link takes the user to Drilldown of Host Risk Score dashboard with the correct time range and the specific host.name applied as a filter on the page.

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[ecezalp]

@elasticmachine merge upstream

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[ecezalp]

@elasticmachine merge upstream

[ajosh0504]

Hi! Just looking at the screenshots attached in the PR Description above- Is there a way to make sure that the Current Risk is sorted in descending order by default i.e. Critical -> Unknown? Sorting by the max/sum of the risk_score field in the ml_host_risk_score_latest index should produce this result. Also, @SourinPaul, do we want to present the risk score itself, now that we have a proper normalized value or still display the risk level and keep the risk score column hidden?

[SourinPaul]

@ajosh0504 maybe this has already been addressed - but I see both scores and classifications in the card samples above. Lmk if I overlooked anything. Thanks.

[ecezalp]

@ajosh0504 maybe this has already been addressed - but I see both scores and classifications in the card samples above. Lmk if I overlooked anything. Thanks.

@ajosh0504 @SourinPaul @monina-n
Hey Sourin, the implementation matches the designs (Figma) please let me know if there are any desired updates to designs and we can coordinate with Monina and ensure that the PR meets the desired acceptance criteria.

[ecezalp]

@elasticmachine merge upstream

[ecezalp]

latest changes

• added a feature: when user clicks on the host name in the table, user is navigated to /hosts/allHosts, and a host name filter is added to the view.
• capitalized "Source" for the dashboard link
• moved isIndexNotFoundError to x-pack/plugins/security_solution/public/common/utils/exceptions/index.ts
• updated isRiskyHostHit as suggested

[ecezalp]

@elasticmachine merge upstream

[rylnd]

I reviewed this again after the "no dashboard" fixes were implemented. I now see the table populated independent of dashboards, and when dashboards are installed I observed the links populate. LGTM

[nkhristinin]

Maybe use a different name here? It's a bit confusing why it's calling LINK_COPY

[nkhristinin]

Tested locally, LGTM!

[ecezalp]

@elasticmachine merge upstream

[ecezalp]

@elasticmachine merge upstream

[ecezalp]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: ab14a4d
• Storybooks Preview
• Documentation Changes

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	2224	2243	+19

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
securitySolution	1293	1296	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	4.2MB	4.3MB	+7.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	108.4KB	108.6KB	+297.0B

Unknown metric groups

API count

id	before	after	diff
securitySolution	1347	1350	+3

References to deprecated APIs

id	before	after	diff
securitySolution	875	877	+2

History

• 💚 Build #155462 succeeded af99857
• 💔 Build #155346 failed 2967919
• 💚 Build #154979 succeeded 29284b9
• 💛 Build #154378 was flaky a9dca41
• 💔 Build #154362 failed 5c68f1f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ecezalp

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

[ecezalp]

testing instructions

1. start ES and kibana and navigate to Overview.
   i. confirm there is no Host Risk Score card.

2. Add xpack.securitySolution.enableExperimental: ['riskyHostsEnabled'] to kibana.dev.yml
   i. confirm there a Host Risk Score card with no rows.
   ii. confirm clicking the Enable Module button takes the user to the host risk score doc.

3. follow the steps in the host risk score doc to enable the module.
   i. confirm there a Host Risk Score card with rows.
   ii. confirm that you can inspect the query on the card.
   iii. confirm that clicking on the host name takes the user to the Host page with the host name applied as a filter.