[Security Solution] Skip isCustomized calculation when the feature flag is off

[xcrzx]

Resolves: #201632

Summary

When the rule customization feature flag is disabled, we should always return isCustomized: false, regardless of any changes introduced to a rule. This ensures that we do not accidentally mark prebuilt rules as customized in 8.16 with the feature flag off. For more details, refer to the related issue: #201632

Main Changes

• The primary change in this PR is encapsulated in the calculateIsCustomized function
• Other changes involve passing the feature flag to this function
• Added integration tests to cover all API CRUD operations that can be performed with rules

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[banderror]

Testing 1

Did the first round of testing locally.

I started with running kibana with the FF disabled and package set to 8.15.1:

xpack.securitySolution.enableExperimental: []
xpack.securitySolution.prebuiltRulesPackageVersion: 8.15.1

• Installed all prebuilt rules
• Edited the Endpoint Security rule individually. Added a webhook action to it.
• Bulk added an action to all prebuilt rules.
• At all times the is_customized property of all those rules was false at both the API level and in Elasticsearch in the saved objects too

Then, I kept the FF disabled and upgraded the package to 8.16.2:

xpack.securitySolution.enableExperimental: []
xpack.securitySolution.prebuiltRulesPackageVersion: 8.16.2

• There were ~680 rules that could be upgraded
• I checked what the upgrade/_review endpoint returns for them and found some rules with -AB case.
• I successfully upgraded one of such rules individually
• Then I upgraded all the rest of them in bulk
• At all times the is_customized property of all those rules remained false at both the API level and in Elasticsearch in the saved objects too

Then, I enabled the feature flag while keeping the package as 8.16.2:

xpack.securitySolution.enableExperimental: ['prebuiltRulesCustomizationEnabled']
xpack.securitySolution.prebuiltRulesPackageVersion: 8.16.2

No new issues were found under the FF turned ON.

Next, I will test how upgrade will work with the 8.16.2-beta.1 package with historical versions.

[banderror]

Testing 2

Did the second round of testing locally similar to the first one, but now upgrading from package 8.15.1 to the 8.16.2-beta.1 one that contains all historical versions.

I started with running kibana with the FF disabled and package set to 8.15.1:

xpack.securitySolution.enableExperimental: []
xpack.securitySolution.prebuiltRulesPackageVersion: 8.15.1

• Installed all prebuilt rules
• Edited the Endpoint Security rule individually. Added a webhook action to it.
• Bulk added an action to all prebuilt rules.
• At all times the is_customized property of all those rules was false at both the API level and in Elasticsearch in the saved objects too

Then, I kept the FF disabled and upgraded the package to 8.16.2-beta.1:

xpack.securitySolution.enableExperimental: []
xpack.securitySolution.prebuiltRulesPackageVersion: 8.16.2-beta.1

• There were 664 rules that could be upgraded
• I successfully upgraded a few of them individually
• Then I upgraded all the rest of them in bulk
• At all times the is_customized property of all those rules remained false at both the API level and in Elasticsearch in the saved objects too

Then, I enabled the feature flag AND bumped the package to 8.16.2:

xpack.securitySolution.enableExperimental: ['prebuiltRulesCustomizationEnabled']
xpack.securitySolution.prebuiltRulesPackageVersion: 8.16.2

• There were 64 rules that could be upgraded and 72 that could be installed
• 26 of those 64 rules had only the version field in their diffs (related slack thread (internal)):

• I upgraded 7 of those rules one-by-one (some of them directly from the table, some of them from the flyout) and got 7 customized rules in the end. Example of one of those upgrades below.

Upgrade review API response (I left only the rule that I'm upgrading and removed the rest from it):

{
  "stats": {
      "num_rules_to_upgrade_total": 59,
      "num_rules_with_conflicts": 38,
      "num_rules_with_non_solvable_conflicts": 0,
      "tags": [
          "Use Case: Identity and Access Audit",
          "Data Source: Okta",
          "Tactic: Credential Access",
          "Domain: Cloud",
          "Use Case: Threat Detection",
          "Use Case: UEBA",
          "Tactic: Persistence",
          "Data Source: Github",
          "Tactic: Initial Access",
          "Rule Type: BBR",
          "Tactic: Impact",
          "Domain: SaaS",
          "Tactic: Execution",
          "Tactic: Defense Evasion",
          "Data Source: Elastic Defend",
          "Rule Type: Higher-Order Rule",
          "Domain: Endpoint",
          "Tactic: Lateral Movement",
          "Use Case: Network Security Monitoring"
      ]
  },
  "rules": [
      {
          "id": "25bb34ea-293c-4131-b012-d56c7c4fc725",
          "rule_id": "94e734c0-2cda-11ef-84e1-f661ea17fbce",
          "revision": 2,
          "current_rule": {
              "id": "25bb34ea-293c-4131-b012-d56c7c4fc725",
              "updated_at": "2024-11-29T17:03:02.791Z",
              "updated_by": "elastic",
              "created_at": "2024-11-29T16:45:06.573Z",
              "created_by": "elastic",
              "name": "Multiple Okta User Authentication Events with Client Address",
              "tags": [
                  "Use Case: Identity and Access Audit",
                  "Data Source: Okta",
                  "Tactic: Credential Access"
              ],
              "interval": "5m",
              "enabled": false,
              "revision": 2,
              "description": "Detects when a certain threshold of Okta user authentication events are reported for multiple users from the same client address. Adversaries may attempt to launch a credential stuffing or password spraying attack from the same device by using a list of known usernames and passwords to gain unauthorized access to user accounts.",
              "risk_score": 21,
              "severity": "low",
              "note": "## Triage and analysis\n\n### Investigating Multiple Okta User Authentication Events with Client Address\n\nThis rule detects when a certain threshold of Okta user authentication events are reported for multiple users from the same client address. Adversaries may attempt to launch a credential stuffing attack from the same device by using a list of known usernames and passwords to gain unauthorized access to user accounts. Note that Okta does not log unrecognized usernames supplied during authentication attempts, so this rule may not detect all credential stuffing attempts or may indicate a targeted attack.\n\n#### Possible investigation steps:\nSince this is an ES|QL rule, the `okta.actor.alternate_id` and `okta.client.ip` values can be used to pivot into the raw authentication events related to this activity.\n- Identify the users involved in this action by examining the `okta.actor.id`, `okta.actor.type`, `okta.actor.alternate_id`, and `okta.actor.display_name` fields.\n- Determine the device client used for these actions by analyzing `okta.client.ip`, `okta.client.user_agent.raw_user_agent`, `okta.client.zone`, `okta.client.device`, and `okta.client.id` fields.\n- Review the `okta.security_context.is_proxy` field to determine if the device is a proxy.\n    - If the device is a proxy, this may indicate that a user is using a proxy to access multiple accounts for password spraying.\n- With the list of `okta.actor.alternate_id` values, review `event.outcome` results to determine if the authentication was successful.\n    - If the authentication was successful for any user, pivoting to `event.action` values for those users may provide additional context.\n- With Okta end users identified, review the `okta.debug_context.debug_data.dt_hash` field.\n    - Historical analysis should indicate if this device token hash is commonly associated with the user.\n- Review the `okta.event_type` field to determine the type of authentication event that occurred.\n    - If the event type is `user.authentication.sso`, the user may have legitimately started a session via a proxy for security or privacy reasons.\n    - If the event type is `user.authentication.password`, the user may be using a proxy to access multiple accounts for password spraying.\n    - If the event type is `user.session.start`, the source may have attempted to establish a session via the Okta authentication API.\n- Examine the `okta.outcome.result` field to determine if the authentication was successful.\n- Review the past activities of the actor(s) involved in this action by checking their previous actions.\n- Evaluate the actions that happened just before and after this event in the `okta.event_type` field to help understand the full context of the activity.\n    - This may help determine the authentication and authorization actions that occurred between the user, Okta and application.\n\n### False positive analysis:\n- A user may have legitimately started a session via a proxy for security or privacy reasons.\n- Users may share an endpoint related to work or personal use in which separate Okta accounts are used.\n    - Architecturally, this shared endpoint may leverage a proxy for security or privacy reasons.\n    - Shared systems such as Kiosks and conference room computers may be used by multiple users.\n    - Shared working spaces may have a single endpoint that is used by multiple users.\n\n### Response and remediation:\n- Review the profile of the users involved in this action to determine if proxy usage may be expected.\n- If the user is legitimate and the authentication behavior is not suspicious based on device analysis, no action is required.\n- If the user is legitimate but the authentication behavior is suspicious, consider resetting passwords for the users involves and enabling multi-factor authentication (MFA).\n    - If MFA is already enabled, consider resetting MFA for the users.\n- If any of the users are not legitimate, consider deactivating the user's account.\n- Conduct a review of Okta policies and ensure they are in accordance with security best practices.\n- Check with internal IT teams to determine if the accounts involved recently had MFA reset at the request of the user.\n    - If so, confirm with the user this was a legitimate request.\n    - If so and this was not a legitimate request, consider deactivating the user's account temporarily.\n        - Reset passwords and reset MFA for the user.\n- If this is a false positive, consider adding the `okta.debug_context.debug_data.dt_hash` field to the `exceptions` list in the rule.\n    - This will prevent future occurrences of this event for this device from triggering the rule.\n    - Alternatively adding `okta.client.ip` or a CIDR range to the `exceptions` list can prevent future occurrences of this event from triggering the rule.\n        - This should be done with caution as it may prevent legitimate alerts from being generated.\n",
              "license": "Elastic License v2",
              "output_index": "",
              "timestamp_override": "event.ingested",
              "author": [
                  "Elastic"
              ],
              "false_positives": [
                  "Users may share an endpoint related to work or personal use in which separate Okta accounts are used.",
                  "Shared systems such as Kiosks and conference room computers may be used by multiple users."
              ],
              "from": "now-9m",
              "rule_id": "94e734c0-2cda-11ef-84e1-f661ea17fbce",
              "max_signals": 100,
              "risk_score_mapping": [],
              "severity_mapping": [],
              "threat": [
                  {
                      "framework": "MITRE ATT&CK",
                      "tactic": {
                          "id": "TA0006",
                          "name": "Credential Access",
                          "reference": "https://attack.mitre.org/tactics/TA0006/"
                      },
                      "technique": [
                          {
                              "id": "T1110",
                              "name": "Brute Force",
                              "reference": "https://attack.mitre.org/techniques/T1110/",
                              "subtechnique": [
                                  {
                                      "id": "T1110.003",
                                      "name": "Password Spraying",
                                      "reference": "https://attack.mitre.org/techniques/T1110/003/"
                                  }
                              ]
                          },
                          {
                              "id": "T1110",
                              "name": "Brute Force",
                              "reference": "https://attack.mitre.org/techniques/T1110/",
                              "subtechnique": [
                                  {
                                      "id": "T1110.004",
                                      "name": "Credential Stuffing",
                                      "reference": "https://attack.mitre.org/techniques/T1110/004/"
                                  }
                              ]
                          }
                      ]
                  }
              ],
              "to": "now",
              "references": [
                  "https://support.okta.com/help/s/article/How-does-the-Device-Token-work?language=en_US",
                  "https://developer.okta.com/docs/reference/api/event-types/",
                  "https://www.elastic.co/security-labs/testing-okta-visibility-and-detection-dorothy",
                  "https://sec.okta.com/articles/2023/08/cross-tenant-impersonation-prevention-and-detection",
                  "https://www.okta.com/resources/whitepaper-how-adaptive-mfa-can-help-in-mitigating-brute-force-attacks/",
                  "https://www.elastic.co/security-labs/monitoring-okta-threats-with-elastic-security",
                  "https://www.elastic.co/security-labs/starter-guide-to-understanding-okta"
              ],
              "version": 3,
              "exceptions_list": [],
              "immutable": true,
              "rule_source": {
                  "type": "external",
                  "is_customized": false
              },
              "related_integrations": [],
              "required_fields": [],
              "setup": "The Okta Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.",
              "type": "esql",
              "language": "esql",
              "query": "FROM logs-okta*\n| WHERE\n    event.dataset == \"okta.system\"\n    AND (event.action == \"user.session.start\" OR event.action RLIKE \"user\\\\.authentication(.*)\")\n    AND okta.outcome.reason == \"INVALID_CREDENTIALS\"\n| KEEP okta.client.ip, okta.actor.alternate_id, okta.actor.id, event.action, okta.outcome.reason\n| STATS\n    source_auth_count = COUNT_DISTINCT(okta.actor.id)\n    BY okta.client.ip, okta.actor.alternate_id\n| WHERE\n    source_auth_count > 5\n| SORT\n    source_auth_count DESC\n",
              "actions": [
                  {
                      "id": "e7c8dc8e-230d-4b56-9987-848978c32705",
                      "params": {
                          "body": "{\n    \"hello\": \"world\"\n}"
                      },
                      "action_type_id": ".webhook",
                      "uuid": "2fff8e43-c870-4016-870f-dcebb0b7067e",
                      "frequency": {
                          "summary": true,
                          "notifyWhen": "onActiveAlert",
                          "throttle": null
                      },
                      "group": "default"
                  }
              ]
          },
          "target_rule": {
              "name": "Multiple Okta User Authentication Events with Client Address",
              "description": "Detects when a certain threshold of Okta user authentication events are reported for multiple users from the same client address. Adversaries may attempt to launch a credential stuffing or password spraying attack from the same device by using a list of known usernames and passwords to gain unauthorized access to user accounts.",
              "risk_score": 21,
              "severity": "low",
              "timestamp_override": "event.ingested",
              "license": "Elastic License v2",
              "note": "## Triage and analysis\n\n### Investigating Multiple Okta User Authentication Events with Client Address\n\nThis rule detects when a certain threshold of Okta user authentication events are reported for multiple users from the same client address. Adversaries may attempt to launch a credential stuffing attack from the same device by using a list of known usernames and passwords to gain unauthorized access to user accounts. Note that Okta does not log unrecognized usernames supplied during authentication attempts, so this rule may not detect all credential stuffing attempts or may indicate a targeted attack.\n\n#### Possible investigation steps:\nSince this is an ES|QL rule, the `okta.actor.alternate_id` and `okta.client.ip` values can be used to pivot into the raw authentication events related to this activity.\n- Identify the users involved in this action by examining the `okta.actor.id`, `okta.actor.type`, `okta.actor.alternate_id`, and `okta.actor.display_name` fields.\n- Determine the device client used for these actions by analyzing `okta.client.ip`, `okta.client.user_agent.raw_user_agent`, `okta.client.zone`, `okta.client.device`, and `okta.client.id` fields.\n- Review the `okta.security_context.is_proxy` field to determine if the device is a proxy.\n    - If the device is a proxy, this may indicate that a user is using a proxy to access multiple accounts for password spraying.\n- With the list of `okta.actor.alternate_id` values, review `event.outcome` results to determine if the authentication was successful.\n    - If the authentication was successful for any user, pivoting to `event.action` values for those users may provide additional context.\n- With Okta end users identified, review the `okta.debug_context.debug_data.dt_hash` field.\n    - Historical analysis should indicate if this device token hash is commonly associated with the user.\n- Review the `okta.event_type` field to determine the type of authentication event that occurred.\n    - If the event type is `user.authentication.sso`, the user may have legitimately started a session via a proxy for security or privacy reasons.\n    - If the event type is `user.authentication.password`, the user may be using a proxy to access multiple accounts for password spraying.\n    - If the event type is `user.session.start`, the source may have attempted to establish a session via the Okta authentication API.\n- Examine the `okta.outcome.result` field to determine if the authentication was successful.\n- Review the past activities of the actor(s) involved in this action by checking their previous actions.\n- Evaluate the actions that happened just before and after this event in the `okta.event_type` field to help understand the full context of the activity.\n    - This may help determine the authentication and authorization actions that occurred between the user, Okta and application.\n\n### False positive analysis:\n- A user may have legitimately started a session via a proxy for security or privacy reasons.\n- Users may share an endpoint related to work or personal use in which separate Okta accounts are used.\n    - Architecturally, this shared endpoint may leverage a proxy for security or privacy reasons.\n    - Shared systems such as Kiosks and conference room computers may be used by multiple users.\n    - Shared working spaces may have a single endpoint that is used by multiple users.\n\n### Response and remediation:\n- Review the profile of the users involved in this action to determine if proxy usage may be expected.\n- If the user is legitimate and the authentication behavior is not suspicious based on device analysis, no action is required.\n- If the user is legitimate but the authentication behavior is suspicious, consider resetting passwords for the users involves and enabling multi-factor authentication (MFA).\n    - If MFA is already enabled, consider resetting MFA for the users.\n- If any of the users are not legitimate, consider deactivating the user's account.\n- Conduct a review of Okta policies and ensure they are in accordance with security best practices.\n- Check with internal IT teams to determine if the accounts involved recently had MFA reset at the request of the user.\n    - If so, confirm with the user this was a legitimate request.\n    - If so and this was not a legitimate request, consider deactivating the user's account temporarily.\n        - Reset passwords and reset MFA for the user.\n- If this is a false positive, consider adding the `okta.debug_context.debug_data.dt_hash` field to the `exceptions` list in the rule.\n    - This will prevent future occurrences of this event for this device from triggering the rule.\n    - Alternatively adding `okta.client.ip` or a CIDR range to the `exceptions` list can prevent future occurrences of this event from triggering the rule.\n        - This should be done with caution as it may prevent legitimate alerts from being generated.\n",
              "output_index": "",
              "version": 103,
              "tags": [
                  "Use Case: Identity and Access Audit",
                  "Data Source: Okta",
                  "Tactic: Credential Access"
              ],
              "enabled": false,
              "risk_score_mapping": [],
              "severity_mapping": [],
              "interval": "5m",
              "from": "now-9m",
              "to": "now",
              "actions": [],
              "exceptions_list": [],
              "author": [
                  "Elastic"
              ],
              "false_positives": [
                  "Users may share an endpoint related to work or personal use in which separate Okta accounts are used.",
                  "Shared systems such as Kiosks and conference room computers may be used by multiple users."
              ],
              "references": [
                  "https://support.okta.com/help/s/article/How-does-the-Device-Token-work?language=en_US",
                  "https://developer.okta.com/docs/reference/api/event-types/",
                  "https://www.elastic.co/security-labs/testing-okta-visibility-and-detection-dorothy",
                  "https://sec.okta.com/articles/2023/08/cross-tenant-impersonation-prevention-and-detection",
                  "https://www.okta.com/resources/whitepaper-how-adaptive-mfa-can-help-in-mitigating-brute-force-attacks/",
                  "https://www.elastic.co/security-labs/monitoring-okta-threats-with-elastic-security",
                  "https://www.elastic.co/security-labs/starter-guide-to-understanding-okta"
              ],
              "max_signals": 100,
              "threat": [
                  {
                      "framework": "MITRE ATT&CK",
                      "tactic": {
                          "id": "TA0006",
                          "name": "Credential Access",
                          "reference": "https://attack.mitre.org/tactics/TA0006/"
                      },
                      "technique": [
                          {
                              "id": "T1110",
                              "name": "Brute Force",
                              "reference": "https://attack.mitre.org/techniques/T1110/",
                              "subtechnique": [
                                  {
                                      "id": "T1110.003",
                                      "name": "Password Spraying",
                                      "reference": "https://attack.mitre.org/techniques/T1110/003/"
                                  }
                              ]
                          },
                          {
                              "id": "T1110",
                              "name": "Brute Force",
                              "reference": "https://attack.mitre.org/techniques/T1110/",
                              "subtechnique": [
                                  {
                                      "id": "T1110.004",
                                      "name": "Credential Stuffing",
                                      "reference": "https://attack.mitre.org/techniques/T1110/004/"
                                  }
                              ]
                          }
                      ]
                  }
              ],
              "setup": "The Okta Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.",
              "related_integrations": [],
              "required_fields": [],
              "id": "25bb34ea-293c-4131-b012-d56c7c4fc725",
              "rule_id": "94e734c0-2cda-11ef-84e1-f661ea17fbce",
              "immutable": true,
              "rule_source": {
                  "type": "external",
                  "is_customized": false
              },
              "updated_at": "2024-11-29T17:37:11.757Z",
              "updated_by": "elastic",
              "created_at": "2024-11-29T16:45:06.573Z",
              "created_by": "elastic",
              "revision": 3,
              "type": "esql",
              "language": "esql",
              "query": "FROM logs-okta*\n| WHERE\n    event.dataset == \"okta.system\"\n    AND (event.action == \"user.session.start\" OR event.action RLIKE \"user\\\\.authentication(.*)\")\n    AND okta.outcome.reason == \"INVALID_CREDENTIALS\"\n| KEEP okta.client.ip, okta.actor.alternate_id, okta.actor.id, event.action, okta.outcome.reason\n| STATS\n    source_auth_count = COUNT_DISTINCT(okta.actor.id)\n    BY okta.client.ip, okta.actor.alternate_id\n| WHERE\n    source_auth_count > 5\n| SORT\n    source_auth_count DESC\n"
          },
          "diff": {
              "fields": {
                  "version": {
                      "has_base_version": true,
                      "base_version": 3,
                      "current_version": 3,
                      "target_version": 103,
                      "merged_version": 103,
                      "merge_outcome": "TARGET",
                      "diff_outcome": "BASE=A, CURRENT=A, TARGET=B",
                      "has_update": true,
                      "conflict": "NONE"
                  }
              },
              "num_fields_with_updates": 1,
              "num_fields_with_conflicts": 0,
              "num_fields_with_non_solvable_conflicts": 0
          }
      }
  ]
}

Upgrade perform API request and response:

{"mode":"SPECIFIC_RULES","rules":[{"rule_id":"94e734c0-2cda-11ef-84e1-f661ea17fbce","version":103,"revision":2,"fields":{}}],"pick_version":"MERGED"}

{
    "summary": {
        "total": 1,
        "skipped": 0,
        "succeeded": 1,
        "failed": 0
    },
    "results": {
        "updated": [
            {
                "name": "Multiple Okta User Authentication Events with Client Address",
                "description": "Detects when a certain threshold of Okta user authentication events are reported for multiple users from the same client address. Adversaries may attempt to launch a credential stuffing or password spraying attack from the same device by using a list of known usernames and passwords to gain unauthorized access to user accounts.",
                "risk_score": 21,
                "severity": "low",
                "timestamp_override": "event.ingested",
                "timestamp_override_fallback_disabled": false,
                "license": "Elastic License v2",
                "note": "## Triage and analysis\n\n### Investigating Multiple Okta User Authentication Events with Client Address\n\nThis rule detects when a certain threshold of Okta user authentication events are reported for multiple users from the same client address. Adversaries may attempt to launch a credential stuffing attack from the same device by using a list of known usernames and passwords to gain unauthorized access to user accounts. Note that Okta does not log unrecognized usernames supplied during authentication attempts, so this rule may not detect all credential stuffing attempts or may indicate a targeted attack.\n\n#### Possible investigation steps:\nSince this is an ES|QL rule, the `okta.actor.alternate_id` and `okta.client.ip` values can be used to pivot into the raw authentication events related to this activity.\n- Identify the users involved in this action by examining the `okta.actor.id`, `okta.actor.type`, `okta.actor.alternate_id`, and `okta.actor.display_name` fields.\n- Determine the device client used for these actions by analyzing `okta.client.ip`, `okta.client.user_agent.raw_user_agent`, `okta.client.zone`, `okta.client.device`, and `okta.client.id` fields.\n- Review the `okta.security_context.is_proxy` field to determine if the device is a proxy.\n    - If the device is a proxy, this may indicate that a user is using a proxy to access multiple accounts for password spraying.\n- With the list of `okta.actor.alternate_id` values, review `event.outcome` results to determine if the authentication was successful.\n    - If the authentication was successful for any user, pivoting to `event.action` values for those users may provide additional context.\n- With Okta end users identified, review the `okta.debug_context.debug_data.dt_hash` field.\n    - Historical analysis should indicate if this device token hash is commonly associated with the user.\n- Review the `okta.event_type` field to determine the type of authentication event that occurred.\n    - If the event type is `user.authentication.sso`, the user may have legitimately started a session via a proxy for security or privacy reasons.\n    - If the event type is `user.authentication.password`, the user may be using a proxy to access multiple accounts for password spraying.\n    - If the event type is `user.session.start`, the source may have attempted to establish a session via the Okta authentication API.\n- Examine the `okta.outcome.result` field to determine if the authentication was successful.\n- Review the past activities of the actor(s) involved in this action by checking their previous actions.\n- Evaluate the actions that happened just before and after this event in the `okta.event_type` field to help understand the full context of the activity.\n    - This may help determine the authentication and authorization actions that occurred between the user, Okta and application.\n\n### False positive analysis:\n- A user may have legitimately started a session via a proxy for security or privacy reasons.\n- Users may share an endpoint related to work or personal use in which separate Okta accounts are used.\n    - Architecturally, this shared endpoint may leverage a proxy for security or privacy reasons.\n    - Shared systems such as Kiosks and conference room computers may be used by multiple users.\n    - Shared working spaces may have a single endpoint that is used by multiple users.\n\n### Response and remediation:\n- Review the profile of the users involved in this action to determine if proxy usage may be expected.\n- If the user is legitimate and the authentication behavior is not suspicious based on device analysis, no action is required.\n- If the user is legitimate but the authentication behavior is suspicious, consider resetting passwords for the users involves and enabling multi-factor authentication (MFA).\n    - If MFA is already enabled, consider resetting MFA for the users.\n- If any of the users are not legitimate, consider deactivating the user's account.\n- Conduct a review of Okta policies and ensure they are in accordance with security best practices.\n- Check with internal IT teams to determine if the accounts involved recently had MFA reset at the request of the user.\n    - If so, confirm with the user this was a legitimate request.\n    - If so and this was not a legitimate request, consider deactivating the user's account temporarily.\n        - Reset passwords and reset MFA for the user.\n- If this is a false positive, consider adding the `okta.debug_context.debug_data.dt_hash` field to the `exceptions` list in the rule.\n    - This will prevent future occurrences of this event for this device from triggering the rule.\n    - Alternatively adding `okta.client.ip` or a CIDR range to the `exceptions` list can prevent future occurrences of this event from triggering the rule.\n        - This should be done with caution as it may prevent legitimate alerts from being generated.\n",
                "output_index": "",
                "version": 103,
                "tags": [
                    "Use Case: Identity and Access Audit",
                    "Data Source: Okta",
                    "Tactic: Credential Access"
                ],
                "enabled": false,
                "risk_score_mapping": [],
                "severity_mapping": [],
                "interval": "5m",
                "from": "now-240s",
                "to": "now",
                "actions": [
                    {
                        "action_type_id": ".webhook",
                        "group": "default",
                        "id": "e7c8dc8e-230d-4b56-9987-848978c32705",
                        "params": {
                            "body": "{\n    \"hello\": \"world\"\n}"
                        },
                        "uuid": "2fff8e43-c870-4016-870f-dcebb0b7067e",
                        "frequency": {
                            "summary": true,
                            "notifyWhen": "onActiveAlert",
                            "throttle": null
                        }
                    }
                ],
                "exceptions_list": [],
                "author": [
                    "Elastic"
                ],
                "false_positives": [
                    "Users may share an endpoint related to work or personal use in which separate Okta accounts are used.",
                    "Shared systems such as Kiosks and conference room computers may be used by multiple users."
                ],
                "references": [
                    "https://support.okta.com/help/s/article/How-does-the-Device-Token-work?language=en_US",
                    "https://developer.okta.com/docs/reference/api/event-types/",
                    "https://www.elastic.co/security-labs/testing-okta-visibility-and-detection-dorothy",
                    "https://sec.okta.com/articles/2023/08/cross-tenant-impersonation-prevention-and-detection",
                    "https://www.okta.com/resources/whitepaper-how-adaptive-mfa-can-help-in-mitigating-brute-force-attacks/",
                    "https://www.elastic.co/security-labs/monitoring-okta-threats-with-elastic-security",
                    "https://www.elastic.co/security-labs/starter-guide-to-understanding-okta"
                ],
                "max_signals": 100,
                "threat": [
                    {
                        "framework": "MITRE ATT&CK",
                        "tactic": {
                            "id": "TA0006",
                            "name": "Credential Access",
                            "reference": "https://attack.mitre.org/tactics/TA0006/"
                        },
                        "technique": [
                            {
                                "id": "T1110",
                                "name": "Brute Force",
                                "reference": "https://attack.mitre.org/techniques/T1110/",
                                "subtechnique": [
                                    {
                                        "id": "T1110.003",
                                        "name": "Password Spraying",
                                        "reference": "https://attack.mitre.org/techniques/T1110/003/"
                                    }
                                ]
                            },
                            {
                                "id": "T1110",
                                "name": "Brute Force",
                                "reference": "https://attack.mitre.org/techniques/T1110/",
                                "subtechnique": [
                                    {
                                        "id": "T1110.004",
                                        "name": "Credential Stuffing",
                                        "reference": "https://attack.mitre.org/techniques/T1110/004/"
                                    }
                                ]
                            }
                        ]
                    }
                ],
                "setup": "The Okta Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.",
                "related_integrations": [],
                "required_fields": [],
                "id": "25bb34ea-293c-4131-b012-d56c7c4fc725",
                "rule_id": "94e734c0-2cda-11ef-84e1-f661ea17fbce",
                "immutable": true,
                "rule_source": {
                    "type": "external",
                    "is_customized": true
                },
                "updated_at": "2024-11-29T17:39:54.103Z",
                "updated_by": "elastic",
                "created_at": "2024-11-29T16:45:06.573Z",
                "created_by": "elastic",
                "revision": 3,
                "type": "esql",
                "language": "esql",
                "query": "FROM logs-okta*\n| WHERE\n    event.dataset == \"okta.system\"\n    AND (event.action == \"user.session.start\" OR event.action RLIKE \"user\\\\.authentication(.*)\")\n    AND okta.outcome.reason == \"INVALID_CREDENTIALS\"\n| KEEP okta.client.ip, okta.actor.alternate_id, okta.actor.id, event.action, okta.outcome.reason\n| STATS\n    source_auth_count = COUNT_DISTINCT(okta.actor.id)\n    BY okta.client.ip, okta.actor.alternate_id\n| WHERE\n    source_auth_count > 5\n| SORT\n    source_auth_count DESC\n"
            }
        ],
        "skipped": []
    },
    "errors": []
}

• Then I clicked Update all in the table and it upgraded 19 of the 57 rules to upgrade, because the rest 38 rules had SOLVABLE conflicts in the related_integrations field. All those 19 rules were marked as customized. I think this is the same issue that the one above.

• All the rest 38 rules, I was able to upgrade them via the ThreeWayDiff UI by editing and accepting changes, but all of them were marked as customized.

[banderror]

@xcrzx I did two rounds of local testing (please review the results above):

• With the FF turned OFF everything works fine.
• With the FF turned ON in certain conditions:
  • Rules are being marked as customized on upgrade. It would be great to try to fix it in this PR.

I'm blocking the PR temporarily so that hopefully we can fix this issue before merge. If we're too close to the 8.16.2 FF, I'd be ok to merge it as is and fix the issue later.

Meanwhile, I'll review the code changes.

[banderror]

Code changes are relatively straightforward and LGTM 👍

Thank you for splitting the integration tests into two folders: customization_enabled and customization_disabled, and adding the corresponding coverage for when it is disabled. I reviewed the tests and they LGTM too.

I'm leaving minor comments and approving beforehand, so please address at will.

As for the testing done previously and blocking the PR, I discussed this with @xcrzx and the reason why it's broken under the FF enabled is not related to this fix. There's a chance it's the same issue as #201631, or a different one. Also:

The PR was never intended to address issues with rules marked as customized when the feature flag is enabled. Its purpose is to disable the isCustomized calculation entirely when the feature flag is disabled, preventing further issues in 8.16 and 8.17.

and I agree with that. I'll create a ticket for the bug I had discovered during testing. UPD: #202575

Thank you @xcrzx and let's 🚀

[kibanamachine]

Starting backport for target branches: 8.16, 8.17, 8.x

https://github.com/elastic/kibana/actions/runs/12139367321

[kibanamachine]

💔 Some backports could not be created

Status	Branch	Result
❌	8.16	Backport failed because of merge conflicts
✅	8.17
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 201825

Questions ?

Please refer to the Backport tool documentation

[xcrzx]

💚 All backports created successfully

Status	Branch	Result
✅	8.16

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation