[FR] Add Integration Schema Query Validation #2470
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mikaayenson
requested review from
brokensound77 and
terrancedejesus
as code owners
3 tasks
…ration manifest file
Update Jan 17 2023Based on some preliminary testing, some rules will have to be tuned.
There are some other ones, but before rules are tuned, I want to locally go through more rules that fail. Note: You'll need to uncomment the lines to validate if you want to see the errors. |
Mikaayenson
commented
Jan 17, 2023
Mikaayenson
commented
Jan 17, 2023
Mikaayenson
commented
Jan 17, 2023
Mikaayenson
commented
Jan 17, 2023
Mikaayenson
commented
Jan 17, 2023
Mikaayenson
commented
Jan 17, 2023
Mikaayenson
commented
Jan 17, 2023
botelastic
bot
added
Domain: Endpoint
OS: Linux
OS: Windows
Update Feb 1
|
…ub.com:elastic/detection-rules into 1994-add-integration-specific-query-validation
terrancedejesus
approved these changes
Feb 2, 2023
There was a problem hiding this comment.
LGTM, great work!
Checks:
✅ Reviewed rule updates regarding missing system integration tag.
✅ NON_DATASET_PACKAGES in definitions.py looks good
✅ Reviewed KQLValidator and EQLValidator validation logic that relies on get_packaged_integrations and validate_integration if they apply
✅ Reviewed build_integrations_schema function in integrations.py
✅ Reviewed find_latest_compatible_version function in integrations.py
✅ Reviewed get_integration_schema_data function in integrations.py
✅ Tested show-latest-compatible command for endpoint and stack version 8.5
✅ Unit tests passing
…ub.com:elastic/detection-rules into 1994-add-integration-specific-query-validation
|
Stealing from Terrence's checklist above where applicable ^ ✅ Reviewed |
eric-forte-elastic
approved these changes
Feb 2, 2023
| integrations_schemas = load_integrations_schemas() | ||
|
|
||
| # validate the query against related integration fields | ||
| if isinstance(data, QueryRuleData) and data.language != 'lucene' and meta.maturity == "production": |
There was a problem hiding this comment.
Nit: Maybe data.language should use data.get()?
There was a problem hiding this comment.
Then again, this case (where data.language is invalid/not set) might have already been caught in schema validation.
Mikaayenson
deleted the
1994-add-integration-specific-query-validation
branch
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
Removed changes from: - rules/integrations/azure/initial_access_consent_grant_attack_via_azure_registered_application.toml - rules/linux/credential_access_bruteforce_password_guessing.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce_root.toml - rules/windows/collection_email_outlook_mailbox_via_com.toml - rules/windows/credential_access_bruteforce_admin_account.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_followed_by_success.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_same_srcip.toml - rules/windows/credential_access_dcsync_replication_rights.toml - rules/windows/credential_access_disable_kerberos_preauth.toml - rules/windows/credential_access_ldap_attributes.toml - rules/windows/credential_access_lsass_memdump_handle_access.toml - rules/windows/credential_access_remote_sam_secretsdump.toml - rules/windows/credential_access_saved_creds_vault_winlog.toml - rules/windows/credential_access_seenabledelegationprivilege_assigned_to_user.toml - rules/windows/credential_access_shadow_credentials.toml - rules/windows/credential_access_spn_attribute_modified.toml - rules/windows/credential_access_suspicious_winreg_access_via_sebackup_priv.toml - rules/windows/defense_evasion_clearing_windows_security_logs.toml - rules/windows/discovery_privileged_localgroup_membership.toml - rules/windows/discovery_whoami_command_activity.toml - rules/windows/lateral_movement_remote_service_installed_winlog.toml - rules/windows/lateral_movement_remote_task_creation_winlog.toml - rules/windows/lateral_movement_service_control_spawned_script_int.toml - rules/windows/persistence_ad_adminsdholder.toml - rules/windows/persistence_dontexpirepasswd_account.toml - rules/windows/persistence_msds_alloweddelegateto_krbtgt.toml - rules/windows/persistence_remote_password_reset.toml - rules/windows/persistence_scheduled_task_creation_winlog.toml - rules/windows/persistence_scheduled_task_updated.toml - rules/windows/persistence_sdprop_exclusion_dsheuristics.toml - rules/windows/persistence_service_windows_service_winlog.toml - rules/windows/persistence_temp_scheduled_task.toml - rules/windows/persistence_user_account_added_to_privileged_group_ad.toml - rules/windows/privilege_escalation_create_process_as_different_user.toml - rules/windows/privilege_escalation_credroaming_ldap.toml - rules/windows/privilege_escalation_group_policy_iniscript.toml - rules/windows/privilege_escalation_group_policy_privileged_groups.toml - rules/windows/privilege_escalation_group_policy_scheduled_task.toml - rules/windows/privilege_escalation_krbrelayup_service_creation.toml - rules/windows/privilege_escalation_samaccountname_spoofing_attack.toml - rules/windows/privilege_escalation_suspicious_dnshostname_update.toml - rules/windows/privilege_escalation_windows_service_via_unusual_client.toml (selectively cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
Removed changes from: - rules/integrations/azure/initial_access_consent_grant_attack_via_azure_registered_application.toml - rules/linux/credential_access_bruteforce_password_guessing.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce_root.toml - rules/windows/collection_email_outlook_mailbox_via_com.toml - rules/windows/credential_access_bruteforce_admin_account.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_followed_by_success.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_same_srcip.toml - rules/windows/credential_access_dcsync_replication_rights.toml - rules/windows/credential_access_disable_kerberos_preauth.toml - rules/windows/credential_access_ldap_attributes.toml - rules/windows/credential_access_lsass_memdump_handle_access.toml - rules/windows/credential_access_remote_sam_secretsdump.toml - rules/windows/credential_access_saved_creds_vault_winlog.toml - rules/windows/credential_access_seenabledelegationprivilege_assigned_to_user.toml - rules/windows/credential_access_shadow_credentials.toml - rules/windows/credential_access_spn_attribute_modified.toml - rules/windows/credential_access_suspicious_winreg_access_via_sebackup_priv.toml - rules/windows/defense_evasion_clearing_windows_security_logs.toml - rules/windows/discovery_privileged_localgroup_membership.toml - rules/windows/discovery_whoami_command_activity.toml - rules/windows/lateral_movement_remote_service_installed_winlog.toml - rules/windows/lateral_movement_remote_task_creation_winlog.toml - rules/windows/lateral_movement_service_control_spawned_script_int.toml - rules/windows/persistence_ad_adminsdholder.toml - rules/windows/persistence_dontexpirepasswd_account.toml - rules/windows/persistence_msds_alloweddelegateto_krbtgt.toml - rules/windows/persistence_remote_password_reset.toml - rules/windows/persistence_scheduled_task_creation_winlog.toml - rules/windows/persistence_scheduled_task_updated.toml - rules/windows/persistence_sdprop_exclusion_dsheuristics.toml - rules/windows/persistence_service_windows_service_winlog.toml - rules/windows/persistence_temp_scheduled_task.toml - rules/windows/persistence_user_account_added_to_privileged_group_ad.toml - rules/windows/privilege_escalation_create_process_as_different_user.toml - rules/windows/privilege_escalation_credroaming_ldap.toml - rules/windows/privilege_escalation_group_policy_iniscript.toml - rules/windows/privilege_escalation_group_policy_privileged_groups.toml - rules/windows/privilege_escalation_group_policy_scheduled_task.toml - rules/windows/privilege_escalation_krbrelayup_service_creation.toml - rules/windows/privilege_escalation_samaccountname_spoofing_attack.toml - rules/windows/privilege_escalation_suspicious_dnshostname_update.toml - rules/windows/privilege_escalation_windows_service_via_unusual_client.toml (selectively cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
Removed changes from: - rules/integrations/azure/initial_access_consent_grant_attack_via_azure_registered_application.toml - rules/linux/credential_access_bruteforce_password_guessing.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce_root.toml - rules/windows/collection_email_outlook_mailbox_via_com.toml - rules/windows/credential_access_bruteforce_admin_account.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_followed_by_success.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_same_srcip.toml - rules/windows/credential_access_dcsync_replication_rights.toml - rules/windows/credential_access_disable_kerberos_preauth.toml - rules/windows/credential_access_ldap_attributes.toml - rules/windows/credential_access_lsass_memdump_handle_access.toml - rules/windows/credential_access_remote_sam_secretsdump.toml - rules/windows/credential_access_saved_creds_vault_winlog.toml - rules/windows/credential_access_seenabledelegationprivilege_assigned_to_user.toml - rules/windows/credential_access_shadow_credentials.toml - rules/windows/credential_access_spn_attribute_modified.toml - rules/windows/credential_access_suspicious_winreg_access_via_sebackup_priv.toml - rules/windows/defense_evasion_clearing_windows_security_logs.toml - rules/windows/discovery_privileged_localgroup_membership.toml - rules/windows/discovery_whoami_command_activity.toml - rules/windows/lateral_movement_remote_service_installed_winlog.toml - rules/windows/lateral_movement_remote_task_creation_winlog.toml - rules/windows/lateral_movement_service_control_spawned_script_int.toml - rules/windows/persistence_ad_adminsdholder.toml - rules/windows/persistence_dontexpirepasswd_account.toml - rules/windows/persistence_msds_alloweddelegateto_krbtgt.toml - rules/windows/persistence_remote_password_reset.toml - rules/windows/persistence_scheduled_task_creation_winlog.toml - rules/windows/persistence_scheduled_task_updated.toml - rules/windows/persistence_sdprop_exclusion_dsheuristics.toml - rules/windows/persistence_service_windows_service_winlog.toml - rules/windows/persistence_temp_scheduled_task.toml - rules/windows/persistence_user_account_added_to_privileged_group_ad.toml - rules/windows/privilege_escalation_create_process_as_different_user.toml - rules/windows/privilege_escalation_credroaming_ldap.toml - rules/windows/privilege_escalation_group_policy_iniscript.toml - rules/windows/privilege_escalation_group_policy_privileged_groups.toml - rules/windows/privilege_escalation_group_policy_scheduled_task.toml - rules/windows/privilege_escalation_krbrelayup_service_creation.toml - rules/windows/privilege_escalation_samaccountname_spoofing_attack.toml - rules/windows/privilege_escalation_suspicious_dnshostname_update.toml - rules/windows/privilege_escalation_windows_service_via_unusual_client.toml (selectively cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
Removed changes from: - rules/integrations/azure/initial_access_consent_grant_attack_via_azure_registered_application.toml - rules/linux/credential_access_bruteforce_password_guessing.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce.toml - rules/linux/credential_access_potential_linux_ssh_bruteforce_root.toml - rules/windows/collection_email_outlook_mailbox_via_com.toml - rules/windows/credential_access_bruteforce_admin_account.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_followed_by_success.toml - rules/windows/credential_access_bruteforce_multiple_logon_failure_same_srcip.toml - rules/windows/credential_access_dcsync_replication_rights.toml - rules/windows/credential_access_disable_kerberos_preauth.toml - rules/windows/credential_access_ldap_attributes.toml - rules/windows/credential_access_lsass_memdump_handle_access.toml - rules/windows/credential_access_remote_sam_secretsdump.toml - rules/windows/credential_access_saved_creds_vault_winlog.toml - rules/windows/credential_access_seenabledelegationprivilege_assigned_to_user.toml - rules/windows/credential_access_shadow_credentials.toml - rules/windows/credential_access_spn_attribute_modified.toml - rules/windows/credential_access_suspicious_winreg_access_via_sebackup_priv.toml - rules/windows/defense_evasion_clearing_windows_security_logs.toml - rules/windows/discovery_privileged_localgroup_membership.toml - rules/windows/discovery_whoami_command_activity.toml - rules/windows/lateral_movement_remote_service_installed_winlog.toml - rules/windows/lateral_movement_remote_task_creation_winlog.toml - rules/windows/lateral_movement_service_control_spawned_script_int.toml - rules/windows/persistence_ad_adminsdholder.toml - rules/windows/persistence_dontexpirepasswd_account.toml - rules/windows/persistence_msds_alloweddelegateto_krbtgt.toml - rules/windows/persistence_remote_password_reset.toml - rules/windows/persistence_scheduled_task_creation_winlog.toml - rules/windows/persistence_scheduled_task_updated.toml - rules/windows/persistence_sdprop_exclusion_dsheuristics.toml - rules/windows/persistence_service_windows_service_winlog.toml - rules/windows/persistence_temp_scheduled_task.toml - rules/windows/persistence_user_account_added_to_privileged_group_ad.toml - rules/windows/privilege_escalation_create_process_as_different_user.toml - rules/windows/privilege_escalation_credroaming_ldap.toml - rules/windows/privilege_escalation_group_policy_iniscript.toml - rules/windows/privilege_escalation_group_policy_privileged_groups.toml - rules/windows/privilege_escalation_group_policy_scheduled_task.toml - rules/windows/privilege_escalation_krbrelayup_service_creation.toml - rules/windows/privilege_escalation_samaccountname_spoofing_attack.toml - rules/windows/privilege_escalation_suspicious_dnshostname_update.toml - rules/windows/privilege_escalation_windows_service_via_unusual_client.toml (selectively cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
Removed changes from: - rules/windows/collection_email_outlook_mailbox_via_com.toml (selectively cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
(cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
(cherry picked from commit 1784429)
protectionsmachine
pushed a commit
that referenced
this pull request
Feb 2, 2023
(cherry picked from commit 1784429)
This was referenced Feb 2, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issues
#1994
Summary
find_latest_compatible_versionthat scans all integration versions to determine the compatibility window to validate the rule with the integrations.show-latest-compatiblethat prints the latest compatible integration version given a package and stack version.build_integration_manifestsmethod that was introduced when theintegrationfield schema changed to support lists.IntegrationManifestSchemaschema to include thedownloadfield that is used to pull the schemas.test_integration_tagunit test to catch integrations that are missing based on rule index.Limit backports to 8.3+ #2450[FR] Recommend Latest Compatible Integration Versions #2495Manual reconciliation of the lock fileRelated Integrationslogic to install most recent compatible version kibana#149606Testing
view-ruleon several different integrations and try with multiplemin_stackversions.python -m detection_rules view-rule rules/integrations/google_workspace/collection_google_drive_ownership_transferred_via_google_workspace.tomlpython -m detection_rules view-rule rules/integrations/aws/collection_cloudtrail_logging_created.tomlpython -m detection_rules view-rule rules/integrations/azure/initial_access_consent_grant_attack_via_azure_registered_application.tomlpython -m detection_rules view-rule rules/apm/apm_403_response_to_a_post.tomlshow-latest-compatibleto verify command shows latest compatible version.python -m detection_rules dev integrations show-latest-compatible -p apm -s 8.2.0build-schemasandbuild-manifestcommands to test rebuilding the schemas and manifestpython -m detection_rules dev integrations build-schemas -opython -m detection_rules dev integrations build-manifests -omake testto ensure all unit tests pass.