[Security Solution][Detection Engine] deprecates signals migration APIs

[vitaliidm]

Summary

• addresses [Detection Engine][9.0] Deprecate legacy siem signals migration endpoints #195523
• deprecates https://www.elastic.co/guide/en/security/current/signals-migration-api.html APIs according to internal 9.x readiness guideline
• adds deprecation feature in Kibana Upgrade Assistant. When non-migrated siem signals detected, it will be shown at a critical level
• adds snapshot telemetry
• while working on PR, discovered and fixed few issues in APIs
  • get migration status API did not work correctly with new .alerts-* indices, listing them as outdated
  • finalize migration API did account for spaces, when adding alias to migrated index
  • remove migration API failed due to lack of permissions to removed migration task from .tasks index

How to test

How to create legacy siem index?

run script that used for FTR tests

node scripts/es_archiver --kibana-url=http://elastic:changeme@localhost:5601 --es-url=http://elastic:changeme@localhost:9200 load x-pack/test/functional/es_archives/signals/legacy_signals_index

node scripts/es_archiver --kibana-url=http://elastic:changeme@localhost:5601 --es-url=http://elastic:changeme@localhost:9200 load x-pack/test/functional/es_archives/signals/legacy_signals_index_another_1

These would create legacy siem indices. But be aware, it might break Kibana .alerts indices creation. But sufficient for testing

How to test deprecated APIs?

1. Run API https://www.elastic.co/guide/en/security/current/signals-migration-api.html
2. Observe warning deprecation on Kibana Upgrade page

Kibana Upgrade list of deprecations

How to test deprecated feature?

1. Observe critical feature deprecation on Kibana Upgrade page, if you set up legacy siem signals

Kibana Upgrade feature deprecation flyout

How to test snapshot telemetry

Snapshot
For snapshot telemetry use API call
OR
Check snapshots in Kibana adv settings -> Global Settings Tab -> Usage collection section -> Click on cluster data example link -> Check legacy_siem_signals fields in flyout

Snapshot telemetry

[elasticmachine]

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

• Click to trigger kibana-pull-request for this PR!
• Click to trigger kibana-deploy-project-from-pr for this PR!

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 1fe0ffe

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/securitysolution-es-utils	76	77	+1

Unknown metric groups

API count

id	before	after	diff
@kbn/securitysolution-es-utils	87	88	+1

ESLint disabled line counts

id	before	after	diff
securitySolution	558	562	+4

Total ESLint disabled count

id	before	after	diff
securitySolution	642	646	+4

History

• 💚 Build #255325 succeeded 308bc50
• 💔 Build #255039 failed 002cf96
• 💔 Build #253310 failed aac7319
• 💔 Build #252862 failed 59659ef
• 💔 Build #251829 failed 70c68c2
• 💔 Build #251494 failed c61e0ed

cc @vitaliidm