Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[8.x] [SecuritySolution] Update Entity analytics BE to support servic…
…e entity type (elastic#203409) (elastic#205384) # Backport This will backport the following commits from `main` to `8.x`: - [[SecuritySolution] Update Entity analytics BE to support service entity type (elastic#203409)](elastic#203409) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Pablo Machado","email":"pablo.nevesmachado@elastic.co"},"sourceCommit":{"committedDate":"2025-01-02T12:50:08Z","message":"[SecuritySolution] Update Entity analytics BE to support service entity type (elastic#203409)\n\n## Summary\n\nUpdate Entity Analytics BE to support the new entity type \"service\".\n* Hide all functionality behind an Experimental Flag\n(`serviceEntityStoreEnabled`)\n* Update asset criticality assignment\n* Update Bulk upload logic\n* Update Risk score calculation\n* Create plugin setup mappings migration\n * Add service to risk score indices and templates\n * Add service to asset criticality index\n* Create a reusable migration workflow where we only need to update the\nmappings and bump the version\n* Add a risk score transform migration when the schedule is now called\n * It will delete and reinstall the transform to apply the changes \n\n### issues\n* I had to update the API doc to include service even though it is\nbehind an Experimental Flag\n* The risk scope mappings migration runs on every space. If the users\nhave thousands of spaces, it could take some time.\n\n### What is not included?\n* UI changes\n\n\n## Documentation for Entity Analytics future migrations\n\n### How to add a new field to the risk score index and template\nmappings?\n* Update the mapping object\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L102)\n* Pump the `mappingsVersion` version\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_engine/utils/saved_object_configuration.ts#L31)\n\n### How to add a new field to the asset criticality index?\n* Update the mapping object\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts#L22)\n* Pump the `ASSET_CRITICALITY_MAPPINGS_VERSIONS` version\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts#L20)\n\n### How to update the risk score transform config?\n* Update the transform config\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L162)\n* Pump the `version`\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L190)\n\n*note: If you change the `latest` property, the transform will reinstall\nafter the engine task runs\n\n## How to test it?\n* Enable the fla `serviceEntityStoreEnabled`\n* Start ES and an old version of Kibana\n* Populate it with data, start the risk engine\n * You could also run the document generator `yarn start entity-store` \n* Make sure you have some alerts with `service.name` field populated\n* Migrate to the version on this PR\n* Run the risk engine\n* You should see risk score documents created for service entities\n* All asset criticality API should support `service` entities\n\n## Checklist\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"1fbd86f199c738790bf53b92fd249abfbaed84f7","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team: SecuritySolution","Theme: entity_analytics","Feature:Entity Analytics","Team:Entity Analytics","backport:version","v8.18.0"],"number":203409,"url":"https://github.com/elastic/kibana/pull/203409","mergeCommit":{"message":"[SecuritySolution] Update Entity analytics BE to support service entity type (elastic#203409)\n\n## Summary\n\nUpdate Entity Analytics BE to support the new entity type \"service\".\n* Hide all functionality behind an Experimental Flag\n(`serviceEntityStoreEnabled`)\n* Update asset criticality assignment\n* Update Bulk upload logic\n* Update Risk score calculation\n* Create plugin setup mappings migration\n * Add service to risk score indices and templates\n * Add service to asset criticality index\n* Create a reusable migration workflow where we only need to update the\nmappings and bump the version\n* Add a risk score transform migration when the schedule is now called\n * It will delete and reinstall the transform to apply the changes \n\n### issues\n* I had to update the API doc to include service even though it is\nbehind an Experimental Flag\n* The risk scope mappings migration runs on every space. If the users\nhave thousands of spaces, it could take some time.\n\n### What is not included?\n* UI changes\n\n\n## Documentation for Entity Analytics future migrations\n\n### How to add a new field to the risk score index and template\nmappings?\n* Update the mapping object\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L102)\n* Pump the `mappingsVersion` version\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_engine/utils/saved_object_configuration.ts#L31)\n\n### How to add a new field to the asset criticality index?\n* Update the mapping object\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts#L22)\n* Pump the `ASSET_CRITICALITY_MAPPINGS_VERSIONS` version\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts#L20)\n\n### How to update the risk score transform config?\n* Update the transform config\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L162)\n* Pump the `version`\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L190)\n\n*note: If you change the `latest` property, the transform will reinstall\nafter the engine task runs\n\n## How to test it?\n* Enable the fla `serviceEntityStoreEnabled`\n* Start ES and an old version of Kibana\n* Populate it with data, start the risk engine\n * You could also run the document generator `yarn start entity-store` \n* Make sure you have some alerts with `service.name` field populated\n* Migrate to the version on this PR\n* Run the risk engine\n* You should see risk score documents created for service entities\n* All asset criticality API should support `service` entities\n\n## Checklist\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"1fbd86f199c738790bf53b92fd249abfbaed84f7"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/203409","number":203409,"mergeCommit":{"message":"[SecuritySolution] Update Entity analytics BE to support service entity type (elastic#203409)\n\n## Summary\n\nUpdate Entity Analytics BE to support the new entity type \"service\".\n* Hide all functionality behind an Experimental Flag\n(`serviceEntityStoreEnabled`)\n* Update asset criticality assignment\n* Update Bulk upload logic\n* Update Risk score calculation\n* Create plugin setup mappings migration\n * Add service to risk score indices and templates\n * Add service to asset criticality index\n* Create a reusable migration workflow where we only need to update the\nmappings and bump the version\n* Add a risk score transform migration when the schedule is now called\n * It will delete and reinstall the transform to apply the changes \n\n### issues\n* I had to update the API doc to include service even though it is\nbehind an Experimental Flag\n* The risk scope mappings migration runs on every space. If the users\nhave thousands of spaces, it could take some time.\n\n### What is not included?\n* UI changes\n\n\n## Documentation for Entity Analytics future migrations\n\n### How to add a new field to the risk score index and template\nmappings?\n* Update the mapping object\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L102)\n* Pump the `mappingsVersion` version\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_engine/utils/saved_object_configuration.ts#L31)\n\n### How to add a new field to the asset criticality index?\n* Update the mapping object\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts#L22)\n* Pump the `ASSET_CRITICALITY_MAPPINGS_VERSIONS` version\n[here](https://github.com/elastic/kibana/blob/8333bea86febd6dab97198d51bc3bb8f8cfa07ed/x-pack/plugins/security_solution/server/lib/entity_analytics/asset_criticality/constants.ts#L20)\n\n### How to update the risk score transform config?\n* Update the transform config\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L162)\n* Pump the `version`\n[here](https://github.com/elastic/kibana/blob/6f8b5f6c51f5926929e2ae17450b11686085a980/x-pack/plugins/security_solution/server/lib/entity_analytics/risk_score/configurations.ts#L190)\n\n*note: If you change the `latest` property, the transform will reinstall\nafter the engine task runs\n\n## How to test it?\n* Enable the fla `serviceEntityStoreEnabled`\n* Start ES and an old version of Kibana\n* Populate it with data, start the risk engine\n * You could also run the document generator `yarn start entity-store` \n* Make sure you have some alerts with `service.name` field populated\n* Migrate to the version on this PR\n* Run the risk engine\n* You should see risk score documents created for service entities\n* All asset criticality API should support `service` entities\n\n## Checklist\n- [x] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"1fbd86f199c738790bf53b92fd249abfbaed84f7"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
- Loading branch information