[8.x] [Security Assistant] Adds new Knowledge Base Management Settings UI (#192665) #194074
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…lastic#192665) ## Summary This PR updates the Knowledge Base Management Settings page to use the new `entries` API introduced in elastic#186566. Many thanks to @angorayc for her work on the Assistant Management Settings overhaul, and initial implementation of this new KB Management UI over in elastic#186847. <p align="center"> <img width="600" src="https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa" /> </p> ### Feature Flag & Setup The changes in this PR, as with the other [recent V2 KB enhancements](elastic#186566), are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` ~They also require a code change in the `AIAssistantService` to enable the new mapping (since setup happens on plugin start before FF registration), so be sure to update `fieldMap` to `knowledgeBaseFieldMapV2` below before testing:~ This is no longer the case as of [cdec104](elastic@cdec104). Just changing the above feature flag is now sufficient, just note that if upgrading and the KB was previously setup, you'll need to manually delete the data stream (`DELETE /_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or the management table will be littered with the old ESQL docs instead of being a single aggregate entry. Once configured, the new Knowledge Base Management Settings will become available in Stack Management. The old settings UI is currently still available via the Settings Modal, but will soon be removed and replaced with links to the new interface via the Assistant Settings Context Menu (replacing the existing `cog`). Please see the designs ([Security GenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0) / [Unified AI Assistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0)) for all changes. > [!IMPORTANT] > There are no migrations in place between the legacy and v2 KB mappings, so be sure to start with a clean ES data directory. ### Testing To aid with developing the UI, I took the opportunity to start fleshing out the KB Entries API integration tests. These live in [x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries), and are currently configured to only run on `@ess`, as running `tiny_elser` in serverless and MKI environments can be tricky (more on that later). To start the server and run the tests, from the `x-pack/test/security_solution_api_integration/` directory run `yarn genai_kb_entries:server:ess`, and once started, `yarn genai_kb_entries:runner:ess`. ##### Changes in support of testing In order to setup the API integration tests for use with the Knowledge Base, some functional changes needed to be made to the assistant/config: 1. Since ELSER is a heavy model to run in CI, the ML folks have created `pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()` helper off the `ml` client that we use to get the `modelld` for installing ELSER, ingest pipelines, etc, cannot be overridden ([elastic#193633](elastic#193633)), so we must have some other means of doing that. So to get things working in the test env, I've plumbed through an optional `modelId` override to the POST knowledge base route (`/ internal/ elastic_assistant/ knowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides the aiAssistantService `getELSER()` function [when fetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354) a `kbDataClient` using the request, which appears to be the only way to also trigger a reinitialization of the ingest pipeline (which required the `modelId`), since that usually only occurs on plugin start. If there is a cleaner way to perform this reinitialization, please let me know! 2. Turns out [`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587) can't be run in test env's with `ssl:true`, which is the default security config. You can read more about that issue in [elastic#193477](elastic#193477), but the current workaround is to turn off `ssl` for this specific test configuration, so that's why [`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22) looks a little different. If there's a better way to manage this config, also please let me know! ##### Additional notes We don't currently have a `securityAssistant` API client/service to use in integration tests, so I've just been creating one-off functions using `supertest` for now. I don't have the bandwidth to work this now, but perhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I did need to test multi-user and multi-space scenarios, so I ported over the same [auth helpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth) I saw used in other suites. Would be nice if these were bundled into the client as well ala how the o11y folks have done it [here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34). Perhaps this is also on the list of things for @maximpn to generate from OAS's.... 🙃 ### RBAC In plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all the places I came across that will require an RBAC check/change. This includes some of the API integration tests, which I currently have skipped as they would fail without RBAC. ### Other notable changes * There are now dedicated `legacy` and `v2` helper functions when managing persistence/retrieval of knowledge base entries. This should help with tearing out the old KB later, and better readability now. * I've tried to remove dependency on the `ElasticsearchStore` as much as possible. The store's only use should now be within tools as a retriever [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405), and in post_evaluate [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179). If we adopt the new [`naturalLanguageToESQL`](elastic#192042) tool in `8.16` (or update our existing ESQL tool to use the `kbDataClient` for retrieval), we should be able to get rid of this entirely. * Added a [`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1) file for adding spaces, roles, users, and a sample `slackbot` index for use with [sample `IndexEntries` here](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56). ### // TODO In effort to make incremental progress and facilitate early knowledge share with @patrykkopycinski, I'm capping this PR where it's at, and so here are the remaining items to complete full integration of the new Knowledge Base Management Settings interface: - [ ] Support `Update` action - [ ] Move from `EuiInMemoryTable` - [ ] Finalize `Setup` UI - [ ] Cleanup `Save` loaders - [ ] Plumb through `{{knowledge_history}}` prompt template and include use's `required` entries All this work is behind the aforementioned feature flag and required code change, and this changeset has also been manually upgrade tested to ensure there are no issues that would impact the regularly scheduled serverless releases. This is more of a note to reviewers when testing that full functionality is not present. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit 63730ea)
8 tasks
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Unknown metric groupsAPI count
ESLint disabled line counts
References to deprecated APIs
Total ESLint disabled count
To update your PR or re-run it, just comment with: |
patrykkopycinski
approved these changes
Sep 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.x:Questions ?
Please refer to the Backport tool documentation