[Saved Queries] Rework saved query privileges #202863
Open
+9,435
−1,495
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davismcphee
added
release_note:breaking
backport:skip
kibanamachine
added a commit
that referenced
this pull request
Jan 16, 2025
…t explicit feature replacements (#206660) (#206925) # Backport This will backport the following commits from `main` to `8.x`: - [feat(security): extend `Feature` definition to support explicit feature replacements (#206660)](#206660) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Aleh Zasypkin","email":"aleh.zasypkin@elastic.co"},"sourceCommit":{"committedDate":"2025-01-16T11:35:32Z","message":"feat(security): extend `Feature` definition to support explicit feature replacements (#206660)\n\n## Summary\n\nToday, when a developer deprecates a feature and replaces its privileges\nwith those of another feature, we reasonably assume that the new feature\nfully replaces the old one in all possible contexts - whether in role\nmanagement UIs or in the Spaces feature toggles visibility UI. However,\nwhen deprecated privileges are replaced by the privileges of multiple\nfeatures, such as in [this\ncase](https://github.com/elastic/kibana/pull/202863#discussion_r1892672114)\nwhere the Discover/Dashboard/Maps feature privileges are replaced by the\nprivileges of Discover_v2/Dashboard_v2/Maps_v2, respectively, **and**\nthe privileges of the Saved Query Management feature, the choice is\nambiguous.\n\nWhich of these features should be treated as the replacement for the\ndeprecated feature in contexts that deal with entire features (like the\nSpaces feature toggles visibility UI) rather than individual privileges\n(like in role management UIs)? Should all referenced features be\nconsidered replacements? Or just a subset - or even a single feature? If\nso, which one? Currently, we treat all referenced features as\nreplacements for the deprecated feature, which creates problems, as\ndescribed in detail in [this\ndiscussion](https://github.com/elastic/kibana/pull/202863#discussion_r1892672114).\n\nThis PR allows developers to customize this behavior by specifying which\nfeatures Kibana should treat as direct successors to deprecated features\nin contexts that deal with whole features rather than individual\nprivileges:\n\n```ts\ndeps.features.registerKibanaFeature({\n deprecated: {\n notice: 'The feature is deprecated because … well, there’s a reason.',\n --> replacedBy: ['feature_id_v2'], <--\n },\n id: 'feature_id'\n name: `Case #4 feature ${suffix} (DEPRECATED)`,\n …\n});\n```\n\n## How to test\n\n1. Run test server\n```bash\nnode scripts/functional_tests_server.js --config x-pack/test/security_api_integration/features.config.ts\n```\n\n2. Execute the following request from the Dev Tools (`case_4_feature_a`\nis a deprecated feature that is replaced by multiple features and\n**doesn't use** `deprecated.replacedBy`)\n```http\nPUT kbn:/api/spaces/space/default?overwrite=true\n{\n \"id\":\"default\",\n \"name\":\"Default\",\n \"description\":\"This is your default space!\",\n \"color\":\"#00bfb3\",\n \"disabledFeatures\":[\"case_4_feature_a\"],\n \"_reserved\":true,\n \"imageUrl\":\"\",\n \"initials\":\"D\"\n}\n```\n\n3. Observe that in response deprecated `case_4_feature_a` is replaced by\ntwo features (you can also check\nhttp://localhost:5620/app/management/kibana/spaces/edit/default to see\nhow it's reflected in UI)\n```http\n{\n \"id\": \"default\",\n \"name\": \"Default\",\n \"description\": \"This is your default space!\",\n \"color\": \"#00bfb3\",\n \"initials\": \"D\",\n \"imageUrl\": \"\",\n \"disabledFeatures\": [\n \"case_4_feature_a_v2\",\n \"case_4_feature_c\"\n ],\n \"_reserved\": true\n}\n```\n\n4. Execute the following request from the Dev Tools (`case_4_feature_b`\nis a deprecated feature that is replaced by multiple features, but\n**uses** `deprecated.replacedBy` to set the conceptual\nfeature-successor)\n```http\nPUT kbn:/api/spaces/space/default?overwrite=true\n{\n \"id\":\"default\",\n \"name\":\"Default\",\n \"description\":\"This is your default space!\",\n \"color\":\"#00bfb3\",\n \"disabledFeatures\":[\"case_4_feature_b\"],\n \"_reserved\":true,\n \"imageUrl\":\"\",\n \"initials\":\"D\"\n}\n```\n\n5. Observe that in response deprecated `case_4_feature_b` is replaced by\na single feature (you can also check\nhttp://localhost:5620/app/management/kibana/spaces/edit/default to see\nhow it's reflected in UI)\n```http\n{\n \"id\": \"default\",\n \"name\": \"Default\",\n \"description\": \"This is your default space!\",\n \"color\": \"#00bfb3\",\n \"initials\": \"D\",\n \"imageUrl\": \"\",\n \"disabledFeatures\": [\n \"case_4_feature_b_v2\"\n ],\n \"_reserved\": true\n}\n```\n\n__Required by:__\nhttps://github.com//pull/202863#discussion_r1892672114\n\n//cc @davismcphee","sha":"dd3ce0e7f534279f48be8c125853c89aa92969e2","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Feature:Security/Spaces","release_note:skip","Feature:Security/Authorization","v9.0.0","backport:prev-minor"],"title":"feat(security): extend `Feature` definition to support explicit feature replacements","number":206660,"url":"https://github.com/elastic/kibana/pull/206660","mergeCommit":{"message":"feat(security): extend `Feature` definition to support explicit feature replacements (#206660)\n\n## Summary\n\nToday, when a developer deprecates a feature and replaces its privileges\nwith those of another feature, we reasonably assume that the new feature\nfully replaces the old one in all possible contexts - whether in role\nmanagement UIs or in the Spaces feature toggles visibility UI. However,\nwhen deprecated privileges are replaced by the privileges of multiple\nfeatures, such as in [this\ncase](https://github.com/elastic/kibana/pull/202863#discussion_r1892672114)\nwhere the Discover/Dashboard/Maps feature privileges are replaced by the\nprivileges of Discover_v2/Dashboard_v2/Maps_v2, respectively, **and**\nthe privileges of the Saved Query Management feature, the choice is\nambiguous.\n\nWhich of these features should be treated as the replacement for the\ndeprecated feature in contexts that deal with entire features (like the\nSpaces feature toggles visibility UI) rather than individual privileges\n(like in role management UIs)? Should all referenced features be\nconsidered replacements? Or just a subset - or even a single feature? If\nso, which one? Currently, we treat all referenced features as\nreplacements for the deprecated feature, which creates problems, as\ndescribed in detail in [this\ndiscussion](https://github.com/elastic/kibana/pull/202863#discussion_r1892672114).\n\nThis PR allows developers to customize this behavior by specifying which\nfeatures Kibana should treat as direct successors to deprecated features\nin contexts that deal with whole features rather than individual\nprivileges:\n\n```ts\ndeps.features.registerKibanaFeature({\n deprecated: {\n notice: 'The feature is deprecated because … well, there’s a reason.',\n --> replacedBy: ['feature_id_v2'], <--\n },\n id: 'feature_id'\n name: `Case #4 feature ${suffix} (DEPRECATED)`,\n …\n});\n```\n\n## How to test\n\n1. Run test server\n```bash\nnode scripts/functional_tests_server.js --config x-pack/test/security_api_integration/features.config.ts\n```\n\n2. Execute the following request from the Dev Tools (`case_4_feature_a`\nis a deprecated feature that is replaced by multiple features and\n**doesn't use** `deprecated.replacedBy`)\n```http\nPUT kbn:/api/spaces/space/default?overwrite=true\n{\n \"id\":\"default\",\n \"name\":\"Default\",\n \"description\":\"This is your default space!\",\n \"color\":\"#00bfb3\",\n \"disabledFeatures\":[\"case_4_feature_a\"],\n \"_reserved\":true,\n \"imageUrl\":\"\",\n \"initials\":\"D\"\n}\n```\n\n3. Observe that in response deprecated `case_4_feature_a` is replaced by\ntwo features (you can also check\nhttp://localhost:5620/app/management/kibana/spaces/edit/default to see\nhow it's reflected in UI)\n```http\n{\n \"id\": \"default\",\n \"name\": \"Default\",\n \"description\": \"This is your default space!\",\n \"color\": \"#00bfb3\",\n \"initials\": \"D\",\n \"imageUrl\": \"\",\n \"disabledFeatures\": [\n \"case_4_feature_a_v2\",\n \"case_4_feature_c\"\n ],\n \"_reserved\": true\n}\n```\n\n4. Execute the following request from the Dev Tools (`case_4_feature_b`\nis a deprecated feature that is replaced by multiple features, but\n**uses** `deprecated.replacedBy` to set the conceptual\nfeature-successor)\n```http\nPUT kbn:/api/spaces/space/default?overwrite=true\n{\n \"id\":\"default\",\n \"name\":\"Default\",\n \"description\":\"This is your default space!\",\n \"color\":\"#00bfb3\",\n \"disabledFeatures\":[\"case_4_feature_b\"],\n \"_reserved\":true,\n \"imageUrl\":\"\",\n \"initials\":\"D\"\n}\n```\n\n5. Observe that in response deprecated `case_4_feature_b` is replaced by\na single feature (you can also check\nhttp://localhost:5620/app/management/kibana/spaces/edit/default to see\nhow it's reflected in UI)\n```http\n{\n \"id\": \"default\",\n \"name\": \"Default\",\n \"description\": \"This is your default space!\",\n \"color\": \"#00bfb3\",\n \"initials\": \"D\",\n \"imageUrl\": \"\",\n \"disabledFeatures\": [\n \"case_4_feature_b_v2\"\n ],\n \"_reserved\": true\n}\n```\n\n__Required by:__\nhttps://github.com//pull/202863#discussion_r1892672114\n\n//cc @davismcphee","sha":"dd3ce0e7f534279f48be8c125853c89aa92969e2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/206660","number":206660,"mergeCommit":{"message":"feat(security): extend `Feature` definition to support explicit feature replacements (#206660)\n\n## Summary\n\nToday, when a developer deprecates a feature and replaces its privileges\nwith those of another feature, we reasonably assume that the new feature\nfully replaces the old one in all possible contexts - whether in role\nmanagement UIs or in the Spaces feature toggles visibility UI. However,\nwhen deprecated privileges are replaced by the privileges of multiple\nfeatures, such as in [this\ncase](https://github.com/elastic/kibana/pull/202863#discussion_r1892672114)\nwhere the Discover/Dashboard/Maps feature privileges are replaced by the\nprivileges of Discover_v2/Dashboard_v2/Maps_v2, respectively, **and**\nthe privileges of the Saved Query Management feature, the choice is\nambiguous.\n\nWhich of these features should be treated as the replacement for the\ndeprecated feature in contexts that deal with entire features (like the\nSpaces feature toggles visibility UI) rather than individual privileges\n(like in role management UIs)? Should all referenced features be\nconsidered replacements? Or just a subset - or even a single feature? If\nso, which one? Currently, we treat all referenced features as\nreplacements for the deprecated feature, which creates problems, as\ndescribed in detail in [this\ndiscussion](https://github.com/elastic/kibana/pull/202863#discussion_r1892672114).\n\nThis PR allows developers to customize this behavior by specifying which\nfeatures Kibana should treat as direct successors to deprecated features\nin contexts that deal with whole features rather than individual\nprivileges:\n\n```ts\ndeps.features.registerKibanaFeature({\n deprecated: {\n notice: 'The feature is deprecated because … well, there’s a reason.',\n --> replacedBy: ['feature_id_v2'], <--\n },\n id: 'feature_id'\n name: `Case #4 feature ${suffix} (DEPRECATED)`,\n …\n});\n```\n\n## How to test\n\n1. Run test server\n```bash\nnode scripts/functional_tests_server.js --config x-pack/test/security_api_integration/features.config.ts\n```\n\n2. Execute the following request from the Dev Tools (`case_4_feature_a`\nis a deprecated feature that is replaced by multiple features and\n**doesn't use** `deprecated.replacedBy`)\n```http\nPUT kbn:/api/spaces/space/default?overwrite=true\n{\n \"id\":\"default\",\n \"name\":\"Default\",\n \"description\":\"This is your default space!\",\n \"color\":\"#00bfb3\",\n \"disabledFeatures\":[\"case_4_feature_a\"],\n \"_reserved\":true,\n \"imageUrl\":\"\",\n \"initials\":\"D\"\n}\n```\n\n3. Observe that in response deprecated `case_4_feature_a` is replaced by\ntwo features (you can also check\nhttp://localhost:5620/app/management/kibana/spaces/edit/default to see\nhow it's reflected in UI)\n```http\n{\n \"id\": \"default\",\n \"name\": \"Default\",\n \"description\": \"This is your default space!\",\n \"color\": \"#00bfb3\",\n \"initials\": \"D\",\n \"imageUrl\": \"\",\n \"disabledFeatures\": [\n \"case_4_feature_a_v2\",\n \"case_4_feature_c\"\n ],\n \"_reserved\": true\n}\n```\n\n4. Execute the following request from the Dev Tools (`case_4_feature_b`\nis a deprecated feature that is replaced by multiple features, but\n**uses** `deprecated.replacedBy` to set the conceptual\nfeature-successor)\n```http\nPUT kbn:/api/spaces/space/default?overwrite=true\n{\n \"id\":\"default\",\n \"name\":\"Default\",\n \"description\":\"This is your default space!\",\n \"color\":\"#00bfb3\",\n \"disabledFeatures\":[\"case_4_feature_b\"],\n \"_reserved\":true,\n \"imageUrl\":\"\",\n \"initials\":\"D\"\n}\n```\n\n5. Observe that in response deprecated `case_4_feature_b` is replaced by\na single feature (you can also check\nhttp://localhost:5620/app/management/kibana/spaces/edit/default to see\nhow it's reflected in UI)\n```http\n{\n \"id\": \"default\",\n \"name\": \"Default\",\n \"description\": \"This is your default space!\",\n \"color\": \"#00bfb3\",\n \"initials\": \"D\",\n \"imageUrl\": \"\",\n \"disabledFeatures\": [\n \"case_4_feature_b_v2\"\n ],\n \"_reserved\": true\n}\n```\n\n__Required by:__\nhttps://github.com//pull/202863#discussion_r1892672114\n\n//cc @davismcphee","sha":"dd3ce0e7f534279f48be8c125853c89aa92969e2"}}]}] BACKPORT--> Co-authored-by: Aleh Zasypkin <aleh.zasypkin@elastic.co>
|
@jeramysoucy That makes sense, thanks for the explanation. I updated the suite in |
@jeramysoucy I also re-added the changes that were reverted here: 034c5db. I'll see what fails in CI tomorrow, might just be some snapshots to update. |
I did some digging on this, and discovered why the test was failing. The deprecated privileges, not being explicitly hidden, means that they will be included when querying for all registered privileges. I believe we actually want this to happen, so that we can confirm that the deprecated composite features look as expected, because they will still be used by legacy roles. So you're right, we just need to update the snapshots to include the deprecated features. I tried this out locally, and it works as expected. cc @azasypkin |
|
@jeramysoucy Thanks for looking into it and pushing the updated snapshots! Seems like that fixed the CI failures other than some unrelated Cypress ones (running CI again now). |
andrew-goldstein
approved these changes
Jan 17, 2025
adcoelho
approved these changes
Jan 18, 2025
mattkime
approved these changes
Jan 20, 2025
janmonschke
approved these changes
Jan 20, 2025
cnasikas
reviewed
Jan 20, 2025
There was a problem hiding this comment.
Shouldn't we keep the old roles in this file to ensure that the tests are working as expected with the old roles?
There was a problem hiding this comment.
Yes that makes sense, I've reverted these changes here: 547744d.
jeramysoucy
requested changes
Jan 20, 2025
There was a problem hiding this comment.
Thanks for your patience on this one! I just had one last question re: the deprecated feature overrides being updated to V2 privileges. We don't think it should have any implications in this specific case, but overrides for deprecated features should remain as they were.
config/serverless.yml
Outdated
| @@ -16,16 +16,40 @@ xpack.features.overrides: | |||
| privileges: | |||
| ### Dashboard's `All` feature privilege should implicitly grant `All` access to Maps and Visualize features. | |||
| all.composedOf: | |||
| - feature: "maps" | |||
| - feature: "maps_v2" | |||
There was a problem hiding this comment.
Is there a reason the original override was changed to reference the new V2 privileges? We'd typically expect the previous overrides to stay as they were.
There was a problem hiding this comment.
I think I may have done this when troubleshooting some YML config issues I was running into originally, but I don't think there's a good reason for it. I reverted those changes here: 3469d55.
There was a problem hiding this comment.
Looks like some snapshots failed after this, updated them here: 82727a4.
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
Unknown metric groupsAPI count
History
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR reworks saved query privileges to rely solely on a single global
savedQueryManagementprivilege, and eliminates app-specific overrides. This change simplifies the security model for users, fixes bugginess in the saved query management UI, and reduces code complexity associated with maintaining two separate security mechanisms (app-specific overrides and global saved query management privileges).Background
Saved queries allow users to store a combination of KQL or Lucene queries, filters, and time filters to use across various applications in Kibana. Access to saved query saved objects are currently granted by the following feature privileges:
There is also a saved query management UI within the Unified Search bar shared by applications across Kibana:
The way access to this UI is managed in Kibana is currently confusing and buggy:
feature_discover.allandfeature_dashboard.allthey will be able to load and save queries in Discover and Dashboard.feature_discover.allandfeature_dashboard.readthey will be able to load queries in both Discover and Dashboard, but only save queries in Discover (even though they have write access to the SO, and API access). Instead they have to navigate to Discover to save a query before navigating back to Dashboard to load it, making for a confusing and frustrating UX.Existing improvements
In v8.11.0, we introduced a new "Saved Query Management" privilege, allowing users to access saved queries across all of Kibana with a single global privilege:
When this privilege is added to a role, it solves the
feature_discover.allandfeature_dashboard.readissue mentioned above. However, it does not fix any of the mentioned issues for roles without the new privilege. We have so far postponed further improvements to avoid a breaking change.Approach
To fully resolve these issues and migrate to a single global privilege, these changes have been made:
To implement this with minimal breaking changes, we've used the Kibana privilege migration framework. This allows us to seamlessly migrate existing roles containing feature privileges that currently provide access to saved queries, ensuring they are assigned the global saved query management privilege on upgrade.
As a result, we had to deprecate the following feature privileges, replacing them with V2 privileges without saved query SO access:
Each area of code that currently relies on any of these feature privileges had to be updated to instead access
feature_X_V2instead (as well as future code).This PR still introduces a minor breaking change, since users who have
feature_discover.allandfeature_dashboard.readare now able to save queries in Dashboard after upgrade, but we believe this is a better UX (and likely the expected one) and worth a small breaking change.Testing
feature.privilegetofeature_v2.privilege, which is backward compatible.savedQueryManagementfeature should now globally control access to saved query management in Unified Search for all new user roles. Regardless of privileges for Discover, Dashboard, Maps, or Visualize, new user roles should follow this behaviour:savedQueryManagementisnone, the user cannot see or access the saved query management UI or APIs.savedQueryManagementisread, the user can load queries from the UI and access read APIs, but cannot save queries from the UI or make changes to queries through APIs.savedQueryManagementisall, the user can both load and save queries from the UI and through APIs.Checklist
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelinesIdentify risks
This PR risks introducing unintended breaking changes to user privileges related to saved queries if the deprecated features have not been properly migrated, and users could gain or lose access to saved query management on upgrade. This would be bad if it happened, but not overly severe since it wouldn't grant them access to any ES data they couldn't previously access (only query saved objects). We have automated testing in place to help ensure features have been migrated correctly, but the scope of these changes are broad and touch many places in the codebase.
Additionally, the UI capabilities types are not very strict, and are referenced with string paths in many places, which makes changing them riskier than changing strictly typed code. A combination of regex searches and temporarily modifying the
Capabilitiestype to cause type errors for deprecated privileges was used to identify references in code. Reviewers should consider if there are any other ways that UI capabilities can be referenced which were not addressed in this PR.Our automated tests already help mitigate the risk, but it's important that code owners thoroughly review the changes in their area and consider if they could have unintended consequences. The Platform Security team should also review this PR thoroughly, especially since some changes were made to platform code around privilege handling. The Data Discovery team will also manually test the behaviour when upgrading existing user roles with deprecated feature privileges as part of 9.0 upgrade testing.