-
Notifications
You must be signed in to change notification settings - Fork 275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] Dashboards object sharing (replace Dashboards tenants) #1869
Comments
We love the concepts of tenants, however I am slightly worried about this proposal to replace them on my understanding of what I've read. There's a lot written above so I may be misinterpreting some of it - if this is the case please help me understand better the proposal! We use tenants to organise data from various products into verticals where each tenant (product) is compartmentalised and separate from each other one. This means that we can set up visualisations, dashboards and index patterns for one product are completely separate from another. This seems to be use-case 2 raised in the proposal above: "to create a clear data separation of indices and visuals on the data in such indices" The shortcomings listed above state that the "Complete Dashboards experience is anchored around the current tenant that the user has selected instead of being anchored around the user and their permissions", however this is exactly how we use tenants and do not wish to move away from this model. If a user has access to all tenants we still want to be able to show only data for the current product as conceptually they are completely separate from other products. I don't want to see dashboards and visualisations for Product 2 when I am looking at Product 1.
This sounds good
A good idea, but only if not the default. I'd like to see and focus on the data I am interested in for the product I am looking at without having to search through hundreds of other visualisations.
Not sure I understand, these seem to contradict each other. Would new user created objects shared or private by default? If new user objects are shared with users of the same role then how will this work when a user has multiple roles?
How will index patterns be assigned to users when using an external auth eg. LDAP? We don't want to have to define index patterns for hundreds of users individually. Our users don't understand index patterns, nor do they wish to know about index patterns. Most just want to view a dashboard and see pretty graphs.
Sounds good, as long as defaults can be set.
Sounds like a notification system would also be required here to notify admins that they have pending requests
As long as there is the ability to select a specific role and view and explore data through the lens of that role this seems fine. All in all, I feel it may just become for more difficult to administrate for more than a handful of users. I also feel that it will raise the barrier for entry with users having to understand more about permissions and therefore make the software harder to use. Don't fall into the same trap that Elastic seems to have fallen into in wanting every end user to understand how Kibana/Dashboards works. Thanks for putting together the proposal. I'm really glad of this discussion but I'm not sure I quite see how the proposal as it stands can satisfy use-case 2 raised above: "to create a clear data separation of indices and visuals on the data in such indices". Would love to hear feedback on the above. |
Will have to keep this brief, so sorry for the stream of consciousness response... As for the notion of "tenancies" being misinterpreted as providing data separation, fair enough - but we use it in the manner suggested, to provide compartmentalization of dashboards and visualizations. Like @jgough above, we don't wish to lose that aspect of the tenancy nomenclature. However, having a means to place certain indices in specific storage spaces in order to provide data separation would be desirable. But that seems beyond the scope of your proposal. One of my frustrations is in changing visualizations and dashboards where the index pattern name has changed, resulting in a different UUID, which requires a fairly manual process to update the underlying visualizations - I can't say how the proposed idea of tying index patterns to the user may help with this - I'll just note that changing an index pattern (uuid) shouldn't invalidate all of the dashboards and visualizations that were based on the previous iteration. We use doc-level security to provide fine-grained access control, and would be nice if the security plugin permitted a "self-service" role request mechanism - that is, a user can request a role, and the admin role can approve it. I like this. The way we use Opensearch is to drop it into a client's site, and more often than not, the client doesn't provide us access to their SSO service, so we use the security plugin. So, like self-service role requests, it would be nice to have a self-service sign-up so users can enter their own account info (subject to admin approval) - or at least, allow a "change password at next login" and password aging/complexity rules - it's kind of awkward to generate a password for a user account, find a way to transmit it to them securely, but with no assurance they'll ever change it, or will change it to something dumb, like "password". As noted above, some kind of notification system that there are outstanding requests would be necessary (that is, some kind of external notification system - i.e. slack or email.) Tagging - The notion of "tagging" sounds like something I've been thinking about, but am unsure how it would be implemented in practice - seems to me the current design of dashboards / visualizations are great at providing filtering of events - but they're read-only, and don't provide a convenient mechanism for updating fields, for example:
Using the console or query workbench isn't really appropriate for this type of operation. Thanks for the detailed proposal, there are some interesting ideas in there. At the same time, I'll have to echo @jgough's concerns above - the goals are great, but if it raises the administrative complexity and overhead for simple use cases, it could be counter-productive in driving adoption. |
@squiddy-gh and @jgough - I'm hearing the need for some kind of either workspace, or other organizational concept that would allow users to easily share objects together without having to look through and parse through a bunch of additional "noise" to get to what they are looking for. This makes me wonder if either a hierarchichal organization structure could be added as an option (perhaps folders that users can save objects into, so that all objects in the same folder can have shared permissions, visibility, sharing), or even the notion of a workspace. My concern with workspaces, is that this makes sharing objects between spaces more challenging, and it tends to give people the incorrect assumption that workspaces provide data isolation (true multitenancy), when in reality it usually results simply in dashboard objects isolation. As we continue to evolve the concept of tenancy, I want to ensure we don't throw out useful concepts, so I really appreciate the additional perspective here. |
Are there many forum posts around this issue and the problems people are having? I had a quick search but couldn't find anything. I'd really like to understand if I'm an outlier in liking the existing tenant system. |
This is an interesting topic. I share the opinion that the current multi-tenant mechanism leaves a lot to be desired. So, I strongly appreciate your initiative. Still, the topic has lots of different facets, dimensions and external constraints, which makes it hard to find a solution which suits all needs. First of all, it would be nice if the feature would be designed in such a way that 3rd party security plugins like Search Guard would remain inter-operable with Dashboards. Now to the actual proposal. For now, I just have a random list of observations and thoughts. I think there are a couple of issues/challenges. But life would be boring without challenges, wouldn't it? :-) You are writing in the "Shortcomings" section:
IMHO, the m:n relationship between roles and tenants is desirable. This is a standard behavior that also holds for indices and other permissions: Roles can give access to many indices and one index can be accessed by many indices. Only this gives administrators great flexibility in modeling their organizational structure (which can be amazingly complex). A very simple example for roles which leverage the m:n relationship are roles like This m:n relationship between roles and granted privileges can be found in most software secured by role-based access control. Thus, I would not consider it too hard to understand to the experienced administrator. In a similar context:
I would not recommend to let users share objects to roles or backend roles. Right now, roles and backend roles are a concept that is totally invisible to users. Roles and backend roles are only visible to administrators. Thus, the names of roles are designed to suit the needs of administrators, not the needs of users. Also, roles are not necessarily modeled after organizational structures. There are the pre-defined standard roles Going further in the "Shortcomings" section:
And in the "Proposed solution" section:
The change of index pattern association should be not necessary if OpenSearch security is running with the enabled Consider the following example: Cluster has indices:
User is only allowed to access With However, if the user has enabled Thus, with
Going to a general level: Of course, it is challenging to manage complex privilege configurations, especially when these include many dimensions and when the organization is large. However, IMHO, this is an inherent property of complex organizations, which never can be completely avoided. Also, it is not exclusive to Dashboards, but to any application that accesses OpenSearch indices. Thus, the "significant back and forth" you described above will be always present to a certain extent. I believe that better diagnostic information would be better suited to improve this than user-level index patterns.
Saved objects in Dashboards are extensively interconnected. For example, a dashboard contains several visualizations; all these entities are individual saved objects. I think we need some definition what is supposed to happen when a user tries to share an object which includes other objects the user does not share.
In many security configurations - especially for clusters with SSO-based authentication like SAML or OIDC - it is either difficult or even impossible to get information about users other than the currently logged in user. Consider the following example:
Dashboards is now unable to provide any guidance with the sharing process, as it does not have any information on the new user. It is impossible to implement a user search functionality that would be able to search for the new user. If you allow entering the bare user name, Dashboards would be unable to verify whether the user exists or not. A workaround would be that Dashboards would keep a list of users which have logged in before. Thus, the new user would need to login at least once until they can receive shared objects. Then, however, the opposite case would be an issue: If a user is deleted in the IdP, Dashboards would get no information about this. Thus, other users could still share objects with that user, even though that user would never be able to see these.
Minor, non-relevant nit-pick: currently, Dashboards objects are owned by the tenant that creates them. If you want to move the ownership to users, we need a more precise definition what ownership means. Also, it should be clarified whether and how ownership can be transferred, for example for the case that a user leaves an organization. |
@nibix Thanks so much for taking the time to evaluate and share your feedback on this proposal. That is a fair point that a role like Eg: Lets take When The change of index pattern association is because currently tenants own Dashboards objects (thanks for pointing it out, fixed it main issue) and one cannot share an object without first asking the admin to also include the said user in the same tenant. The requirement of including a user/role in a tenant just so another user can share a Dashboard object (eg: visualization) is confusing. Many customers have pointed this out as an issue that inhibits easy sharing of Dashboard objects. I wasn't referring to the do_not_fail_on_forbidden setting which allows the user to choose between all_or_none results vs excluding the index pattern matched indices that the user doesn't have access to. Apologies if i wasn't completely clear about that. You have a valid point on adding diagnostics to make the mapping and access easier. I can see that complex organizations will have some level of this back and forth, but we need to find the right balance and mechanisms to simplify the interactions. Regarding nested objects sharing - the idea is when a dashboard is shared, the nested objects within the same, will also be shared as a part of it. With tenants, this is easy. But if we consider a different model without tenants, then we need to account for this. Thank you for sharing your thoughts on the user discoverability. That is an excellent point that raises the questions, who are you sharing the object with and how does one identify/lookup such a user/role. This one needs some more thinking and discussion. Again, thanks for sharing your thoughts and moving the discussion along. This is the type of feedback that would help refine this proposal to create an experience that is broadly applicable to users of OpenSearch. |
Thank you @jgough and @squiddy-gh on sharing your thoughts regarding the benefits of compartmentalization and how it is suitable for your usecases. While many users have requested to disable tenants entirely or make the experience simpler, i understand that usecases like yours still benefit from the current design. I will need to think a little bit more about this and how we can keep the benefits of the current design while simplifying its use for all users. So definitely agree that we dont want to change it and make it complex to use and manage. Ill share more on this. @squiddy-gh regarding the UUID issue, while not directly a part of the security plugin, is important and im tagging @ahopp who may have ideas around this. The idea for a self service basic auth user signup is fantastic. I like your suggestion about adding password policies related to aging, length/strength, force users to change the default password etc. While not in the scope of this proposal, it will enhance the user experience. I will track it separately. (Also, feel free to open a feature request issue here in GitHub) Thanks again for sharing your ideas and reviewing this proposal. |
I don't really understand this scenario. In the status quo, there are three cases:
If you change the index pattern to be user-specific, would RoleA see the contents of |
Conversation moved here opensearch-project/OpenSearch-Dashboards#4298, linking to preserve these thoughts into the new Workspaces effort. |
Background
OpenSearch users have adopted the security plugin to manage the data permissions of users across various teams within their organization. In addition to providing fine grained access control for actions on the data and the cluster, various authentication mechanisms including basic and SAML, the security plugin also provides the ability to isolate OpenSearch Dashboards objects using a concept of tenants. An OpenSearch cluster is often shared across different teams or customers using services of a common vendor. In such scenarios we need to provide admins with the necessary tools to easily isolate data sets as well as any derived Dashboards objects including visualizations, dashboards, reports etc.
Problem statement
OpenSearch users want to share and collaborate on Dashboard objects with one or more users/teams in a simple and easy workflow, while having the ability to see and filter through private or shared objects in the same view. In addition, users want a simple way to isolate the data and Dashboard objects in a scoped view so only users of that role can access the data and associated Dashboards objects (eg: visualizations, saved queries, reports, dashboards etc) to generate valuable insights and support in making business decisions.
Existing solution for sharing Dashboards objects
Tenants in Dashboards is a popular feature that provides the ability to end users to create shared collaboration spaces for saving Dashboards objects such as visualizations, dashboards, reports, saved searches, default index patterns. This is the only solution that helps admins to configure their clusters to provide a clear and isolated view to multiple teams in their organization or to multiple clients that may be hosted on the same cluster. This customized view helps make business decisions faster by providing relevant insights to specific sets of users.
Tenants are configured by the admin who is responsible for all the security configuration and permissions mapping for users in the organization. Tenants are most commonly used in the following ways
Shortcomings of the existing solution
While tenants help in separating Dashboards objects, they are often misunderstood to also provide a traditional multi-tenant architecture that provides data separation and/or infrastructure isolation (hardware, power etc). This is simply untrue as tenancy is designed as a Dashboards concept and should not be mixed with the separate role to permissions mapping needed for the data indices stored in OpenSearch.
Following is a comprehensive list of shortcomings of tenants as a mechanism to share and collaborate
This has resulted in a sub-par experience for OpenSearch users who often ask to disable tenancy entirely.
Proposed solution
OpenSearch users have made us aware of many of the shortcomings of the current solution that is difficult to use, while lacking some critical features. This is an opportunity to rethink the approach to enabling sharing with OpenSearch Dashboards.
The shortcomings highlighted in the above section have led us to rethinking the approach to sharing Dashboards objects.
As we have seen above, the current mechanism doesnt meet users requirements of sharing and collaborating on Dashboards objects in a simple and easy way. This is an opportunity to redesign and simplify the core workflows and functionality that is currently partially fulfilled by tenancy, to create a simpler user sharing experience. Following is a list of recommended features
New concepts
As we work on rethinking how we enable OpenSearch users to share and collaborate, we can preserve some core concepts, while introducing new ones. Following are concepts that need to be introduced for a seamless sharing experience.
User profile
Currently we support very limited user specific settings that are remembered on login, but only from that specific browser (eg: last used tenant). We don't have any mechanisms in place that remember user preferences across logins. Also some experiences need to evolve from being tied to tenants, instead to be part of a user profile. For eg: index pattern is currently tied to a tenant or advanced settings (such as settings for search, visualization, timeline, notifications, allowing leading wildcards, dark mode etc) are per tenant. Every user can have a unique data permissions, preferences and should be able to select an index pattern or Dashboards settings that are relevant to the queries they run. This will also eliminate the issue of users with different roles logging into the same tenant and at least one of them not being able to see any data visualized due to the default index pattern not matching their data permissions or being forced to use the advanced settings of the specific tenant. User profile will also be helpful if we decouple Dashboards from OpenSearch, as it will help narrow down the specific query in the context of that user.
Sharing and permissions
Sharing will replace the core functionality that tenancy provides today, by allowing a mechanism to collaborate with other users and roles. Each Dashboard object can be individually shared with different permissions (read, read/write) with other users or roles. Each user will have the ability to share the Dashboard object that they create, but admin can limit users from sharing altogether (eg: for read only roles).
Dashboards objects ownership
Currently Dashboards objects are owned by the tenant that creates them. One or more users may be assigned to that role and direct ownership of the object isn't saved. As we introduce the concept of sharing, it is critical to have ownership of individual Dashboard objects. By default any newly created object will be owned by the user creating it.
Aggregate object views
Currently if a user has access to multiple tenants, they need to switch between tenants, in order to see the relevant Dashboard objects of that tenant. We will introduce a new concept of aggregate views wherein the user can see the full list of all Dashboard object categories including objects that they created and objects that are shared with them. We will provide powerful filtering mechanisms that would allow users to easily find the objects they are looking for. Eg: On the dashboards page, users can see a list of all the dashboards that they created as well as dashboards that were shared with them by other users. Similarly for different categories of Dashboard objects including visualizations, saved searches etc, each of these objects will provide a simplified listing of all objects of that type. The user will not have to leave the page and jump between tenants to find the objects they are looking for. Aggregate object views will provide a single pane of glass for Dashboards objects.
As a result of the sharing mechanism, we will also do away with the inability of moving dashboard objects from one tenant to other. Now there is no need to do that as the user can see all the objects in a single unified pane.
Administrator or security manager dashboard
One of the other problematic areas that goes hand in hand with tenancy is data isolation. While tenancy wasn't designed for data permissions, we are proposing a simpler design that would allow to make this experience easier. When a user views a shared dashboard object that they don't have data permissions to visualize, there would be a mechanism for the user to “Request permission”. Such requests will show up in a admin/security manager panel where they can be approved or denied with the click of a button. We will also provide mechanisms to group actions for easier management. Once approved, the requesting user can see the data that they need to see. In the background, the system will add the data permissions (with explicit approval from the admin) needed by the user to view the requested data.
Tagging
As clusters scale, there will be several users and continuous stream of Dashboards objects creation activity. In order to simplify viewing, filtering, sharing and grouping, we will introduce the mechanism of tagging objects. This will allow users to share various categories of Dashboards objects in one flow. Tagging is also useful to take bulk actions such as deleting objects, group tagging, sharing etc. Eg: If a user wants to share all relevant objects related to “incident-april-5-2022”, they can do so by simply filtering for objects with a specific tag and share them in one workflow. Note that since tags are relevant to the specific user, when objects are shared, the associated tags are not. Every user can tag and group objects how they see fit for their usecase without being overwhelmed with tags associated by object creators.
Simplifying role permissions
Currently roles are assigned access to tenants with either a read or read/write permissions. This creates confusion between boundaries of data permissions and dashboard object permissions, leaving admins often struggling to find the perfect combination of data, plugin, cluster and tenant permissions. In addition some roles are exclusively defined for plugins making it time consuming and difficult to arrive at the perfect combination of permissions for a given user or role. We will aggregate all permissions within one workflow so admins can decide the right cluster, index, plugin and sharing permissions in the same role mapping workflow. Admins can turn on “Sharing by default” in any role, so any user who has assumed that role, can automatically share their newly created Dashboard objects with other users of the same role meeting the needs of users who want data isolation but also share Dashboard objects within the same role. This will give admins a comprehensive view of the role permissions across data, cluster, plugins and sharing.
Migration of existing tenants and dependent plugins
While we introduce new, powerful and simplified concepts, we need to find a proper migration path for clusters currently enabled with tenancy. We need to do this without changing the scope of the permissions and without requiring significant overhead on the administrator for any reconfiguration. There are some plugins that also use the concept of tenancy (eg: Reporting). We need to provide a clean migration path for such plugins.
Index patterns are being migrated from being associated with a role to being associated with a user. This is a significant shift in usage behavior and we need to ensure that post migration the scope of permissions stays the same while each user also has access to index patterns previously associated with the tenants that their roles had access to.
Requirements
Attached requirements highlight how we wish to integrate the core concepts of sharing into existing workflows while carefully introducing new concepts to meet the users requirements (See attached spreadsheet)
Sharing with Dashboards GitHub v1.xlsx
Ask
We are requesting the community to evaluate this RFC, share their feedback, comments and proposals.
The text was updated successfully, but these errors were encountered: