[Question] How should service account permissions be stored and where?

[stephen-crawford]

Problem Statement

Where and how should extension permissions be stored?

$\textcolor{orange}{\textsf{Orange text is a header for a decision that is not made }}$
$\textcolor{teal}{\textsf{Teal text is a header for a decision that is made }}$
$\textcolor{violet}{\textsf{Violet text is the recommended option }}$
$\textcolor{lime}{\textsf{Lime text is the chosen option }}$

Action items decided on this issue:

• Where should service accounts be stored?
• How should permissions be stored?
• Should roles be used for extensions?
• How should extensions be tracked or managed?

Where and how extension permissions are stored is an important consideration when designing the overall authorization structure. Currently, user permissions are stored as part of the configuration repository. The configuration repository is shard'd onto every node in the cluster and the Security Plugin is stored on every index as part of a cache. Every time there is a change in the Security settings, the cache is invalidated and reloaded from the shard'd repository which will be eventually consistent. The current system has issues as the number of changes touching the configuration repository grow. Every time there is a change, the repository is updated on all the nodes and all caches are invalidated and require reloading. This leads to a number of scaling issues and also causes multiple requests to be sent throughout the cluster every time there is an API call that touches the security plugin.

$\textcolor{teal}{\textsf{Where should service account permissions be stored?}}$

There are two main locations where it makes sense to store extension permissions.

1. $\textcolor{lime}{\textsf{Store service account permissions with user permissions}}$ The first option for where to store extension permissions is alongside user permissions in the configuration repository. This option is easy to implement since the repository is already setup for handling permission insertion and querying. Likewise, this option simplifies the following question of how extension permissions should be stored.

Pros	Cons
Easy	Exacerbates scaling issues
Answers how the permissions should be stored at the same time	Compounds cluster latency

What do code changes look like?

2. $\textcolor{violet}{\textsf{Store the extension permissions data as part of a separate system index}}$ The second option is to store the extension permissions data as part of a separate system index. This option would minimize the number of full-cluster reloads since it would be separate from the configuration repository where the rest of the settings are stored. This system index would need to be present and updated on every node but there would be less cluster restarting. This separated index could also be used as a more general permissions index and either include the user permission settings or the entirety of the Security settings. The downside of this option is it requires more engineering time to implement than the first approach. Since there is not an index already present that we will be using, it will require looking into making a new security index and storing the settings there. Nodes could then update their caches from the security system index without having to reload their entire configurations.

Pros	Cons
Fixes scaling issues	Requires creating a new index for storing permissions
Could migrate user permissions to this new index as well	Not clear how to guarantee synchronicity between nodes

What do code changes look like?

$\textcolor{teal}{\textsf{What data structure should permissions be stored in?}}$

For choosing a data structure, it seems there are numerous valid options with the two most prominent being maps and lists, and maps and tries.

1. $\textcolor{violet}{\textsf{Maps and lists}}$ With maps and lists, an extension could be used as the key and have all its individual permissions stored in a list as the key's value. This option will have O(n) time complexity since you will lookup the extension as the key and then have to iterate over the elements in the list for the target permission.

2. With maps and tries, the extension would still be used as the key but now the various permissions would be stored in a trie structure. When resolving a permission you would access the key value and then parse the trie until you reached a leaf indicating the permission was not found or you found the appropriate permission.

Example diagram:

flowchart TD;
    A[*] -->B(indices:);
    A --> C(cluster:);
    B --> D(admin/);
    C --> E(admin/);
    D --> F(read/);
    F --> G(*);
    F --> H(my_index);
    H --> I(1);
    H --> J(2);
    E --> K(snapshot);

Loading

3. $\textcolor{lime}{\textsf{Treat the permissions the same as user permissions and store them in the same manner.}}$

NOTE: This question is closely related to the previous question about where extension permissions are stored. If extension permissions are stored alongside user permissions then it makes sense to simply store extension permissions in the same manner as user permissions. This option could use the extension ID as the stand-in for the username and treat everything else the same.

$\textcolor{teal}{\textsf{Should roles be used for extensions?}}$

For user permissions, the Security Plugin makes use of roles to resolve authorization. Roles are used to grant numerous users the same set of permissions based on their membership. These roles are then resolved into their constituent permissions when a request is processed.

As an example consider two users Sean and Nushi. These users work at Library.com, whose administrator has made roles Librarian, Accountant, and Administrator. The Librarian has permissions read, archive, write. The Accountant has permissions read, write, calculate. The Administrator has permissions read, write, archive, calculate, delete. Sean is assigned roles Librarian and Accountant. Nushi is assigned roles Accountant and Administrator.

When Sean then attempts to read a book, the cluster first accesses his assigned roles. It sees he is assigned Librarian and Accountant and views the permissions associated with Librarian. Because Librarian has the permission read Sean is able to read the book. When Sean then tries to delete the book because he feels it is outdated, the cluster checks the permissions for Librarian once more. Permission for delete is not part of that role so the cluster then checks the Accountant permissions. Again, delete is not part of that role's permissions so Sean is unable to perform the action. Nushi then comes along and tries to delete the book. Because Nushi is an Administrator, when the cluster resolves their roles, it sees they are able to perform the action. The book is deleted.

Roles are used for user permissions because of the nature of large organizations where it is important to be able to assign the same set of permissions to many people. For example, everyone at Library.com is an employee, but as shown above, not everyone is an administrator. A cluster administrator would not want to have to iteratively assign the same set of basic permissions i.e. read, to the thousands of employees at the company. They also do not want everyone to be able to delete books.

Returning to the topic at hand, there are three options for dealing with roles with extensions.

NOTE: Role support could be added at a later point, but removing role support after adding it would be inadvisable.

1. $\textcolor{lime}{\textsf{Extensions make use of roles}}$ It is reasonable that cluster administrators may want to grant all their extensions a set of basic permissions. For this to happen, extensions could be stored alongside users and assigned a service account which could be processed in a similar manner to a normal user account. When an extension then attempted to execute an action, the permissions could be checked by resolving the roles in a similar manner to that described above.

Pros	Cons
Very clear implementation path	Requires service account mechanism or using extension ID
Fast turn around	Administrators probably do not want to grant extensions a bunch of permissions
Easy to repeat permission grants	Far fewer extensions than users so less need

What do code changes look like?

f roles are used, then the Extension object (if implementation goes that route) should allow for storage of associated roles as part of the metadata. The roles can be stored in an attribute of the class instance and then queried when resolving the permissions. Alternatively, an extension service account could be used as a reference to the given extension and the account could be stored in the same manner user accounts are stored. When the roles were to be resolved, they could be resolved exactly like a user's roles would be.

2. Extensions do not use roles for permission resolution but do allow for roles (or something like roles) for granting permissions. One of the benefits of roles is that administrators are able to batch grant permissions to extensions. However, the resolution of permissions through roles is O(n^2) time complexity and roles as a concept may be less clearly relevant to extensions. An alternative to roles which are simply permission groups could be used for assigning permissions to extensions.

Pros	Cons
Still allows for batch grants	Have to store the permission groups somewhere
Not particularly complicated	Administrators probably do not want to grant extensions a bunch of permissions
Easy to repeat permission grants	May lead to unintentional grants

What do code changes look like?

If roles are only used for granting permissions but not resolving them, then it will be necessary to make further changes to the code base. Specifically, this option requires either an Extension object class which can have the assigned permissions associated with each extension as an attribute or an extension ID of some kind that can be used to reference a list of permissions somewhere. This option could still allow for extensions to slot in with user accounts but the method of resolving would require changes in the PrivilegesEvaluator of the Security Plugin so that permissions can be resolved without roles. Changes will also be needed in the permission granting code since role-based permission grants to extensions will need to be immediately resolved into the constituent privileges.

3. $\textcolor{violet}{\textsf{No roles and no permission groups}}$ The final option is to not use roles at all and not allow for permission groups. Batch permission assignment would still be possible, but there would not be a persistent object which archived a set of permissions to be granted. Assuming extensions have a set of basic access permissions they request on installation, there is less of a need for batch permission assignment. Resolving permissions via a list or trie is also faster than the current implementation.

Pros	Cons
Fastest permission resolution	Requires new permission resolution process
Makes unintentional permission grants harder	Cannot resolve permissions in the same way that user permissions are resolved
Follows principle of least privileges	More manual intervention required

What do code changes look like?

This option would require the same code changes as the previous option minus the code needed to make role-based granting resolve immediately into permissions.

$\textcolor{teal}{\textsf{How are extensions tracked or managed?}}$

In order to handle extensions, it is important to have some method of tracking them. There has to be something that allows OpenSearch to identify which extension is which and have confidence it is operating on the correct extension's behalf. Likewise, the Security Plugin will benefit from having a reference to different extensions and their metadata. Extension metadata could include things like the permissions an extension has, an extension's ID, and references to any predefined roles the extension expects to operate with. There are two main options for handling this tracking.

1. The first option is a dedicated Extension object class with attributes of permissionsList/rolesList (depending on whether roles are used or not), extensionId representing a unique extension reference, actionToPermissionsMapping representing the specific permissions that are required for an action to be able to complete its registered actions, and predefinedRoles which lists the roles it registered on installation.

classDiagram
    class Extension{
        +rolesList/permissionsList
        +extensionId
        +predefinedRoles
        +actionToPermissionsMapping
        getExtensionID()
        getPredefinedRoles()
        getPermissionsList()
        getActionToPermissionsMapping()
        equals()
    }

Loading

Pros	Cons
Object-oriented design is intuitive	Requires entire new object when we can mostly reuse user account system
Easy to reference a given extension throughout the Security Plugin	Takes up memory
Can be populated when extension is installed	Requires extension designers to specify a lot of information

What do code changes look like?

If an Extension object class were to be used, the code changes would be significant. It would require adding an entire new Extension object into OpenSearch core which would then be inherited by the Security Plugin. This option would also require the registration of all predefined roles and action to required permissions when an extension was first installed. On top of the object class, code changes would need to be made to the PrivilegesEvaluator in the Security Plugin to access the appropriate Extension object and parse its permissions when authorizing its requests. Further code changes would likely be required elsewhere in the Security Plugin for retrieving the Extension ID from a token and comparing it to an Extension object's associated Id.

2. $\textcolor{lime}{\textsf{Use service accounts.}}$ The second option is to use service accounts that have restricted privileges. For each extension, an internal user will be generated on installation that matches the extension id. This internal user will represent the service account and have limited capabilities. Specifically, service accounts can only have a single role, cannot have any backend roles, and are passwordless. With service accounts, whenever a request is received at core, the extension will be identified and the corresponding service account will be parsed to find the associated role. The role will effectively be a list of its unique permissions.

Pros	Cons
Can all be stored in the configuration cache	Requires having the extension ID since you cannot query it
Works alongside user accounts	Makes fake internal users that require tracking
Requires fewer changes	Does not allow for additional metadata

What do code changes look like?

This option requires fewer code changes than creating an Extension object would. Code changes would be required in OpenSearch core for the creation of an internal user whenever an extension was installed as well as for adding permissions to its single role. Code changes would not be required in the PrivilegesEvaluator since the role based resolution would be the same as what is currently in place.

[cwperks]

@scrawfor99 This is a great write-up! My thoughts are that there should be one policy associated with an extension that will dictate how requests originating from the extension can interact with a cluster.

In my mind, the policy will look similar to how roles are defined in the security plugin currently. For example:

anomaly_detection_extension_policy:
  description: "Anomaly Detection Extension Policy File"
  cluster_permissions:
    - "cluster_composite_ops"
  index_permissions:
    - index_patterns:
        - "logs-*"
      allowed_actions:
        - "read"

This policy would be configured by a cluster admin on installation and an extension could provide a default recommended configuration that the cluster admin could accept/edit before accepting.

Over the weekend, I was thinking about introducing the notion of a service account and restricting the service account to 1 role for this use-case. If that route were chosen, the identity associated with the extension could reside in the security index under internalusers and the role associated with the identity could be stored in roles. There could also be new CTypes created for the extension identities and policies associated with them. serviceaccounts and extension_policies respectively.

[peternied]

This issue is covering storage in terms of 'physical storage' location and I am also interested in the code structure of extensions objects themselves and the relationship between a service account, its permissions, vs the capabilities/limitations/scopes of an extension. Is everything on a single object, are there multiple objects, how does information get looked up, etc...

[stephen-crawford]

Hi @peternied, thank you for the comment. I will try to address that more clearly in an additional question. I know @cwperks had been wondering about this and left a comment on #2572 about service accounts. I will add a decision framework for that here and link on the other issue.

[stephen-crawford]

Hi @cwperks, thank you for your thoughtful feedback. I like your idea about the policy file. I have actually covered some similar ideas over on #2552 which may interest you. In the meantime, I am going to take some time to add a decision/question for service account vs. extension object vs. something else as the last item on this issue. I will first reference what you wrote over on #2572 though.

[stephen-crawford]

Hi @opensearch-project/security, I am going to make some executive decisions to try to drive these questions to a resolution. For each of the points below, I will be moving forward unless I hear objections by Monday 4/3.

• Where should extension service accounts be stored: option 1. They should be stored with internal users for the time being. This is the fastest approach. We can move things later if we need.
• How should permissions be stored: Permissions will be stored in the same structure as user permissions. This is again the fastest approach and can be modified later.
• Should roles be used for extensions?: Option 1: Extensions make use of role(s). This is again the easiest to implement since we can just treat extensions like users and resolve permissions in a similar manner.

[cwperks]

@scrawfor99 I really want to be clear with language.

1. What do you mean by extension stored as a user? Do you mean the service account associated with the extension? If so, then I think re-using internalusers is a good place and a 2-way door decision, but make sure that they are flagged accordingly.

2. What do you mean by roles here exactly? There should be a governing policy for an extension that looks exactly like a roles definition. The only difference in a design that I have in mind is that it is not mappable to a user, it has a naming convention (i.e. extension/ad would be the role associated with the governing policy for ad) and initially cannot support FLS/DLS. I think re-using roles is fine, but these will also need to be flagged accordingly.

i.e.

# Governing policy for requests coming from AD extension
extension/ad:
    extension_policy: true # Should not be mapped to a user
    reserved: true
    cluster_permissions:
        - 'cluster_monitor'
    index_permissions:
        - index_patterns:
            - 'logs-*'
        allowed_actions:
            - 'indices_monitor'
            - 'indices:admin/aliases/get'
            - 'indices:admin/mappings/get'

5. If you mean the role associated with the service account token, then I think we can also use a role for that and have a naming convention.

i.e.

extension/ad_svc:
    reserved: true
    index_permissions:
        - index_patterns:
            - '.opendistro-anomaly-results*'
            - '.opendistro-anomaly-detector*'
            - '.opendistro-anomaly-checkpoints'
            - '.opendistro-anomaly-detection-state'
        allowed_actions:
            - 'indices_all'

could be the roles definition for a service account token created for the service account associated with AD.

[peternied]

Should we rename this to something more along the lines of service account permissions?