Introduce Application Privileges to Roles #30164

tvernum · 2018-04-26T07:03:53Z

This commit introduces "Application Privileges" (aka custom privileges) to the X-Pack security model.

Application Privileges are managed within Elasticsearch, and can be tested with the _has_privileges API, but do not grant access to any actions or resources within Elasticsearch.
Their purpose is to allow applications outside of Elasticsearch to represent and store their own privileges model within Elasticsearch roles.

Specifically, this adds

GET/PUT/DELETE actions for defining application level privileges
application privileges in role definitions
application privileges in the has_privileges API

Closes: #29820

Adds - CRUD actions for defining application level privileges - application privileges in role definitions - application privileges in the has_privileges API

elasticmachine · 2018-04-26T07:03:55Z

Pinging @elastic/es-security

tvernum · 2018-04-26T07:17:42Z

@kobelb This is ready for you guys to test out and offer feedback.
I'm afraid there's no docs yet, but I'll give you some pointers in case you want to play with it before the docs are ready.

For REST Endpoints see

To see the usage of those APIs check

https://github.com/tvernum/elasticsearch/blob/feature/2767-custom-priv/x-pack/plugin/src/test/resources/rest-api-spec/test/privileges/20_has_application_privs.yml
That test walks through the whole process of defining privileges, attaching them to roles, granting those roles to users, and then testing whether the users have the matching privileges.

kobelb · 2018-04-26T18:46:44Z

This is awesome, thanks so much @tvernum. I just started integration it with our stuff, and I'll let you know how it progresses.

I noticed that users with the superuser role don't intrinsically get all application privileges, would it be possible to change this so they do?

tvernum · 2018-04-26T22:43:15Z

I noticed that users with the superuser role don't intrinsically get all application privileges, would it be possible to change this so they do?

Thanks for the reminder, I had forgotten about that.

kobelb · 2018-04-27T18:15:48Z

Also, in the _has_privileges call, it would seem that we'd want to be using "actions" the key in the application array, as opposed to "privileges", as we're checking to see if they have the specified actions.

kobelb · 2018-04-27T18:37:27Z

You might be planning to explain this in the docs, but would you mind explaining the role that the : plays when defining the actions for the custom privileges? I was able to create the following privilege/actions:

{
  "kibana": {
    "read": {
      "application": "kibana",
      "name": "read",
      "actions": [
        "version:7.0.0-alpha1",
        "saved-objects/dashboard/get",
      ],
      "metadata": {}
    }
  }
}

but then when I went to check the _has_privileges with

{
    "applications":[
        {
            "application":"kibana",
            "resources":["*"],
            "privileges":["saved-objects/dashboard/get"]
        }
    ]
}

it was returning false.

I noticed that I probably should be using the action: prefix so that I can always be doing the version check properly, and then everything started to work, but I feel like I'm missing something behind how all these components work together.

kobelb · 2018-04-27T20:06:12Z

Apologies for all the questions, does Elasticsearch do some type of caching with the application privileges? I've noticed a few situations where things were behaving weirdly and not authorizing users as I expected where if I restarted Elasticsearch and ran the steps again, it's begin working again. It might just be a "user error" as well.

tvernum · 2018-04-30T00:28:57Z

it would seem that we'd want to be using "actions" the key in the application array, as opposed to "privileges", as we're checking to see if they have the specified actions.

We could, but I've tried to mimic the cluster and index fields in has_privileges which use privileges.
The "privileges" field accepts either actions or high-level-privilege names. For your purposes it only makes sense to include action names, but the field allows a mix of both.

would you mind explaining the role that the : plays when defining the actions for the custom privileges

There isn't supposed to be anything to explain, and I don't understand why you saw the behaviour you did.
The only rule is that we need some way to distinguish between privilege names and action names, and we do that with / or : (or *, but that's actually playing the part of a wildcard).
You definitely shouldn't see any difference between / and :, except that security generally treats a leading / as indicating that the pattern is a regex, not a simple wildcard.

does Elasticsearch do some type of caching with the application privileges?

There shouldn't be any. There's caching on roles, but I pulled out what I think we the last piece of caching on privielges because it was broken.

tvernum · 2018-04-30T08:03:51Z

I don't understand why you saw the behaviour you did.

I haven't been able to reproduce this. I suspect it's related to whatever caching issue you saw (whether real or user-error), but if you can still reproduce it let me know.

My test case is:
Privileges

POST /_xpack/security/privilege
{"kibana": {
   "login":{"application":"kibana","name":"login","actions":["action:login","version:6.4.*"]},
   "read":{"application":"kibana","name":"read","actions":["action:login","version:6.4.*","data:read/*","saved-objects/dashboard/get"]}
}}

Role

POST /_xpack/security/role/test
{"applications":[{"application":"kibana","privileges":["read"],"resources":["*"]}]}

User

POST /_xpack/security/user/test
{ "password" : "magic123", "roles" : [ "test" ] }

Check

GET /_xpack/security/user/_has_privileges
{
  "index": [],  
  "applications": [ { "application": "kibana", "resources":["test"], "privileges":["saved-objects/dashboard/get"] } ]
}
# ...
{
  "username" : "test",
  "has_all_requested" : true,
  "cluster" : { },
  "index" : { },
  "application" : {
    "kibana" : {
      "test" : {
        "saved-objects/dashboard/get" : true
      }
    }
  }
}

jaymode

I left some initial feedback. I haven't made it through the PR yet but wanted to provide what I have so far.

jaymode · 2018-04-26T19:23:10Z

...ain/java/org/elasticsearch/xpack/core/security/action/privilege/DeletePrivilegesRequest.java

+    }
+
+    public DeletePrivilegesRequest(String application, String[] privileges) {
+        application(application);


can we use this.application = application and this.privileges = privileges? I find this much easier to follow than going to a method to see that this is all the method does.

jaymode · 2018-04-26T19:36:41Z

...a/org/elasticsearch/xpack/core/security/action/privilege/DeletePrivilegesRequestBuilder.java

+        this(client, DeletePrivilegesAction.INSTANCE);
+    }
+
+    public DeletePrivilegesRequestBuilder(ElasticsearchClient client, DeletePrivilegesAction action) {


I don't think we need this constructor and can just use the one above and call super with a instance of the request

Unfortunately need it in order to implement DeletePrivilegesAction.newRequestBuilder, but I've made it package protected.

jaymode · 2018-04-26T19:38:19Z

...in/java/org/elasticsearch/xpack/core/security/action/privilege/DeletePrivilegesResponse.java

+    }
+
+    public DeletePrivilegesResponse(Collection<String> found) {
+        this.found = new HashSet<>(found);


lets wrap in unmodifiable set here

jaymode · 2018-04-26T19:38:59Z

...in/java/org/elasticsearch/xpack/core/security/action/privilege/DeletePrivilegesResponse.java

+    }
+
+    public Set<String> found() {
+        return Collections.unmodifiableSet(this.found);


if we wrap in unmodifiable set in the constructor and serialization, this can just be return found;

jaymode · 2018-04-26T19:39:49Z

...in/java/org/elasticsearch/xpack/core/security/action/privilege/DeletePrivilegesResponse.java

+    public void readFrom(StreamInput in) throws IOException {
+        super.readFrom(in);
+        final int size = in.readVInt();
+        found = new HashSet<>(size);


Lets create a local value, add all of the values and then assign found to an unmodifiable set

jaymode · 2018-04-30T20:35:42Z

.../main/java/org/elasticsearch/xpack/core/security/authz/permission/ApplicationPermission.java

+                return new PermissionEntry(k, Automatons.unionAndMinimize(Arrays.asList(existing.resources, patterns)));
+            }
+        }));
+        this.permissions = new ArrayList<>(permissionsByPrivilege.values());


make this an unmodifiable list?

jaymode · 2018-04-30T20:36:38Z

.../main/java/org/elasticsearch/xpack/core/security/authz/permission/ApplicationPermission.java

+    public boolean grants(ApplicationPrivilege other, String resource) {
+        Automaton resourceAutomaton = Automatons.patterns(resource);
+        final boolean matched = permissions.stream().anyMatch(e -> e.grants(other, resourceAutomaton));
+        logger.debug("Permission [{}] {} grant [{} , {}]", this, matched ? "does" : "does not", other, resource);


this feels like a trace level log to me

jaymode · 2018-04-30T20:38:37Z

.../main/java/org/elasticsearch/xpack/core/security/authz/permission/ApplicationPermission.java

+        this.permissions = new ArrayList<>(permissionsByPrivilege.values());
+    }
+
+    public boolean grants(ApplicationPrivilege other, String resource) {


Can you add javadocs?

jaymode · 2018-04-30T20:40:42Z

...rc/main/java/org/elasticsearch/xpack/core/security/authz/privilege/ApplicationPrivilege.java

+    }
+
+    public ApplicationPrivilege(String application, String privilegeName, String... patterns) {
+        this(application, Sets.newHashSet(privilegeName), patterns, Collections.emptyMap(), true);


use Collections.singleton here

jaymode · 2018-04-30T20:41:17Z

...rc/main/java/org/elasticsearch/xpack/core/security/authz/privilege/ApplicationPrivilege.java

+        this(application, Sets.newHashSet(privilegeName), patterns, Collections.emptyMap(), true);
+    }
+
+    private ApplicationPrivilege(String application, Set<String> name, Collection<String> patterns, Map<String, Object> metadata,


do we need this constructor?

kobelb · 2018-05-03T19:47:32Z

@tvernum you were right that it appears to be a caching issue. If I perform the following steps to create the privileges, and a role/user with those privileges, and then update the privileges, the _has_privileges API returns false until I clear the cache for that role:

POST _xpack/security/privilege
{
  "kibana": {
    "read":{
      "application":"kibana",
      "name":"read",
      "actions":[
        "version:7.0.0-alpha1",
        "action:saved-objects/config/get"
      ]
    }
  }
}

POST _xpack/security/role/kibana_rbac_user
{
  "cluster": [],
  "indices": [],
  "applications": [
    {
      "application": "kibana",
      "privileges": [ "read" ],
      "resources": [ "default" ]
    }
  ]
}

PUT _xpack/security/user/brandon
{
  "password" : "password",
  "roles" : [ "kibana_rbac_user" ],
  "full_name" : "Brandon",
  "email" : "someone@gmail.com"
}

POST _xpack/security/user/_has_privileges
{
    "applications":[
        {
            "application":"kibana",
            "resources":["default"],
            "privileges":["action:saved-objects/config/get"]
        }
    ]
}

^^ this returns has_all_requested: true

POST _xpack/security/privilege
{
  "kibana": {
    "read":{
      "application":"kibana",
      "name":"read",
      "actions":[
        "version:7.0.0-alpha1",
        "action:saved-objects/config/get",
        "action:saved-objects/config/search"
      ]
    }
  }
}

POST _xpack/security/user/_has_privileges
{
    "applications":[
        {
            "application":"kibana",
            "resources":["default"],
            "privileges":["action:saved-objects/config/search"]
        }
    ]
}

^^ this returns has_all_requested: false

Is this expected behavior? We're planning on occasionally updating the privileges map, and having to clear the cache of every role that has those privileges is rather cumbersome.

tvernum · 2018-05-04T01:08:59Z

Is this expected behavior?

It wasn't something I predicted, but it's consistent with some other caches in security.
If you change role-mappings it doesn't affect users that already cached.

Updates to roles are live because the cached link between Users and Roles is simply by name, so if the role changes in the cache the User will see the new role on next request.

But if you change a role-mapping so that LDAP user "bob" is supposed to now have "kibana_user", that won't be effective until the cache for "bob" is cleared.

We're planning on occasionally updating the privileges map, and having to clear the cache of every role that has those privileges is rather cumbersome.

You can clear the cache for all roles in 1 API call, though obviously that's a pretty blunt instrument.

I'll look at options, but my gut feel is that I'd like to update the clear roles API to be able to accept an application name so you can clear all roles that have "kibana" privileges once you've finished adding/deleting/updating privileges.

tvernum · 2018-05-04T07:01:51Z

We're planning on occasionally updating the privileges map,

@kobelb How often does this happen? We can change the clear-cache API to ben more granular, but if the updates are infrequent, I think it would be simpler just to clear the whole cache when you update.

jinmu03 · 2018-05-04T07:19:37Z

@tvernum will a more granular control of the cache have any big impact on the performance of the API?

kobelb · 2018-05-07T14:59:27Z

@tvernum the current plan is to execute the Privileges PUT every time that an instance of Kibana starts up for all the custom privileges to ensure that we have the information in Elasticsearch to perform the authorization using the current privileges.

In an ideal configuration, this would only happen once for every instance of Kibana for a long period of time; however, in an unideal situation or one where new instances were being provisioned frequently, it could happen somewhat frequently. Unfortunately, it depends...

tvernum · 2018-05-28T03:54:21Z

@jaymode @albertzaharovits
This is ready for another review.

@albertzaharovits I addressed your comment on the JSON input, but I'm going to hold off on your two
performance related suggestions until after I split the ApplicationPrivilege class.
My plan is to do something like we do for Roles, where the Role class represents the runtime behaviour and the RoleDescriptor is the stored format.

kobelb · 2018-06-01T19:14:44Z

@tvernum I just came across an interesting behavior, when I create a privilege with a capital letter in it, then the _has_privilege check always comes back false, but if change it to be all lower case letters then it works correctly.

jaymode

I left a few minor comments, otherwise I am good with merging this to the feature branch and moving forward.

jaymode · 2018-06-04T03:47:38Z

server/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java

@@ -969,6 +970,16 @@ public void writeList(List<? extends Writeable> list) throws IOException {
        }
    }

+    /**
+     * Writes a list of generic objects via a {@link Writer}


s/list/collection

jaymode · 2018-06-04T03:52:49Z

.../java/org/elasticsearch/xpack/core/security/action/privilege/GetPrivilegesResponseTests.java

+        assertThat(copy.privileges(), Matchers.equalTo(original.privileges()));
+    }
+
+    private <T> T[] randomArray(int max, IntFunction<T[]> arrayConstructor, Supplier<T> valueConstructor) {


lets put this into ESTestCase with javadocs and remove the duplication

tvernum · 2018-06-06T04:42:10Z

when I create a privilege with a capital letter in it, then the _has_privilege check always comes back false

I've tracked down the cause of this. I have a fix for it in a followup PR.

tvernum added 2 commits April 26, 2018 14:33

Support application (custom) privileges

fdf628b

Adds - CRUD actions for defining application level privileges - application privileges in role definitions - application privileges in the has_privileges API

Static caches are evil

f7463bc

tvernum added >feature review WIP v7.0.0 :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC v6.4.0 labels Apr 26, 2018

tvernum requested review from jaymode and albertzaharovits April 26, 2018 07:03

tvernum added 2 commits April 30, 2018 14:06

Merge branch 'master' into feature/2767-custom-priv

1af402c

Grant all application privileges to superuser

9b1ff6e

jaymode reviewed Apr 30, 2018

View reviewed changes

tvernum added 2 commits May 1, 2018 15:49

Address review feedback

662fd50

Merge branch 'master' into feature/2767-custom-priv

ccbb6eb

Merge branch 'master' into feature/2767-custom-priv

80977b0

tvernum added 5 commits May 25, 2018 18:05

Infer app/privilege name in PutPrivileges request

48820a2

Merge branch 'master' into feature/2767-custom-priv

99d4dd1

Fix fragile test

502dd4b

Merge branch 'master' into feature/2767-custom-priv

24fcff9

Fix import

2bcdadb

tvernum changed the base branch from master to security-app-privs May 28, 2018 03:48

tvernum added 2 commits May 30, 2018 18:53

Merge branch 'master' into feature/2767-custom-priv

ffd3058

Merge branch 'master' into feature/2767-custom-priv

85a11e0

jaymode approved these changes Jun 4, 2018

View reviewed changes

tvernum added 3 commits June 4, 2018 22:23

Merge branch 'master' into feature/2767-custom-priv

26629a2

Address feedback

f9a47a4

Merge branch 'master' into feature/2767-custom-priv

63512f1

This comment has been minimized.

Sign in to view

tvernum added 4 commits June 5, 2018 17:21

Update for elastic#30966

8098b96

Merge branch 'master' into feature/2767-custom-priv

d40da46

Merge branch 'security-app-privs' into feature/2767-custom-priv

92e8307

Fix ML use of HasPrivileges

5d80b39

tvernum added 3 commits June 6, 2018 17:07

Update doc snippets for API changes

6f6bf4c

Fix RoleDescriptor.toString

e8191a0

Merge branch 'master' into feature/2767-custom-priv

4ebba4a

kobelb mentioned this pull request Jun 6, 2018

RBAC Phase 1 elastic/kibana#19710

Closed

tvernum added 2 commits June 7, 2018 10:39

Merge branch 'security-app-privs' into feature/2767-custom-priv

ea5eeab

Fix broken test

f43f04c

tvernum removed WIP v6.4.0 v7.0.0 labels Jun 7, 2018

tvernum merged commit 03e5e72 into elastic:security-app-privs Jun 7, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Application Privileges to Roles #30164

Introduce Application Privileges to Roles #30164

tvernum commented Apr 26, 2018 •

edited

Loading

elasticmachine commented Apr 26, 2018

tvernum commented Apr 26, 2018

kobelb commented Apr 26, 2018

tvernum commented Apr 26, 2018

kobelb commented Apr 27, 2018

kobelb commented Apr 27, 2018 •

edited

Loading

kobelb commented Apr 27, 2018

tvernum commented Apr 30, 2018 •

edited

Loading

tvernum commented Apr 30, 2018

jaymode left a comment

jaymode Apr 26, 2018

jaymode Apr 26, 2018

tvernum May 1, 2018

jaymode Apr 26, 2018

jaymode Apr 26, 2018

jaymode Apr 26, 2018

jaymode Apr 30, 2018

jaymode Apr 30, 2018

jaymode Apr 30, 2018

jaymode Apr 30, 2018

jaymode Apr 30, 2018

kobelb commented May 3, 2018

tvernum commented May 4, 2018

tvernum commented May 4, 2018

jinmu03 commented May 4, 2018

kobelb commented May 7, 2018 •

edited

Loading

tvernum commented May 28, 2018

kobelb commented Jun 1, 2018

jaymode left a comment

jaymode Jun 4, 2018

jaymode Jun 4, 2018

This comment has been minimized.

tvernum commented Jun 6, 2018 •

edited

Loading

Introduce Application Privileges to Roles #30164

Introduce Application Privileges to Roles #30164

Conversation

tvernum commented Apr 26, 2018 • edited Loading

elasticmachine commented Apr 26, 2018

tvernum commented Apr 26, 2018

kobelb commented Apr 26, 2018

tvernum commented Apr 26, 2018

kobelb commented Apr 27, 2018

kobelb commented Apr 27, 2018 • edited Loading

kobelb commented Apr 27, 2018

tvernum commented Apr 30, 2018 • edited Loading

tvernum commented Apr 30, 2018

jaymode left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kobelb commented May 3, 2018

tvernum commented May 4, 2018

tvernum commented May 4, 2018

jinmu03 commented May 4, 2018

kobelb commented May 7, 2018 • edited Loading

tvernum commented May 28, 2018

kobelb commented Jun 1, 2018

jaymode left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

tvernum commented Jun 6, 2018 • edited Loading

tvernum commented Apr 26, 2018 •

edited

Loading

kobelb commented Apr 27, 2018 •

edited

Loading

tvernum commented Apr 30, 2018 •

edited

Loading

kobelb commented May 7, 2018 •

edited

Loading

tvernum commented Jun 6, 2018 •

edited

Loading