user management in Meta Server #257

boshengchen · 2019-04-03T13:08:14Z

related issue #180.

nebula-community-bot · 2019-04-03T13:09:05Z

Can one of the admins verify this patch?

dangleptr

Thanks for taking care of it.

As for User design. I think we should have some concepts firstly.

Role: it includes admin and normal types
Entity: it includes Space, Schema
Privilege: it includes Create/Drop/Alter/Read/Write/Grant

Each user has one role. Admin could grant any Privilege to one user with some entity.

By default, we have Privilege Table list below:

\	User	Space	Schema
Admin	cdarwg	cdarw	cdarw
Normal	-	rw	r

wdyt ?

dangleptr · 2019-04-04T06:22:14Z

src/interface/meta.thrift

@@ -181,6 +184,48 @@ struct GetPartsAllocResp {
 3: map<common.PartitionID, list<common.HostAddr>>(cpp.template = "std::unordered_map") parts,
 }

+// user management
+struct CreateUserReq {
+ 1: common.GraphSpaceID space_id,


Why we need space_id here when create user?

My idea is that every schema has its own users. Each user has one role. There are only three types of roles：
admin: this is a super role, anything can be done. includes Space, Schema
user: a producer role, can only access(read and write) the data what he belongs to.
guest: is a read-only role for a given Graph Space.

@boshengchen Please refer the issue #190. A user does not have to be associated with any space, although in most cases, a user has a default space associated. A user could be associated with multiple spaces, but could only have one default space. A user could have different roles in different spaces

dangleptr · 2019-04-04T06:23:28Z

src/common/base/ThriftTypes.h

@@ -25,6 +25,8 @@ using EdgeType = int32_t;
 using EdgeRanking = int64_t;
 using EdgeVersion = int64_t;

+using RoleType = int8_t;


We should use one enum to represent RoleType.

Second @dangleptr

If you want to use 8-bit enum, you can do

enum class RoleType : public int8_t { ... };

darionyaphet · 2019-04-04T02:48:12Z

src/meta/test/ProcessorTest.cpp

+ auto key = MetaUtils::userKey(1, "nebula");
+ std::string val;
+ auto ret = kv.get()->get(0, 0, std::move(key), &val);
+ ASSERT_TRUE(ret == kvstore::ResultCode::SUCCEEDED);


ASSERT_EQ would be better ?

good catch.

darionyaphet · 2019-04-04T02:54:12Z

src/interface/meta.thrift

+// user management
+struct CreateUserReq {
+ 1: common.GraphSpaceID space_id,
+ 2: bool missing_ok,


when user existed and missing_ok is set, the result will be SUCC ? It's make a confuse. What's your opinion?

missing_ok should be turn on when "IF NOT EXISTS" or " IF EXISTS" is set. the result will be SUCCEEDED whether the user exists or not.
for example:

CREATE USER [IF NOT EXISTS] username [WITH] [PASSWORD 'password'] [ROLE role_option]

DROP USER [ IF EXISTS ] username

It would be nice to have above explanation as a comment

darionyaphet · 2019-04-04T02:54:26Z

src/interface/meta.thrift

+
+struct DropUserReq {
+ 1: common.GraphSpaceID space_id,
+ 2: bool missing_ok,


darionyaphet · 2019-04-04T03:42:19Z

src/interface/meta.thrift

@@ -181,6 +184,48 @@ struct GetPartsAllocResp {
 3: map<common.PartitionID, list<common.HostAddr>>(cpp.template = "std::unordered_map") parts,
 }

+// user management
+struct CreateUserReq {
+ 1: common.GraphSpaceID space_id,


CreateUser should not indicate space, it's be related with Grunt

darionyaphet · 2019-04-04T06:54:50Z

src/interface/common.thrift

+struct UserItem {
+ 1: string user_name,
+ 2: string user_pwd,
+ 3: RoleType role,


RoleType is undefined ？enum is a good choice

darionyaphet · 2019-04-04T07:07:34Z

src/meta/test/ProcessorTest.cpp

+ {
+ ASSERT_EQ(TestUtils::createUser(kv.get(), 1, true, "nebula", "nebula", RoleTypeE::T_ADMIN),
+ cpp2::ErrorCode::SUCCEEDED);
+ ASSERT_EQ(TestUtils::createUser(kv.get(), 1, true, "nebula", "nebula", RoleTypeE::T_ADMIN),


duplicated with previous code :)

test purpose : if user exist and missing_ok is turn on

Please add a comment to help other people understand the logic

darionyaphet · 2019-04-04T07:08:39Z

src/meta/test/ProcessorTest.cpp

+ cpp2::ErrorCode::SUCCEEDED);
+ }
+ {
+ cpp2::DropUserReq req(FRAGILE, 1, true, "nebula");


CreateUser, DropUser and another command could join into one

darionyaphet · 2019-04-04T07:15:11Z

src/interface/meta.thrift

+}
+
+struct ListUsersReq {
+ 1: common.GraphSpaceID space_id,


darionyaphet · 2019-04-04T07:15:36Z

src/interface/meta.thrift

+}
+
+struct AlterUserReq {
+ 1: common.GraphSpaceID space_id,


I would suggest to separate the "Change Password" operation from AlterUserReq. "Change password" usually requires the old password

darionyaphet · 2019-04-04T07:20:12Z

src/meta/processors/AuthenticationProcessor.cpp

+ return;
+ }
+ userVal = MetaUtils::userVal(req.get_role(), req.get_password());
+ resp_.set_code(cpp2::ErrorCode::SUCCEEDED);


why set code here ?

sherman-the-tank · 2019-04-05T20:33:15Z

Great work! Thanks for taking care of this

Regarding the meaning of different roles, please refer issue #190

dutor · 2019-04-08T03:16:39Z

src/meta/processors/AuthenticationProcessor.h

+ T_GUEST = 0x03,
+};
+
+#define LOCK_CHECK(lock) \


At the first glance on this macro name, I didn't expect it would acquire the lock.

dutor · 2019-04-08T03:16:56Z

src/meta/processors/AuthenticationProcessor.h

+};
+
+#define LOCK_CHECK(lock) \
+auto& lock##Lock = LockUtils::lock##Lock(); \


Please indent these code properly.

dutor · 2019-04-08T03:24:32Z

src/meta/processors/AuthenticationProcessor.cpp

+}
+
+void AlterUserProcessor::process(const cpp2::AlterUserReq& req) {
+ LOCK_CHECK(user);


At the first glance, I thought the userLock had not been released when this function finishes.
I sugguest to rename doPut to doPutLocked, and add another interface doPut if needed who acquires the lock and invoke doPutLocked. Besides, these interfaces should be commented on the locks' usage.

@dangleptr What do you think?

sherman-the-tank · 2019-04-08T12:24:07Z

Jenkins go

nebula-community-bot · 2019-04-08T12:39:10Z

Unit testing passed.

sherman-the-tank

Thanks for taking care of this.

I think my biggest concern is the PR makes the user as an entity in a given space (the key/value is designed base on that assumption), but that's not the case (Please refer the issue #190). I think we need to redesign the way we store the user information

sherman-the-tank · 2019-04-08T12:48:50Z

src/common/base/ThriftTypes.h

@@ -25,6 +25,8 @@ using EdgeType = int32_t;
 using EdgeRanking = int64_t;
 using EdgeVersion = int64_t;

+using RoleType = int8_t;


Second @dangleptr

If you want to use 8-bit enum, you can do

enum class RoleType : public int8_t { ... };

sherman-the-tank · 2019-04-08T12:49:47Z

src/interface/common.thrift

@@ -21,6 +21,8 @@ typedef i64 (cpp.type = "nebula::VertexID") VertexID
 typedef i32 (cpp.type = "nebula::IPv4") IPv4
 typedef i32 (cpp.type = "nebula::Port") Port

+typedef byte (cpp.type = "nebula::RoleType") RoleType


sherman-the-tank · 2019-04-08T12:52:06Z

src/interface/common.thrift

@@ -73,5 +75,11 @@ struct HostAddr {
 2: Port port,
 }

+struct UserItem {
+ 1: string user_name,
+ 2: string user_pwd,


By looking at the following code, I understand the password here is encoded. I would highly recommend to add a comment here

In addition, I would suggest to rename it to something like "encoded_pwd". Simple and clear

sherman-the-tank · 2019-04-08T12:54:05Z

src/interface/meta.thrift

@@ -24,6 +24,9 @@ enum ErrorCode {
 E_NO_HOSTS = -21,
 E_SPACE_EXISTED = -22,
 E_NOT_FOUND = -23,
+ E_TABLE_LOCKED = -24,
+ E_USER_EXISTED = -25,
+ E_ROLE_ERROR = -26,


If here you meant the user does not have the proper role, I would suggest to name it E_INPROPER_ROLE

sherman-the-tank · 2019-04-08T12:59:33Z

src/interface/meta.thrift

@@ -181,6 +184,48 @@ struct GetPartsAllocResp {
 3: map<common.PartitionID, list<common.HostAddr>>(cpp.template = "std::unordered_map") parts,
 }

+// user management
+struct CreateUserReq {
+ 1: common.GraphSpaceID space_id,


@boshengchen Please refer the issue #190. A user does not have to be associated with any space, although in most cases, a user has a default space associated. A user could be associated with multiple spaces, but could only have one default space. A user could have different roles in different spaces

sherman-the-tank · 2019-04-08T13:18:09Z

src/interface/meta.thrift

+ 1: ErrorCode code,
+ 3: list<common.UserItem> users;
+}
+


We might want to have one more request for assigning users their default spaces

Thanks for your idea @sherman-the-tank . I was updated #190 , Let's discuss this via #190.

sherman-the-tank · 2019-04-08T13:20:03Z

src/meta/MetaServiceHandler.cpp

@@ -9,6 +9,7 @@
 #include "meta/processors/ListHostsProcessor.h"
 #include "meta/processors/ListSpacesProcessor.h"
 #include "meta/processors/GetPartsAllocProcessor.h"
+#include <meta/processors/AuthenticationProcessor.h>


Please do not use <> for our own header files

sherman-the-tank · 2019-04-08T13:28:41Z

src/meta/processors/AuthenticationProcessor.cpp

+namespace meta {
+
+void CreateUserProcessor::process(const cpp2::CreateUserReq& req) {
+ if (spaceExist(req.get_space_id()) == Status::SpaceNotFound()) {


So, we don't need lock here?

Have a read lock in spaceExist.

Moving the lock from spaceExist to the top of the method seems more reasonable, WDYT?

sherman-the-tank · 2019-04-08T13:42:36Z

src/meta/test/ProcessorTest.cpp

+ {
+ ASSERT_EQ(TestUtils::createUser(kv.get(), 1, true, "nebula", "nebula", RoleTypeE::T_ADMIN),
+ cpp2::ErrorCode::SUCCEEDED);
+ ASSERT_EQ(TestUtils::createUser(kv.get(), 1, true, "nebula", "nebula", RoleTypeE::T_ADMIN),


Please add a comment to help other people understand the logic

sherman-the-tank · 2019-04-08T13:47:08Z

src/meta/processors/BaseProcessor.h

@@ -26,6 +26,23 @@ enum IDType {
 EDGE,
 };

+class LockUtils {


Good idea here to have a lock factory class. 👍 I prefer this way rather than simply defining a lock in the base class.

As a general rule, I think any base class should not make its lock visible to the inherited classes, especially these inherited classes could be implemented by many other people

boshengchen · 2019-04-09T02:01:43Z

Thanks for your detailed review @ALL.
Please hold this PR. There are some details to be discussed yet. Let's discuss this in #190.

boshengchen · 2019-04-09T07:52:37Z

Thanks for taking care of this.

I think my biggest concern is the PR makes the user as an entity in a given space (the key/value is designed base on that assumption), but that's not the case (Please refer the issue #190). I think we need to redesign the way we store the user information

This storage structure is inappropriate，I‘ve redesigned it as below, Also comment it in #190 :

key:     __user__+userID   
value: username_size + username + encoded_pwd

key:     __acl__ + userID  + spaceID
value:  RoleType + isDefault

zlcook · 2019-04-09T10:21:40Z

src/meta/processors/AuthenticationProcessor.cpp

+}
+
+void DropUserProcessor::process(const cpp2::DropUserReq& req) {
+ LOCK_CHECK(user);


IMO, bool try_lock_shared_for(t)try to get reader permission on the lock. BUT, here should acquire an exclusive lock instead of shared. That is, SharedMutex::WriteHolder should be used.
@dutor What do you think?

Good finding 👍

I usually recommend to avoid dealing with locks in macro, but it hides (at least makes it not very clear) the detail and is error prone

boshengchen · 2019-05-17T08:02:56Z

Opened a new PR #400 instead of this one. Close this one.

user management in Meta Server

5259b71

boshengchen mentioned this pull request Apr 3, 2019

Support user management and authentication in Meta Server #180

Closed

dangleptr reviewed Apr 4, 2019

View reviewed changes

darionyaphet reviewed Apr 4, 2019

View reviewed changes

dutor reviewed Apr 8, 2019

View reviewed changes

sherman-the-tank requested changes Apr 8, 2019

View reviewed changes

zlcook reviewed Apr 9, 2019

View reviewed changes

boshengchen mentioned this pull request May 17, 2019

User manager in meta #400

Merged

boshengchen closed this May 17, 2019

user management in Meta Server #257

user management in Meta Server #257

Conversation

boshengchen commented Apr 3, 2019

nebula-community-bot commented Apr 3, 2019

dangleptr left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

boshengchen Apr 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sherman-the-tank commented Apr 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sherman-the-tank commented Apr 8, 2019

nebula-community-bot commented Apr 8, 2019

sherman-the-tank left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

boshengchen commented Apr 9, 2019

boshengchen commented Apr 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

boshengchen commented May 17, 2019

dangleptr left a comment •

edited

Loading

boshengchen Apr 4, 2019 •

edited

Loading