Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(query): grant object visibility check #14458

Merged
merged 5 commits into from
Feb 21, 2024

Conversation

TCeason
Copy link
Collaborator

@TCeason TCeason commented Jan 24, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Modify:

  1. all users can see informatio_schema and system.

  2. show databases/tables only display current role owned object. if set
    secondary role all, will display all effective roles owned object.

  3. add new privilege type : CreateDatabase. It's a global object.

  4. create udf and create index need super privilege.

  5. drop database need check Database level drop privilege

Note:

Now create database need create database privilege.

If you want to create a database, should excute

grant create database on *.* to <user_name>

Or

grant create database on *.* to role <role_name>;
grant role <role_name> to <user_name>;

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Jan 24, 2024
@TCeason TCeason marked this pull request as draft January 24, 2024 15:34
@TCeason
Copy link
Collaborator Author

TCeason commented Jan 24, 2024

cc @flaneur2020 wait auth check built in role

@TCeason TCeason force-pushed the rewrite_show branch 2 times, most recently from a89dbd9 to bd8263c Compare January 25, 2024 02:32
@TCeason TCeason changed the title feat(query): db visibility check feat(query): grant object visibility check Jan 25, 2024
@TCeason TCeason force-pushed the rewrite_show branch 3 times, most recently from f84e4cb to 28c0486 Compare January 26, 2024 04:03
@flaneur2020
Copy link
Member

flaneur2020 commented Jan 26, 2024

show databases/tables only display current role owned object. if set
secondary role all, will display all effective roles owned object.

IHMO, it's always showing databases/tables from effective roles.

@flaneur2020
Copy link
Member

cc @flaneur2020 wait auth check built in role

related PR: #14451

Copy link
Member

@flaneur2020 flaneur2020 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TCeason TCeason force-pushed the rewrite_show branch 2 times, most recently from 45ac3d1 to 0c40999 Compare January 27, 2024 12:35
@TCeason TCeason marked this pull request as ready for review January 27, 2024 12:35
@TCeason
Copy link
Collaborator Author

TCeason commented Jan 27, 2024

cc @BohuTANG This pr can be review. And old query normal user need do this grant after upgrade.

Note:

Now create database need create database privilege.

If you want to create a database, should excute

grant create database on *.* to <user_name>

Or

grant create database on *.* to role <role_name>;
grant role <role_name> to <user_name>;

@BohuTANG
Copy link
Member

Good.
What will be affected to the user who is already using Databend or DatabendCloud?

@BohuTANG BohuTANG added the ci-cloud Build docker image for cloud test label Jan 27, 2024
@TCeason
Copy link
Collaborator Author

TCeason commented Jan 27, 2024

Good.

What will be affected to the user who is already using Databend or DatabendCloud?

Create database will return err.

Need root user do this

Ref #14458 (comment)

Copy link
Contributor

Docker Image for PR

  • tag: pr-14458-5677281

note: this image tag is only available for internal use,
please check the internal doc for more details.

@BohuTANG BohuTANG marked this pull request as draft January 27, 2024 14:43
@BohuTANG BohuTANG requested a review from drmingdrmer January 27, 2024 14:48
@TCeason
Copy link
Collaborator Author

TCeason commented Jan 27, 2024

Summary

I test on my local with standalone mode. The same error was repeated.

Meta version v1.2.262

Query version : old-version(1.2.314) -> upgrade to pr version -> rollback to old-version(1.2.314).

How to produce

meta version: v1.2.262-37e04b9922-simd

query version: v1.2.314

MySQL [(none)]> create role test;
Query OK, 0 rows affected (0.033 sec)

MySQL [(none)]> grant all on *.* to role test;
Query OK, 0 rows affected (0.020 sec)

MySQL [(none)]> select version();
+----------------------------------------------------------------------------------------+
| version()                                                                              |
+----------------------------------------------------------------------------------------+
| 8.0.26-v1.2.314-nightly-3ccec470ea(rust-1.77.0-nightly-2024-01-27T16:50:23.223949439Z) |
+----------------------------------------------------------------------------------------+
1 row in set (0.012 sec)

query version: current-pr-commit

MySQL [(none)]> create user u1 identified by '123';
Query OK, 0 rows affected (0.026 sec)

MySQL [(none)]> grant all on *.* to u1;
Query OK, 0 rows affected (0.023 sec)

rollback to v1.2.314

rebuild query, and try to run it. failed:

    Finished dev [unoptimized + debuginfo] target(s) in 2m 35s
Databend Query start failure, cause: FromStdError. Code: 1001, Text = expected value at line 2 column 2.

   0: databend_common_exception::exception_backtrace::capture
             at src/common/exception/src/exception_backtrace.rs:72:17
   1: databend_common_exception::exception::ErrorCode::from_std_error
             at src/common/exception/src/exception.rs:277:24
   2: databend_common_exception::exception_into::<impl core::convert::From<serde_json::error::Error> for databend_common_exception::exception::ErrorCode>::from
             at src/common/exception/src/exception_into.rs:181:9
   3: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/result.rs:1959:27
   4: databend_common_management::serde::pb_serde::check_and_upgrade_to_pb::{{closure}}
             at src/query/management/src/serde/pb_serde.rs:87:16
   5: <databend_common_management::role::role_mgr::RoleMgr as databend_common_management::role::role_api::RoleApi>::get_role::{{closure}}::{{closure}}
             at src/query/management/src/role/role_mgr.rs:183:18
   6: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
   7: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
   8: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
   9: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  10: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  11: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  12: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  13: <databend_common_management::role::role_mgr::RoleMgr as databend_common_management::role::role_api::RoleApi>::get_role::{{closure}}
             at src/query/management/src/role/role_mgr.rs:166:5
  14: <minitrace::future::InSpan<T> as core::future::future::Future>::poll
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/minitrace-0.6.2/src/future.rs:125:19
  15: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/future/future.rs:124:9
  16: databend_common_users::role_mgr::<impl databend_common_users::user_api::UserApiProvider>::get_role::{{closure}}::{{closure}}
             at src/query/users/src/role_mgr.rs:38:65
  17: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  18: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  19: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  20: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  21: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  22: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  23: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  24: databend_common_users::role_mgr::<impl databend_common_users::user_api::UserApiProvider>::get_role::{{closure}}
             at src/query/users/src/role_mgr.rs:35:5
  25: databend_common_users::role_mgr::<impl databend_common_users::user_api::UserApiProvider>::exists_role::{{closure}}::{{closure}}
             at src/query/users/src/role_mgr.rs:72:43
  26: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  27: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  28: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  29: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  30: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  31: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  32: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  33: databend_common_users::role_mgr::<impl databend_common_users::user_api::UserApiProvider>::exists_role::{{closure}}
             at src/query/users/src/role_mgr.rs:70:5
  34: databend_common_users::role_mgr::<impl databend_common_users::user_api::UserApiProvider>::add_role::{{closure}}::{{closure}}
             at src/query/users/src/role_mgr.rs:92:78
  35: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  36: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  37: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  38: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  39: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  40: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  41: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  42: databend_common_users::role_mgr::<impl databend_common_users::user_api::UserApiProvider>::add_role::{{closure}}
             at src/query/users/src/role_mgr.rs:85:5
  43: databend_common_users::user_api::UserApiProvider::try_create::{{closure}}::{{closure}}
             at src/query/users/src/user_api.rs:105:60
  44: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  45: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  46: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  47: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  48: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  49: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  50: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  51: databend_common_users::user_api::UserApiProvider::try_create::{{closure}}
             at src/query/users/src/user_api.rs:80:5
  52: databend_common_users::user_api::UserApiProvider::init::{{closure}}::{{closure}}
             at src/query/users/src/user_api.rs:70:72
  53: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  54: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  55: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  56: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  57: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  58: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  59: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  60: databend_common_users::user_api::UserApiProvider::init::{{closure}}
             at src/query/users/src/user_api.rs:63:5
  61: databend_query::global_services::GlobalServices::init_with::{{closure}}::{{closure}}
             at src/query/service/src/global_services.rs:110:10
  62: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  63: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  64: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  65: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  66: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  67: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  68: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  69: databend_query::global_services::GlobalServices::init_with::{{closure}}
             at src/query/service/src/global_services.rs:56:5
  70: databend_query::global_services::GlobalServices::init::{{closure}}::{{closure}}
             at src/query/service/src/global_services.rs:53:43
  71: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  72: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  73: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  74: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  75: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  76: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  77: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  78: databend_query::global_services::GlobalServices::init::{{closure}}
             at src/query/service/src/global_services.rs:50:5
  79: databend_query::entry::init_services::{{closure}}
             at src/binaries/query/entry.rs:81:32
  80: databend_query::main_entrypoint::{{closure}}
             at src/binaries/query/ee_main.rs:54:26
  81: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:27
  82: async_backtrace::frame::Frame::in_scope::{{closure}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:184:17
  83: std::thread::local::LocalKey<T>::try_with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:286:16
  84: std::thread::local::LocalKey<T>::with
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/thread/local.rs:262:9
  85: async_backtrace::frame::active_frame::with
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:81:9
  86: async_backtrace::frame::Frame::in_scope
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/frame.rs:180:13
  87: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:49:9
  88: <async_backtrace::framed::Framed<F> as core::future::future::Future>::poll{{reify.shim}}
             at /home/eason/.cargo/git/checkouts/async-backtrace-dc8b2c67d0719549/e7e1b5f/backtrace/src/framed.rs:45:5
  89: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/future/future.rs:124:9
  90: futures_util::future::future::FutureExt::poll_unpin
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.28/src/future/future/mod.rs:562:9
  91: <databend_common_base::runtime::catch_unwind::CatchUnwindFuture<F> as core::future::future::Future>::poll::{{closure}}
             at src/common/base/src/runtime/catch_unwind.rs:55:36
  92: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/panic/unwind_safe.rs:272:9
  93: std::panicking::try::do_call
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:554:40
  94: __rust_try
  95: std::panicking::try
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:518:19
  96: std::panic::catch_unwind
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panic.rs:142:14
  97: databend_common_base::runtime::catch_unwind::catch_unwind
             at src/common/base/src/runtime/catch_unwind.rs:27:11
  98: <databend_common_base::runtime::catch_unwind::CatchUnwindFuture<F> as core::future::future::Future>::poll
             at src/common/base/src/runtime/catch_unwind.rs:55:15
  99: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:63
 100: tokio::runtime::coop::with_budget
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:107:5
      tokio::runtime::coop::budget
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:73:5
      tokio::runtime::park::CachedParkThread::block_on
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:31
 101: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/blocking.rs:66:9
 102: tokio::runtime::handle::Handle::block_on::{{closure}}
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/handle.rs:310:13
 103: tokio::runtime::context::runtime::enter_runtime
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/runtime.rs:65:16
 104: tokio::runtime::handle::Handle::block_on
             at /home/eason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/handle.rs:309:9
 105: databend_common_base::runtime::runtime::Runtime::block_on
             at src/common/base/src/runtime/runtime.rs:207:9
 106: databend_query::main
             at src/binaries/query/ee_main.rs:40:33
 107: core::ops::function::FnOnce::call_once
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/ops/function.rs:250:5
 108: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/sys_common/backtrace.rs:155:18
 109: std::rt::lang_start::{{closure}}
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/rt.rs:166:18
 110: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/ops/function.rs:284:13
      std::panicking::try::do_call
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:554:40
      std::panicking::try
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:518:19
      std::panic::catch_unwind
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:554:40
      std::panicking::try
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:518:19
      std::panic::catch_unwind
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/rt.rs:148:20
 111: std::rt::lang_start
             at /rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/rt.rs:165:17
 112: main
 113: <unknown>
 114: __libc_start_main
 115: _start

Analyze

debug log in pb :

{"timestamp":"2024-01-27T17:16:53.496646Z","level":"DEBUG","fields":{"message":"deserialize as pb err: IllegalUserInfoFormat. Code: 2203, Text = , cause: Incompatible: UserPrivilegeType error: Invalid bits for BitFlags<UserPrivilegeType>(0b11011101111111101111, Usage | Create | Select | Insert | Update | Delete | Drop | Alter | Super | CreateUser | CreateRole | Grant | DropRole | DropUser | CreateDataMask | Read | Write): 0b100000000000000000000., rollback to use serde json"}}

And this err is serde json err

Databend Query start failure, cause: FromStdError. Code: 1001, Text = expected value at line 2 column 2.

in there, https://github.com/datafuselabs/databend/blob/f56988bad93eb0a2efee608241f0fd92343da855/src/query/management/src/serde/pb_serde.rs#L84

if deserialize struct err, will degrade to serje json.

So we should focus on the pb deserialize err.

It looks like an incompatible design here.

https://github.com/datafuselabs/databend/blob/2429b56fd2a2d153e44a0daa61b90b3524a733d9/src/meta/proto-conv/src/user_from_to_protobuf_impl.rs#L244

At this point, this pr contains the value CreateDatabase, which was not recognized by older versions. So the judgment fails.

@BohuTANG
Copy link
Member

if deserialize struct err, will degrade to serje json.

So we should focus on the pb deserialize err.

It looks like an incompatible design here.

https://github.com/datafuselabs/databend/blob/2429b56fd2a2d153e44a0daa61b90b3524a733d9/src/meta/proto-conv/src/user_from_to_protobuf_impl.rs#L244

At this point, this pr contains the value CreateDatabase, which was not recognized by older versions. So the judgment fails.

Which PR and where is this incompatibility triggered?

@TCeason TCeason force-pushed the rewrite_show branch 2 times, most recently from 3f05049 to 0503f85 Compare February 19, 2024 03:02
@TCeason TCeason added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Feb 19, 2024
@TCeason
Copy link
Collaborator Author

TCeason commented Feb 19, 2024

In pr #14501, we modify deserialize bitflag way. Now use from_bits_truncate.

So if this pr revert to before version https://github.com/datafuselabs/databend/releases/tag/v1.2.321-nightly, may cause a compatibility problem

@TCeason TCeason force-pushed the rewrite_show branch 2 times, most recently from 2eebf14 to 2837ec7 Compare February 19, 2024 07:47
Modify:

1. all users can see informatio_schema and system.

2, show databases/tables only display current role owned object. if set
secondary role all, will display all effective roles owned object.

3. add new privilege type : CreateDatabase. It's a global object.

4. create udf and create index need super privilege.

Note:

Now create database need create database privilege.

If you want to create a database, should excute `grant create database
on *.* to <user_name> or

```sql
grant create database on *.* to role <role_name>;
grant role <role_name> to <user_name>;
```
@TCeason TCeason marked this pull request as ready for review February 19, 2024 11:16
@databendlabs databendlabs deleted a comment from github-actions bot Feb 19, 2024
@TCeason TCeason removed the ci-cloud Build docker image for cloud test label Feb 19, 2024
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 6 files at r3, 1 of 6 files at r4, all commit messages.
Reviewable status: 3 of 14 files reviewed, 2 unresolved discussions (waiting on @flaneur2020, @JackTan25, and @TCeason)


src/meta/app/src/principal/user_privilege.rs line 77 at r4 (raw file):

    Write = 1 << 19,
    // Privilege to Create database
    CreateDatabase = 1 << 20,

Is this new field compatible with old version of databend-query? i.e., what if an old query reads this value 1<<20 but it does not have this type CreateDatabase?

@TCeason
Copy link
Collaborator Author

TCeason commented Feb 20, 2024

Reviewed 2 of 6 files at r3, 1 of 6 files at r4, all commit messages.
Reviewable status: 3 of 14 files reviewed, 2 unresolved discussions (waiting on @flaneur2020, @JackTan25, and @TCeason)


src/meta/app/src/principal/user_privilege.rs line 77 at r4 (raw file):

    Write = 1 << 19,
    // Privilege to Create database
    CreateDatabase = 1 << 20,

Is this new field compatible with old version of databend-query? i.e., what if an old query reads this value 1<<20 but it does not have this type CreateDatabase?

after this pr #14458 (comment) will ignore the unknown bit.

Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 13 files at r1, 1 of 4 files at r2, 2 of 6 files at r3, 5 of 6 files at r4.
Reviewable status: 13 of 14 files reviewed, 2 unresolved discussions (waiting on @flaneur2020 and @JackTan25)

@TCeason
Copy link
Collaborator Author

TCeason commented Feb 21, 2024

I test this pr on test-cloud env. And now ci has forward compat test already passed. Shell we merge this pr? cc @BohuTANG
@flaneur2020

show warehouses;
│ pc-test_priv    │ Suspended │ XSmall │                  │          600NULL2024-02-19 06:31:50 │
│ tai-test_priv   │ Suspended │ XSmall │ pr-14458-8283a6f │          600NULL2024-02-19 06:49:35

@BohuTANG BohuTANG merged commit cb07067 into databendlabs:main Feb 21, 2024
71 of 72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: grant object visibilty check
4 participants