-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Background
Currently, FE's Thread is managed by ThreadPoolManager, but BE's thread manage is messy, some are PriorityThreadPool some a native pthread, It is inconvient to trace some problem such as thread num increase up to thounds and CPU load is very high.
Now, we introduce kudu's ThreadPool in #2915, and prepare to replace some native pthread to ThreadPool #3416 , and I think we should also introduce kudu's ThreadPoolMgr to manage ThreadPool, and collect metric such as user cpu time used, system cpu time used, context switches, escepcially for plan fragment executor's thread to help us positioning if the thread is doing a long task which was unexpected or they are just waiting by potential deadlock problems.
Suggest Solution
I have investigated kudu's ThreadPool, I think we can use Doris's metric to collect metrics to monitor process's staus and use system profile to monitor thread's status.Following is what we need to do:
- Use kudu's ThreadPool to replace current native thread.especially for plan fragment and it is in processing.
- Add a content in BE's web names
threadzto show all thread's status group by thread's group name. - implement functions to get thread's status by thread_id and add web path handle to use these functions to display thread's status.
This PR should after #4062 in order to render metrics in BE's page.