Skip to content

[Proposal]Use kudu's ThreadPoolMgr && add metrics for BE's thread in BE's WebSite #4085

@WingsGo

Description

@WingsGo

Background

Currently, FE's Thread is managed by ThreadPoolManager, but BE's thread manage is messy, some are PriorityThreadPool some a native pthread, It is inconvient to trace some problem such as thread num increase up to thounds and CPU load is very high.
Now, we introduce kudu's ThreadPool in #2915, and prepare to replace some native pthread to ThreadPool #3416 , and I think we should also introduce kudu's ThreadPoolMgr to manage ThreadPool, and collect metric such as user cpu time used, system cpu time used, context switches, escepcially for plan fragment executor's thread to help us positioning if the thread is doing a long task which was unexpected or they are just waiting by potential deadlock problems.

Suggest Solution

I have investigated kudu's ThreadPool, I think we can use Doris's metric to collect metrics to monitor process's staus and use system profile to monitor thread's status.Following is what we need to do:

  1. Use kudu's ThreadPool to replace current native thread.especially for plan fragment and it is in processing.
  2. Add a content in BE's web names threadz to show all thread's status group by thread's group name.
  3. implement functions to get thread's status by thread_id and add web path handle to use these functions to display thread's status.

This PR should after #4062 in order to render metrics in BE's page.

Metadata

Metadata

Assignees

No one assigned

    Labels

    StaleproposalCategorizes an issue is a proposal

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions