Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][metrics] Add master, worker metrics #10326

Merged
merged 12 commits into from
Jun 9, 2022

Conversation

ruanwenjun
Copy link
Member

Purpose of the pull request

related issue #9324

Brief change log

Add basic metrics about master server.

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

@ruanwenjun ruanwenjun marked this pull request as draft June 1, 2022 08:58
@ruanwenjun ruanwenjun added the feature new feature label Jun 1, 2022
@ruanwenjun ruanwenjun changed the title [Feature][master] Add mater metrics [Feature][master] Add master metrics Jun 1, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jun 1, 2022

Codecov Report

Merging #10326 (d2ee18a) into dev (ffe809b) will decrease coverage by 0.11%.
The diff coverage is 16.11%.

@@             Coverage Diff              @@
##                dev   #10326      +/-   ##
============================================
- Coverage     40.56%   40.45%   -0.12%     
- Complexity     4739     4740       +1     
============================================
  Files           869      874       +5     
  Lines         35273    35465     +192     
  Branches       3935     3938       +3     
============================================
+ Hits          14309    14347      +38     
- Misses        19534    19689     +155     
+ Partials       1430     1429       -1     
Impacted Files Coverage Δ
.../org/apache/dolphinscheduler/common/Constants.java 80.95% <ø> (ø)
...apache/dolphinscheduler/common/model/TaskNode.java 25.20% <ø> (ø)
...ache/impl/ProcessInstanceExecCacheManagerImpl.java 84.61% <0.00%> (-15.39%) ⬇️
...ler/server/master/metrics/MasterServerMetrics.java 0.00% <0.00%> (ø)
.../server/master/metrics/ProcessInstanceMetrics.java 0.00% <0.00%> (ø)
...r/server/master/runner/MasterSchedulerService.java 0.00% <0.00%> (ø)
...erver/master/runner/WorkflowExecuteThreadPool.java 1.40% <0.00%> (-0.07%) ⬇️
...server/master/runner/task/CommonTaskProcessor.java 4.10% <0.00%> (ø)
...er/remote/processor/StateEventCallbackService.java 0.00% <ø> (ø)
...inscheduler/server/worker/metrics/TaskMetrics.java 0.00% <0.00%> (ø)
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffe809b...d2ee18a. Read the comment docs.

@ruanwenjun ruanwenjun changed the title [Feature][master] Add master metrics [Feature][master] Add master, worker metrics Jun 6, 2022
@ruanwenjun ruanwenjun changed the title [Feature][master] Add master, worker metrics [Feature][metrics] Add master, worker metrics Jun 6, 2022
@ruanwenjun ruanwenjun marked this pull request as ready for review June 6, 2022 05:52
@ruanwenjun
Copy link
Member Author

@EricGao888 Please take a look.

Copy link
Member

@kezhenxu94 kezhenxu94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice metrics improvements, I left some comments

docs/docs/en/guide/metrics/metrics.md Outdated Show resolved Hide resolved
docs/docs/en/guide/metrics/metrics.md Show resolved Hide resolved
docs/docs/en/guide/metrics/metrics.md Outdated Show resolved Hide resolved
docs/docs/en/guide/metrics/metrics.md Show resolved Hide resolved
Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some not important things

docs/docs/en/guide/metrics/metrics.md Show resolved Hide resolved
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_addMetrics branch 2 times, most recently from fd5f206 to 992f4fa Compare June 8, 2022 11:53
@sonarcloud
Copy link

sonarcloud bot commented Jun 9, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 10 Code Smells

31.7% 31.7% Coverage
0.0% 0.0% Duplication

Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@caishunfeng caishunfeng merged commit e21d7b1 into apache:dev Jun 9, 2022
@ruanwenjun ruanwenjun deleted the dev_wenjun_addMetrics branch June 9, 2022 05:38
ITBOX-ITBOY pushed a commit to ITBOX-ITBOY/dolphinscheduler that referenced this pull request Jul 8, 2022
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT
@ruanwenjun ruanwenjun added this to the 3.0.0-release milestone Jul 14, 2022
ruanwenjun added a commit that referenced this pull request Jul 19, 2022
* Add mater metrics

* fix UT

* Add url to mysql profile

* Add worker metrics

* Update grafana config

* Add system metrics doc

* Add process failover counter

* Add metrics image

* Change jpg to png

* Add command insert metrics

* Fix UT

* Revert UT

(cherry picked from commit e21d7b1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants