Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

job use future #4654

Merged
merged 5 commits into from
Sep 26, 2022
Merged

job use future #4654

merged 5 commits into from
Sep 26, 2022

Conversation

panda-sheep
Copy link
Contributor

fix comments

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

Use future to make meta jobs of different spaces execute in parallel
note:
Because the balance leader is for all spaces, it will not be dealt with here.
Same as https://github.com/vesoft-inc/nebula-ent/pull/1259
Also resolved issue https://github.com/vesoft-inc/nebula-ent/issues/867

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

SuperYoko
SuperYoko previously approved these changes Sep 19, 2022
@kikimo
Copy link
Contributor

kikimo commented Sep 20, 2022

cannot stop a failed job?

(root@nebula) [test]> show jobs;
+--------+------------------+------------+----------------------------+----------------------------+
| Job Id | Command          | Status     | Start Time                 | Stop Time                  |
+--------+------------------+------------+----------------------------+----------------------------+
| 14     | "ZONE_BALANCE"   | "FAILED"   | 2022-09-19T11:12:54.000000 | 2022-09-19T11:13:25.000000 |
| 13     | "ZONE_BALANCE"   | "FAILED"   | 2022-09-19T11:11:49.000000 | 2022-09-19T11:12:54.000000 |
| 12     | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T09:29:16.000000 | 2022-09-19T09:29:21.000000 |
| 11     | "ZONE_BALANCE"   | "FINISHED" | 2022-09-19T09:29:21.000000 | 2022-09-19T11:11:49.000000 |
| 10     | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:15:59.000000 | 2022-09-19T08:15:59.000000 |
| 9      | "ZONE_BALANCE"   | "FAILED"   | 2022-09-19T08:15:59.000000 | 2022-09-19T09:29:16.000000 |
| 8      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:14:18.000000 | 2022-09-19T08:14:23.000000 |
| 7      | "ZONE_BALANCE"   | "FINISHED" | 2022-09-19T08:14:59.000000 | 2022-09-19T08:15:59.000000 |
| 6      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:07:32.000000 | 2022-09-19T08:07:37.000000 |
| 5      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:05:26.000000 | 2022-09-19T08:05:31.000000 |
| 4      | "ZONE_BALANCE"   | "FINISHED" | 2022-09-19T08:05:31.000000 | 2022-09-19T08:07:32.000000 |
| 3      | "LEADER_BALANCE" | "FINISHED" | 2022-09-19T08:04:04.000000 | 2022-09-19T08:04:09.000000 |
| 2      | "ZONE_BALANCE"   | "STOPPED"  | 2022-09-19T08:07:37.000000 | 2022-09-19T08:10:26.000000 |
+--------+------------------+------------+----------------------------+----------------------------+
Got 13 rows (time spent 2745/3430 us)

Tue, 20 Sep 2022 10:11:12 CST

(root@nebula) [test]> stop job 14;
[ERROR (-1005)]: Finished job or failed job can not be stopped, please start another job instead

Tue, 20 Sep 2022 10:11:17 CST

We will not be able to start a new balance job if a failed job cannnot be stopped.

@panda-sheep
Copy link
Contributor Author

cannot stop a failed job?

When I tested it in the enterprise version, it was OK.
Maybe a pr is not synced to the community version.

I take a look. thx

@panda-sheep
Copy link
Contributor Author

panda-sheep commented Sep 20, 2022

@kikimo https://github.com/vesoft-inc/nebula-ent/pull/861. this pr is not synced to the community version.

@codecov-commenter
Copy link

Codecov Report

Base: 84.67% // Head: 84.70% // Increases project coverage by +0.02% 🎉

Coverage data is based on head (11379f5) compared to base (7d5e93f).
Patch coverage: 92.73% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4654      +/-   ##
==========================================
+ Coverage   84.67%   84.70%   +0.02%     
==========================================
  Files        1358     1358              
  Lines      135654   135694      +40     
==========================================
+ Hits       114868   114934      +66     
+ Misses      20786    20760      -26     
Impacted Files Coverage Δ
src/meta/processors/job/BalanceJobExecutor.h 75.00% <ø> (ø)
src/meta/processors/job/BalancePlan.h 100.00% <ø> (ø)
src/meta/processors/job/JobExecutor.h 100.00% <ø> (+50.00%) ⬆️
src/meta/processors/job/JobManager.h 100.00% <ø> (ø)
src/meta/processors/job/MetaJobExecutor.h 83.33% <81.81%> (-16.67%) ⬇️
src/meta/processors/job/JobManager.cpp 75.64% <84.88%> (+1.12%) ⬆️
...c/meta/processors/job/LeaderBalanceJobExecutor.cpp 86.90% <85.71%> (-0.08%) ⬇️
src/meta/processors/job/ZoneBalanceJobExecutor.cpp 75.00% <90.90%> (+0.22%) ⬆️
src/meta/processors/job/BalanceJobExecutor.cpp 69.67% <100.00%> (+3.00%) ⬆️
src/meta/processors/job/BalancePlan.cpp 72.53% <100.00%> (-3.13%) ⬇️
... and 55 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@liwenhui-soul
Copy link
Contributor

generally LGTM

liwenhui-soul
liwenhui-soul previously approved these changes Sep 21, 2022
kikimo
kikimo previously approved these changes Sep 21, 2022
Copy link
Contributor

@kikimo kikimo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verified, LGTM

@panda-sheep
Copy link
Contributor Author

Don't need merge first, wait for @critical27 to take a look

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job~

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done

Copy link
Contributor

@kikimo kikimo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@critical27 critical27 merged commit 5611ab7 into vesoft-inc:master Sep 26, 2022
@panda-sheep panda-sheep deleted the job_use_future branch September 26, 2022 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants