Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle more job status on SGE system #491

Merged
merged 63 commits into from
Sep 3, 2024
Merged

handle more job status on SGE system #491

Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
11fe336
revise SGE
thangckt Mar 26, 2024
8140a43
Update pbs.py
thangckt Mar 26, 2024
b893544
Update pbs.py
thangckt Mar 26, 2024
82b7262
Update pbs.py
thangckt Mar 26, 2024
01ffd14
add sge_qe_name
thangckt Mar 26, 2024
56e3afe
Update pbs.py
thangckt Mar 26, 2024
703aa49
y
thangckt Mar 26, 2024
2c3b38a
Update submission.py
thangckt Mar 26, 2024
f73e0f9
u
thangckt Mar 26, 2024
957cf4a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 26, 2024
034c9d7
Update pbs.py
thangckt Mar 26, 2024
3f07742
Update pbs.py
thangckt Mar 26, 2024
499c691
Merge branch 'PR' into master
thangckt Mar 26, 2024
76150bf
Merge pull request #1 from thangckt/master
thangckt Mar 26, 2024
ef28a1b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 26, 2024
65dca91
u
thangckt Mar 27, 2024
ffe52e1
Merge pull request #2 from thangckt/master
thangckt Mar 27, 2024
8d7b566
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
8c50c4f
u
thangckt Mar 27, 2024
89d2a41
Merge branch 'PR' into master
thangckt Mar 27, 2024
2093336
Merge pull request #3 from thangckt/master
thangckt Mar 27, 2024
920984d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
c8f5657
y
thangckt Mar 27, 2024
ef6ebeb
Merge branch 'master' of https://github.com/thangckt/dpdispatcher
thangckt Mar 27, 2024
f56fa50
Merge branch 'PR' into master
thangckt Mar 27, 2024
e06c648
Merge pull request #4 from thangckt/master
thangckt Mar 27, 2024
6d6c973
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
dae7909
Update pbs.py
thangckt Mar 27, 2024
4ca179c
Merge pull request #5 from thangckt/master
thangckt Mar 27, 2024
83644c3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 27, 2024
b1758d9
Create _version.py
thangckt Apr 4, 2024
62a3459
Delete dpdispatcher/_version.py
thangckt Apr 4, 2024
774b130
Merge pull request #6 from deepmodeling/master
thangckt Apr 13, 2024
2ae2e62
Update pbs.py
thangckt Apr 13, 2024
ad6b971
Merge branch 'master' of https://github.com/deepmodeling/dpdispatcher…
thangckt May 2, 2024
d02adf6
Merge pull request #8 from deepmodeling/master
thangckt May 7, 2024
3aca952
Merge pull request #9 from deepmodeling/master
thangckt May 19, 2024
9fc0f84
Merge branch 'master' into PR
thangckt May 28, 2024
95b8aa5
Merge pull request #10 from deepmodeling/master
thangckt May 28, 2024
cc7efcd
Merge pull request #11 from thangckt/master
thangckt May 28, 2024
99589d3
u
thangckt May 29, 2024
711da1d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 29, 2024
1e0c21e
Update test_lsf_script_generation.py
thangckt May 29, 2024
10cf5d3
Merge branch 'PR' of https://github.com/thangckt/dpdispatcher into PR
thangckt May 29, 2024
416a767
Merge pull request #12 from thangckt/PR
thangckt May 29, 2024
e530cf6
Update test_lsf_script_generation.py
thangckt May 29, 2024
5adce1b
Merge pull request #13 from deepmodeling/master
thangckt Jul 24, 2024
4084a31
Update pbs.py
thangckt Jul 24, 2024
9b1c566
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2024
fc690ae
change `sge_pe_name` to `pe_name`
thangckt Jul 25, 2024
cfbc5dd
Update ssh_context.py
thangckt Jul 25, 2024
81f52a3
Merge branch 'master' into master
thangckt Jul 25, 2024
7bbc485
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 25, 2024
37da47c
u
thangckt Jul 25, 2024
bca9728
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 25, 2024
02f1449
Merge pull request #14 from deepmodeling/master
thangckt Jul 25, 2024
c13be3b
Update SSHcontext: add execute command
thangckt Jul 27, 2024
a9fbadf
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 27, 2024
fba9574
Update pbs.py
thangckt Jul 29, 2024
0d2cd71
Merge remote-tracking branch 'upstream/master' into dev
thangckt Aug 27, 2024
5f42bc8
Update dlog.py
thangckt Sep 2, 2024
bfe1615
Update dlog.py
thangckt Sep 2, 2024
1ded55e
Merge pull request #16 from deepmodeling/master
thangckt Sep 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions dpdispatcher/machines/pbs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,7 @@
pass

def check_status(self, job):
### https://softpanorama.org/HPC/Grid_engine/Queues/queue_states.shtml
job_id = job.job_id
status_line = None
if job_id == "":
Expand Down Expand Up @@ -294,10 +295,12 @@
else:
status_word = status_line.split()[4]
# dlog.info (status_word)
if status_word in ["qw"]:
if status_word in ["qw", "hqw", "t"]:

Check warning on line 298 in dpdispatcher/machines/pbs.py

View check run for this annotation

Codecov / codecov/patch

dpdispatcher/machines/pbs.py#L298 

Added line #L298 was not covered by tests
return JobStatus.waiting
elif status_word in ["r"]:
elif status_word in ["r", "Rr"]:

Check warning on line 300 in dpdispatcher/machines/pbs.py

View check run for this annotation

Codecov / codecov/patch

dpdispatcher/machines/pbs.py#L300 

Added line #L300 was not covered by tests
return JobStatus.running
elif status_word in ["Eqw", "dr", "dt"]:
return JobStatus.terminated

Check warning on line 303 in dpdispatcher/machines/pbs.py

View check run for this annotation

Codecov / codecov/patch

dpdispatcher/machines/pbs.py#L302-L303 

Added lines #L302 - L303 were not covered by tests
Comment on lines +298 to +303
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider refactoring status checks for better maintainability.

The current structure of multiple if-elif statements might become hard to maintain if more status words are added in the future. Consider refactoring this part of the code to use a dictionary for status mapping, which would make it more maintainable and easier to extend.

Here's a suggested refactoring:

STATUS_MAPPING = {
    'qw': JobStatus.waiting,
    'hqw': JobStatus.waiting,
    't': JobStatus.waiting,
    'r': JobStatus.running,
    'Rr': JobStatus.running,
    'Eqw': JobStatus.terminated,
    'dr': JobStatus.terminated,
    'dt': JobStatus.terminated,
}

def check_status(self, job):
    # ... (previous code remains the same)
    
    status_word = status_line.split()[4]
    return STATUS_MAPPING.get(status_word, JobStatus.unknown)

This approach centralizes the status mapping, making it easier to add or modify statuses in the future without changing the method's logic.

Tools
GitHub Check: codecov/patch

[warning] 298-298: dpdispatcher/machines/pbs.py#L298
Added line #L298 was not covered by tests

[warning] 300-300: dpdispatcher/machines/pbs.py#L300
Added line #L300 was not covered by tests

[warning] 302-303: dpdispatcher/machines/pbs.py#L302-L303
Added lines #L302 - L303 were not covered by tests

else:
return JobStatus.unknown

Expand Down
Loading