Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for Adaptive #63

Merged
merged 70 commits into from
Jul 16, 2018
Merged
Changes from 1 commit
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
7c56b1d
add job ids to dask workers
May 18, 2018
6b35688
pad job id
May 18, 2018
507be82
parse PBS/slurm job ids
May 18, 2018
99d0f1f
track workers individually (sort of)
May 21, 2018
78a22ff
add _adaptive_options
May 21, 2018
62e050c
generalize the parsing of the job id
May 22, 2018
d121180
fix typo
May 22, 2018
2329bfe
changes for review
May 22, 2018
92eaf4e
add pluggin (untested)
May 24, 2018
9084a35
a few fixes + tests
May 24, 2018
4776892
respond to first round of comments
May 29, 2018
ef62f59
fix list addition
May 29, 2018
a4e007a
mark test modules (again)
May 29, 2018
c19e4da
fixes while testing on pbs
May 29, 2018
5d5fd85
remove extra if block
May 29, 2018
115b0c1
use for/else
May 29, 2018
cde3ca4
fix two failing tests
May 30, 2018
75f2c6a
Merge branch 'jobids' of github.com:jhamman/dask-jobqueue into jobids
May 30, 2018
66db52d
respond to review comments
Jun 3, 2018
ea7d56d
fix bug in scale down
Jun 4, 2018
a6d31d2
fix marks
Jun 4, 2018
25965c0
Merge branch 'master' of github.com:dask/dask-jobqueue into jobids
Jun 15, 2018
ab4363a
more docs
Jun 21, 2018
604a563
debugging ci
Jun 21, 2018
1e0455e
more ci
Jun 21, 2018
ace37ad
only stop jobs if there are jobs to stop
Jun 21, 2018
56e2990
refactor remove_worker method
Jun 21, 2018
914244c
debug
Jun 21, 2018
359be59
print debug info
Jun 21, 2018
1441634
longer waits in tests
Jun 21, 2018
0bf53d1
refactor use of jobids, add scheduler plugin
May 18, 2018
cc2628f
debug stop workers
Jun 26, 2018
90dd730
fix tests of stop_workers
Jun 26, 2018
9fe2178
Merge branch 'master' into jobids
mrocklin Jun 26, 2018
18dfe31
cleanup
Jun 26, 2018
d98b141
Merge branch 'master' of github.com:dask/dask-jobqueue into jobids
Jun 26, 2018
b303275
more flexible stop workers
Jun 26, 2018
13e5dc3
Merge branch 'jobids' of github.com:jhamman/dask-jobqueue into jobids
Jun 26, 2018
b4877ad
Merge remote-tracking branch 'origin/jobids' into jobids
Jun 26, 2018
667369e
remove Job class
Jun 27, 2018
bf99d29
Merge branch 'master' of github.com:dask/dask-jobqueue into jobids
Jun 27, 2018
292b595
fix for worker name (again)
Jun 27, 2018
627f873
debug
Jun 27, 2018
f2b2a92
perform a backflip
Jun 27, 2018
1f0dc71
Merge branch 'jobids' of github.com:jhamman/dask-jobqueue into jobids
Jun 27, 2018
a1b102d
Merge branch 'master' of github.com:dask/dask-jobqueue into jobids
Jul 3, 2018
c988f1e
isort after merge conflict resolution
Jul 3, 2018
619047f
more docs stuff
Jul 3, 2018
8db65eb
update worker name template to use bracket delims
Jul 3, 2018
ef16298
add a few more comments
Jul 3, 2018
3803918
roll back changes in tests
Jul 3, 2018
aad58d4
fix slurm tests and missing scheduler plugin after merge conflict res…
Jul 3, 2018
fa1b717
fix threads in test
Jul 3, 2018
aeea2e5
debug on travis
Jul 6, 2018
b93a7c4
simplify tests
Jul 6, 2018
0a2e304
more debugging and more robust sleeps
Jul 7, 2018
1a1fe75
unify basic test
Jul 7, 2018
8c32872
unify adaptive tests
Jul 7, 2018
9c43f43
Merge remote-tracking branch 'origin/jobids' into jobids
Jul 9, 2018
a02abc8
Merge branch 'master' of github.com:dask/dask-jobqueue into jobids
Jul 9, 2018
ee89f20
debug statements and some nice fixups
Jul 9, 2018
4abcea1
future div
Jul 13, 2018
0c7425a
add logging stuff
Jul 13, 2018
5d4552d
-s for pytest
Jul 13, 2018
8a150c9
use --job_id-- for name
mrocklin Jul 13, 2018
ca0c727
fix memory in sge tests
mrocklin Jul 13, 2018
ce007df
remove pending jobs when scaling down
Jul 14, 2018
c23ce7c
remove pending jobs
Jul 14, 2018
7618467
cleanup after lots of debugging
Jul 15, 2018
d5e42b3
additional cleanup
Jul 16, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
debugging ci
Joseph Hamman committed Jun 21, 2018

Verified

This commit was signed with the committer’s verified signature. The key has expired.
makkus Markus Binsteiner
commit 604a5636349dcbbc92247dd85defc50c8fb98f63
1 change: 1 addition & 0 deletions dask_jobqueue/tests/test_sge.py
Original file line number Diff line number Diff line change
@@ -11,6 +11,7 @@
def test_basic(loop): # noqa: F811
with SGECluster(walltime='00:02:00', threads=2, memory='7GB',
loop=loop) as cluster:
print(cluster.job_script())
with Client(cluster, loop=loop) as client:
workers = cluster.start_workers(2)
future = client.submit(lambda x: x + 1, 10)
1 change: 1 addition & 0 deletions dask_jobqueue/tests/test_slurm.py
Original file line number Diff line number Diff line change
@@ -48,6 +48,7 @@ def test_job_script():
memory='7GB') as cluster:

job_script = cluster.job_script()
print(job_script)
assert '#SBATCH' in job_script
assert '#SBATCH -J dask-worker' in job_script
assert '#SBATCH -n 1' in job_script