Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Distributed] all_reduce op and distributed info in graphs #284

Merged
merged 42 commits into from
Jun 29, 2023
Merged
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
343b71d
init
soodoshll Jun 19, 2023
29ef7f5
op
soodoshll Jun 19, 2023
8e06560
update
soodoshll Jun 19, 2023
c8559d1
graph
soodoshll Jun 19, 2023
d97a7f8
update
soodoshll Jun 19, 2023
70f3a91
format
soodoshll Jun 19, 2023
6819ab8
add distributed graph
soodoshll Jun 19, 2023
95cec0c
update
soodoshll Jun 21, 2023
961e99b
support split
soodoshll Jun 22, 2023
334a1eb
update
soodoshll Jun 22, 2023
7dd55c4
update
soodoshll Jun 22, 2023
0c57cff
relaunch test
soodoshll Jun 22, 2023
7b81b0d
Merge branch 'main' of github.com:hidet-org/hidet into nccl-op
soodoshll Jun 22, 2023
047ea87
update
soodoshll Jun 22, 2023
dba85a5
fix
soodoshll Jun 22, 2023
2c6e5b1
format
soodoshll Jun 22, 2023
5d51ed4
fix
soodoshll Jun 22, 2023
f4bf865
[Document] fix installation guide (#288)
soodoshll Jun 22, 2023
64b9f03
[Runtime] Check for input tensor device (#287)
hjjq Jun 22, 2023
57ae2a9
fix
soodoshll Jun 23, 2023
a3d0a71
fix
soodoshll Jun 23, 2023
2ffcfe3
fix
soodoshll Jun 23, 2023
ee60249
update
soodoshll Jun 23, 2023
f3aad89
[FixBug] Don't instantiate symbol for primitive functions (#291)
hjjq Jun 26, 2023
64a632a
file store
soodoshll Jun 27, 2023
c028827
file store
soodoshll Jun 27, 2023
f118fd9
Merge branch 'nccl-op' into fs-store
soodoshll Jun 27, 2023
56a96ca
update
soodoshll Jun 27, 2023
a39c199
update
soodoshll Jun 27, 2023
0a04b82
update
soodoshll Jun 27, 2023
eedaf84
add test
soodoshll Jun 27, 2023
37c8654
format & copyright
soodoshll Jun 27, 2023
3fd7491
update
soodoshll Jun 27, 2023
8bc856f
update
soodoshll Jun 27, 2023
bb4d6d1
format
soodoshll Jun 27, 2023
8518e9e
update
soodoshll Jun 27, 2023
dcb87aa
fix
soodoshll Jun 27, 2023
a2d8be6
format
soodoshll Jun 27, 2023
917d24f
fix
soodoshll Jun 28, 2023
fdf749f
fix
soodoshll Jun 28, 2023
816da19
remove redundant seek
soodoshll Jun 28, 2023
c3eee0d
fix
soodoshll Jun 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix
soodoshll committed Jun 27, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit dcb87aaec9a937c58e3e769a28736e6b1be7efbd
1 change: 0 additions & 1 deletion python/hidet/distributed/group.py
Original file line number Diff line number Diff line change
@@ -78,7 +78,6 @@ def size(self) -> int:
return self._world_size

def all_reduce(self, tensor: Tensor, op: str):
print(tensor, op)
assert not tensor.is_symbolic()
assert tensor.device.is_cuda()
addr = tensor.storage.addr
15 changes: 13 additions & 2 deletions tests/unit_tests/test_store.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
# Licensed under the Apache License, Version 2.0 (the "License");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to place this test to hidet/tests/distributed/test_file_store.py.

# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import pytest
import multiprocessing
from multiprocessing import Process, Queue
@@ -39,8 +50,8 @@ def subproc(q):
q.put(b)

store = FileStore(TMP_PATH)
store.set('key', random.randbytes(8))
new_value = random.randbytes(8)
store.set('key', b'u98guj89ks')
new_value = b'32894728934798'
store.set('key', new_value)
q = Queue()
p = Process(target=subproc, args=(q,))