Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tix/vhddwithlocalreduction #8

Open
wants to merge 79 commits into
base: tix/compareMaster
Choose a base branch
from
Open

Tix/vhddwithlocalreduction #8

wants to merge 79 commits into from

Conversation

Tixxx
Copy link
Owner

@Tixxx Tixxx commented Aug 12, 2019

No description provided.

Tix and others added 30 commits July 16, 2019 13:00
first commit of p2p comm together with parasail op
730e9fb
moved p2p comm implementations to header file
e4e3bb6
added more logging and data types for ms allreduce
4bf49e6
incorporated threadpool and changed global state class.
5fadb9d 
Added test.
committing rest of the parallel code. debugging seg fault..
81f4de3
fixed seg fault for 1 tensor case, still happens for multipl tensors
7babc10
fixed seg fault. added multi-tensor test
b644b1b
@vaeksare
added temp buffer for msallreduce op
bc40e87
@vaeksare
added basic pytorch tests for msallreduce
cb29e32
fixed some issues with broadcast when fusing respones. Added more log…
a7c14a5 
…ging.
@vaeksare
added temp buffer for msallreduce op
c5b1a7f
@vaeksare
added basic pytorch tests for msallreduce
9c0a7ac
@vaeksare
fix merge conflict in global state
46e6ab4
merged with temp_buffer
6d5fd6c
merged with local change
f6e6c89
added single and multiple large tensor test
4402dac
integrated new parasail algorithm
fa658eb
added new parasail algo
b8cab29
fixed averaging bug in tensorflow
6116e7e
added float16 data type
b02994a
merged with vhdd.
c3c0257 
merged with fix in TF averaging logic.
@vaeksare
pytorch workaround
efe1886
@vaeksare
Merge pull request #2 from vaeksare/vaeksare/hvdd
dfda595 
pytorch workaround
integrated with the vhdd bug fix
5509baf
added more optimization flags for compiler
c1e5f9c
replaced local reduction with mpi allreduce
83e68e1
@klipto
Added a test for fp16,32,64 tensor allreduce correctness
a0b9469
@klipto
Merge pull request #4 from Tixxx/adding_test_functionality
3dde0e4 
Added a test for fp16,32,64 tensor allreduce correctness
merge with tf fixes
278e86c
Merge branch 'tix/vhddwithallreduce' into tix/vhddwithlocalreduction
e364f14
Tix and others added 30 commits August 15, 2019 17:18
fixed mem leak.
e3c75f7 
fixed seg fault.
improved stream usage.
PR comments
75363ef 
assign streams based on layerid and number of threads.
Name change for cublas initilization method
Merge pull request #7 from Tixxx/tix/vhddwithlocalreductiongpu
eabaa57 
tixTix/vhddwithlocalreductiongpu
fixed correctness bug
d6408c9
added kernel calls and the hooks for calling them
e69452a
added kernel calls and the hooks for calling them
925d3e4
cuda kernel compiles now
799fc47
added the CMakeList file for cuda kernel
2595113
fixing bugs with setup.py
71a82d9
Merge branch 'saemal/msallreducecudakernels' of https://github.com/Ti…
6604900 
…xxx/horovod into saemal/msallreducecudakernels
fixed a type error in msallreduce
d485099
@olsaarik
Add hierarchical ring vhdd msallreduce
84288ad
cuda kernels compiles now -- need to fix for -arch=sm_ <60
eda4e4e
@olsaarik
Set ring cuda msallreduce as default
a1093e2
@olsaarik
Fix variable declarations
45b3488
fixed bug in setup.py
4491b32
Merge branch 'olsaarik/ringplusvhdd' into saemal/msallreducecudakernels
e8bcec9
merged with ring allreducew
f518e95
Merge branch 'saemal/kernelcallsformsallreduce' of https://github.com…
fc4c733 
…/Tixxx/horovod into saemal/kernelcallsformsallreduce
fixed copying from device to host
e74f098
Merge pull request #10 from Tixxx/saemal/kernelcallsformsallreduce
254cd7f 
Saemal/kernelcallsformsallreduce
testing the ring allreduce
cd4aaed
changed init and finalize logic in ms_cuda_msallreduce
a3d5910
checking before the nd40 goes away
a15ec1d
fixed most bugs
9331635
fixed the ring order
66305fa
tested ring allreduce for msallreduce
03e225d
@saeedmaleki
Merge branch 'tix/vhddwithlocalreduction' into saemal/msallreducecuda…
17e8d9c 
…kernels
Merge pull request #11 from Tixxx/saemal/msallreducecudakernels
44fd7f8 
Saemal/msallreducecudakernels
fixed a bug in synciter -- it was doing a sum for the first level
285850a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Tixxx @vaeksare @klipto @olsaarik @saeedmaleki