puzzled by the latency number

[amithr1]

Hi All,
I was having a difficult time understanding the latency of around 1.6 usec for a 10000000 byte allreduce posted..which gives bw ~ 6 GB/sec. Where do all the other overheads such as kernel launch, cuda synchronization go?All these easily amount to more than 10 usec? I am missing something here..please help me understand..

thanks..

[sjeaugey]

Hi, I'm confused by the figures you have. 10M should be done in 1.6 milliseconds (not microseconds). How did you measure the latency ?

[amithr1]

sorry..it was my mistake. I was not consistent in taking the measurements

[sjeaugey]

Thanks for the clarification. That was too good to be true :-)

Feel free to close the issue if you have all the information you need.