Skip to content

Commit

Permalink
Remove Unnecessary Events from CUPTI Buffer (#1021)
Browse files Browse the repository at this point in the history
Summary:

Currently we use a blocklist to remove events from CUPTI that spam Kineto. With CUDART 12.5 we have a more fine-grained approach to removing events before they even populate the CUPTI buffer.

Differential Revision: D66852611
  • Loading branch information
sraikund16 authored and facebook-github-bot committed Dec 6, 2024
1 parent a52ff32 commit c65cdc6
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions libkineto/src/CuptiActivityApi.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ void CuptiActivityApi::bufferRequested(
size_t* size,
size_t* maxNumRecords) {
std::lock_guard<std::mutex> guard(mutex_);
LOG(VERBOSE) << "CUPTI buffer requested";
if (allocatedGpuTraceBuffers_.size() >= maxGpuBufferCount_) {
stopCollection = true;
LOG(WARNING) << "Exceeded max GPU buffer count ("
Expand Down Expand Up @@ -340,9 +341,21 @@ void CuptiActivityApi::enableCuptiActivities(
}
if (activity == ActivityType::CUDA_RUNTIME) {
CUPTI_CALL(cuptiActivityEnable(CUPTI_ACTIVITY_KIND_RUNTIME));
#if (CUDART_VERSION >= 12050)
CUPTI_CALL(cuptiActivityEnableRuntimeApi(
CUPTI_RUNTIME_TRACE_CBID_cudaGetDevice_v3020, 0));
#endif
}
if (activity == ActivityType::CUDA_DRIVER) {
CUPTI_CALL(cuptiActivityEnable(CUPTI_ACTIVITY_KIND_DRIVER));
#if (CUDART_VERSION >= 12050)
CUPTI_CALL(cuptiActivityEnableDriverApi(
CUPTI_DRIVER_TRACE_CBID_cuKernelGetAttribute, 0));
CUPTI_CALL(cuptiActivityEnableDriverApi(
CUPTI_DRIVER_TRACE_CBID_cuDevicePrimaryCtxGetState, 0));
CUPTI_CALL(cuptiActivityEnableDriverApi(
CUPTI_DRIVER_TRACE_CBID_cuCtxGetCurrent, 0));
#endif
}
if (activity == ActivityType::OVERHEAD) {
CUPTI_CALL(cuptiActivityEnable(CUPTI_ACTIVITY_KIND_OVERHEAD));
Expand Down

0 comments on commit c65cdc6

Please sign in to comment.