Add ExitOnEnqueueCount control (#371)

* Add StopOnEnqueueCount control

If set, CLIntercept will terminate the application on achieved enqueue count.
Useful to debug applications with sporadic issues.

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>

* add ExitOnEnqueueCount control

    If set to nonzero value, application will be forced to exit on achieved enqueue count.
    This can be used to debug sporadic applications issues. In case issue is
    not visible on specified enqueue, not needed to wait application normal exit.

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>

---------

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>

docs/controls.md

@@ -707,6 +707,10 @@ The Intercept Layer for OpenCL Applications will wait for this many milliseconds
 
 ### Execution Controls
 
+##### `ExitOnEnqueueCount` (uint64_t)
+
+If set to a nonzero value, the Intercept Layer for OpenCL Applications will exit the application on achieved enqueue count. This can be used to debug sporadic issues in applications. In case issue is not visible in identified faulting kernel, not needed to wait application normal exit
+
 ##### `NoErrors` (bool)
 
 If set to a nonzero value, the Intercept Layer for OpenCL Applications will cause all OpenCL APIs to return a successful error status.

intercept/src/controls.h

@@ -172,6 +172,7 @@ CLI_CONTROL( cl_uint,       AubCaptureStartWait,                    0,      "The
 CLI_CONTROL( cl_uint,       AubCaptureEndWait,                      0,      "The Intercept Layer for OpenCL Applications will wait for this many milliseconds before ending aub capture.")
 
 CLI_CONTROL_SEPARATOR( Execution Controls: )
+CLI_CONTROL( uint64_t,      ExitOnEnqueueCount,                     0,     "If set to a nonzero value, the Intercept Layer for OpenCL Applications will exit the application on achieved enqueue count. This can be used to debug sporadic issues in applications. In case issue is not visible in identified faulting kernel, not needed to wait application normal exit")
 CLI_CONTROL( bool,          NoErrors,                               false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will cause all OpenCL APIs to return a successful error status." )
 CLI_CONTROL( bool,          NullContextCallback,                    false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will force the context callback to be NULL.  With both context callback logging and NULL context callback set, the context callback will still be logged, but any application context callback will not be called." )
 CLI_CONTROL( bool,          FinishAfterEnqueue,                     false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications inserts a call to clFinish() after every enqueue.  The command queue that the command was just enqueued to is passed to clFinish().  This can be used to debug possible timing or resource management issues and will likely impact performance." )

intercept/src/intercept.h

@@ -1947,10 +1947,18 @@ inline uint64_t CLIntercept::getEnqueueCounter() const
 
 inline uint64_t CLIntercept::incrementEnqueueCounter()
 {
+    uint64_t enqueueCounter = m_EnqueueCounter.load();
+    if( enqueueCounter != 0 ) {
+        if( enqueueCounter >= m_Config.ExitOnEnqueueCount )
+        {
+            log("Exit enqueue counter " + std::to_string(enqueueCounter) + " reached - exiting the application...\n");
+            exit(0);
+        }
+    }
+
     uint64_t reportInterval = m_Config.ReportInterval;
     if( reportInterval != 0 )
     {
-        uint64_t enqueueCounter = m_EnqueueCounter.load();
         if( enqueueCounter != 0 && enqueueCounter % reportInterval == 0 )
         {
             report();