Fix race condition in circular buffer simulation test

[ThomsonTan]

Fixes #841

Changes

This is long existing bug which fails our CI run intermittently. The original intention is that the consumer thread checks whether the exit flag is true, then checks the share buffer with producer thread and makes sure it is empty before exit, but the problem is it checks a cached peek result on the buffer which was generated before checking the exit flag, so when the consumer thread gets exit signal, it can only make sure the old stale snapshot of buffer is empty, not the buffer in real-time, and finally leads to early exit of the consumer thread if there is actually some data in the buffer.

It is very hard to repro this issue in local devbox even with many repeated runs, but it could be reproduced easily with some artificial sleep injected between taking snapshot of buffer and checking exit flag, as below code snippet (the line with sleep_for).

void RunNumberConsumer(CircularBuffer<uint32_t> &buffer,
                       std::atomic<bool> &exit,
                       std::vector<uint32_t> &numbers)
{
  while (true)
  {
    auto allotment = buffer.Peek();
    std::this_thread::sleep_for(std::chrono::milliseconds(100));
    if (exit && allotment.empty())
    {
      return;
    }

For significant contributions please make sure you have completed the following items:

• CHANGELOG.md updated for non-trivial changes
• Unit tests have been added
• Changes in public API reviewed

[codecov]

Codecov Report

Merging #931 (901e3d0) into main (5414ebe) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #931      +/-   ##
==========================================
- Coverage   95.36%   95.36%   -0.00%     
==========================================
  Files         159      159              
  Lines        6776     6775       -1     
==========================================
- Hits         6461     6460       -1     
  Misses        315      315              

Impacted Files	Coverage Δ
sdk/test/common/circular_buffer_test.cc	100.00% <100.00%> (ø)

[lalitb]

Nice. Thanks for fixing it :)