Strictly Single Threaded Usage

[victorstewart]

The applications I'm going to logging are strictly single threaded and pinned to logical CPUs. And I read in the documentation that a background thread is used for dumping the logged data to disk. Is there anyway to... not use a background thread and just periodically dump the log myself?

[bjmiao]

Basically, if you need to periodically dump the log, you break the main workflow and thus you need a background thread. However, I guess you can indeed directly output logs to the disk under current framework.

For C++17 version, I guess you can do the following:

1. Anchor to the file NanoLogCpp17.h, in the function log(...),

• here is where the logging happens, you can substitute this part to a normal fprintf-style piece of codes, in order to dumping the log directly to disk

2. Anchor to the file RuntimeLogger.cc in the constructor function RuntimeLogger

• comment out its last line compressionThread = std::thread(&RuntimeLogger::compressionThreadMain, this);

3. Do some debug stuff if there are potential errors

I did not try it but theoretically it is doable with current version of codes. However, the performance will be largely affected.

[syang0]

Bjmiao is right, currently the background thread is built into NanoLog’s architecture. I did briefly experiment with the second option they suggested, but the patch is outdated now and no longer compiles.

The key concern with disabling a background thread is deadlock/livelock. In particular, the StagingBuffers operate with the assumption that there’s a background thread constantly draining log data out of them. Thus, if the StagingBuffers run out of space, they simply spin for free space. If one disabled the background thread, they’d either have to be very careful of never overruning the buffers, or redesign the buffers so that they have a safety mechanism (for example, they can break out of the spin, take a lock, and perform the output work themselves).

Lastly, yes there are performance implications with changing the system to this design, but they may not be what you expect. I think for most logging operations, performance will actually increase because there will no longer be cache contention and pollution from the background thread constantly modifying the staging buffers. On the flip-side, you’ll have large gaps in computation because your application thread will occasionally have to pause logging in order to drain the buffers themselves.