Would you be interest in the strf library ?

[robhz786]

Hi,

I see that you have tried to persuade the authors of {fmt} to add a function that would return a gather buffer because that would enable contents of arbitrary length to be written into a llfio::file_handle object without dynamic memory allocation. If I understood right, they won't do it. ( fmtlib/fmt#1271 )

You might want to take a look at the strf formatting library. This example shows how strf can be used to write into a llfio::file_handle without dynamic memory allocation. It creates a class that derives from strf::basic_outbuff and that has a stack-allocated buffer whose content is flushed into the llfio::file_handle destination when full. ( this document explains how this works ).

It does not use a gather buffer as you proposed in the {fmt} discussion — that would require some modifications in the library. But it may be an even better solution, because it only makes a syscall when the buffer is flushed, instead of in every invocation of strf::to.

Anyway, I hope this might be interesting to you

Best regards,
Roberto

[ned14]

It's more a case that I wanted to reuse {fmt}'s future constexpr string insert parsing to create an ultra low latency binary logger whereby one logs the string format and the variable values to be formatted on demand, and formatting only ever occurs if we print the thing. I believe Victor since decided to abandon implementing that in a library form, and so there was no longer any purpose in supporting direct gather write support as for most strings you format, the inserts won't be large enough to yield a performance gain (or rather, if you're writing that, by writing it differently it would not be so inefficient).

There was also originally an idea that scatter-gather buffers would be the proper low level abstraction for text handling, but ultimately the unicode overhead vastly dwarfs any gains from zero-copy, so benchmarks kinda nixed that idea.

Hopefully that explains some of the history behind the original proposal to {fmt}, and why it's become a bit moot since. Thanks for raising the issue however.