Experimental: Deferred string copies for StringFormatter.Append

[jamesqo]

Currently, StringFormatter.Append(string) eagerly copies the string into its internal buffer. This is inefficient since we do 2 copies: one to the internal buffer, and one to the result string during ToString.

There is a way to improve on this. Instead of copying the string up-front when Append is called, we record the current index & add the string to a queue to be copied later. Then, during ToString, we copy from our buffer up to that index, copy the contents of our string, then continue copying from our buffer. This way, only 1 copy of the string is ever made.

Visualization for a test case I added in the PR:

var formatter = new StringFormatter();

formatter.Append('H');
formatter.Append("ello");
formatter.Append(' ');
formatter.Append("world");
formatter.Append('!');

// becomes

// Copy "ello" starting at character position 2 / sizeof(char).
// Copy "world" starting at character position 4 / sizeof(char) + "ello".Length.
_copyQueue: [ ["ello", 2], ["world", 4] ]

// Unicode representation of: ['H', ' ', '!']
_buffer: [72, 0, 32, 0, 33, 0]

Please note that this should not be merged yet. The code is a little sloppy-- I used List<T> for queueing the strings instead of a struct, I used Unsafe.CopyBlock instead of Encoding.GetChars which will undoubtedly mess up some inputs with special characters, etc. This is more of a proof-of-concept PR intended to garner initial feedback.

@KrzysztofCwalina What do you think?

[wizzard0]

my 2 cents:
for this type of optimizations, it's very important to keep a performance test suite, due to possible non-obvious interactions in the JIT and similar. Do you have one? Any numbers? :)

Also, possibly provide different API paths if multiple clearly different optimization scenarios emerge (we have generic BCL for optimizing the "general case")

[KrzysztofCwalina]

@jamesqo, this works ok, if you are combining strings. But if somebody appends bunch of ints, date times, etc. or does composite formatting, it ends up being worse, i.e. you have copies and you need to allocate additional strings.

[jamesqo]

@KrzysztofCwalina

But if somebody appends bunch of ints, date times, etc. or does composite formatting, it ends up being worse, i.e. you have copies and you need to allocate additional strings.

Where does this PR cause additional copying/string allocations for those scenarios?

[KrzysztofCwalina]

@dotnet-bot test this please

[shiftylogic]

@jamesqo I think a performance test with both short and long copy queues is necessary to determine the benefit of this change.

[ahsonkhan]

@jamesqo, can you please resolve the conflicts?

@jamesqo I think a performance test with both short and long copy queues is necessary to determine the benefit of this change.

[ahsonkhan]

ping @jamesqo

[jamesqo]

@ahsonkhan Thanks for pinging me. I haven't had time to work on this in the months I've submitted it; I'll close this now and reopen when I get a chance to run perf tests.

BTW, I submitted a similar change that actually implemented this concept in LINQ a while ago (after I opened this PR): dotnet/corefx#14675 It yielded about a 50% reduction in allocations.