[WIP] Chunked array for ByteBuffer #7984
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Still a work in progress for this issue: #7982
This one is a bit tricky. Effectively it's writing data to a memory stream that is constantly getting resized and re-allocated when the buffer size limit is reached. Therefore, we need to solve this by having a memory stream backed by chunked byte arrays. There is nothing built into the core libraries that solve this for us; we need to build it ourselves.
Right now I'm using Span for part of it as it helped me reduce some cognitive load. We can take it out if it's an issue. I know I'm having some trouble with the System.Memory package and the compiler.
ToArray
on theChunkedArray
in places we know have the ability to go passed the LOH.ByteMemory
Fixed LOH byte array allocations in the IDE from metadata #7971 may need to get merged first before we figure this out.ChunkedArray
andChunkedArrayBuilder
Possible Alternatives:
System.Reflection.Metadata.BlobBuilder
(I don't know if this is a good idea because it might be specific for reading assembly metadata and not for general use, @tmat could tell us.)Current impl is based on ideas from
System.Reflection.Metadata.BlobBuilder
.