Use RecyclableMemoryStream #16949
Use RecyclableMemoryStream #16949
Conversation
|
I was waiting for Seb reply on the issue to let the author propose the PR, but seems you do it quickly :) |
...rchardCore.Infrastructure/Shells.Database/Configuration/DatabaseShellConfigurationSources.cs
Outdated
Show resolved
Hide resolved
...re/OrchardCore.Infrastructure/Shells.Database/Configuration/DatabaseShellsSettingsSources.cs
Outdated
Show resolved
Hide resolved
test/OrchardCore.Tests/Localization/PortableObjectStringLocalizerTests.cs
Show resolved
Hide resolved
src/OrchardCore/OrchardCore.Infrastructure/Scripting/Files/FilesScriptEngine.cs
Outdated
Show resolved
Hide resolved
src/OrchardCore/OrchardCore.Infrastructure/Documents/DefaultDocumentSerializer.cs
Outdated
Show resolved
Hide resolved
src/OrchardCore/OrchardCore.Infrastructure/Documents/DefaultDocumentSerializer.cs
Outdated
Show resolved
Hide resolved
src/OrchardCore/OrchardCore.Infrastructure/Documents/DefaultDocumentSerializer.cs
Outdated
Show resolved
Hide resolved
src/OrchardCore/OrchardCore.Infrastructure.Abstractions/MemoryStreamFactory.cs
Outdated
Show resolved
Hide resolved
src/OrchardCore.Modules/OrchardCore.Media.Indexing.Pdf/Services/PdfMediaFileTextProvider.cs
Outdated
Show resolved
Hide resolved
| @@ -0,0 +1,11 @@ | |||
| using Microsoft.IO; | |||
There was a problem hiding this comment.
Do we have an actual infrastructure package that is no "abstractions" or a common project where we put helpers?
There was a problem hiding this comment.
There is Infrastructure project. but not all of these projects depend on Infrastructure they depend on Infrastructure.Abstractions` I am also not happy to add this to the abstraction project. I am open to better placement.
|
This pull request has merge conflicts. Please resolve those before requesting a review. |
| @@ -57,37 +59,24 @@ internal static bool IsCompressed(byte[] data) | |||
| return false; | |||
| } | |||
|
|
|||
| internal static byte[] Compress(byte[] data) | |||
| internal static ReadOnlySpan<byte> Compress(byte[] data) | |||
There was a problem hiding this comment.
This won't work.
The memory is taken from the pool on GetStream() and written into by CopyTo(). And you correctly get the contiguous buffer with GetBuffer().Slice().
However by returning this slice to the caller, you also dispose the stream. Adn at that point the pooled buffer will be returned for another client to use it. If you are the only one on the machine then you won't see an issue. But if there is another request trying to pool some memory, they will then write in the same place you are reading the compressed data from.
The solution here is to pass the stream to this method, and the caller to use MemoryStreamFactory. -> Compress(byte[] data, Stream destinationStream)
There was a problem hiding this comment.
I updated the code, please check it out again
| var neededLength = GetByteArrayLengthFromBase64(encoded); | ||
| Span<byte> bytes = new byte[neededLength]; | ||
| if (!Convert.TryFromBase64String(encoded, bytes, out var bytesWritten)) | ||
| { | ||
| throw new FormatException("Invalid Base64 string."); | ||
| } | ||
| return Encoding.UTF8.GetString(bytes); |
There was a problem hiding this comment.
| var neededLength = GetByteArrayLengthFromBase64(encoded); | |
| Span<byte> bytes = new byte[neededLength]; | |
| if (!Convert.TryFromBase64String(encoded, bytes, out var bytesWritten)) | |
| { | |
| throw new FormatException("Invalid Base64 string."); | |
| } | |
| return Encoding.UTF8.GetString(bytes); | |
| var neededLength = GetByteArrayLengthFromBase64(encoded); | |
| using var memoryStream = MemoryStreamFactory.GetStream(neededLength); | |
| var span = memoryStream.GetSpan(neededLength); | |
| if (!Convert.TryFromBase64String(encoded, span, out var bytesWritten)) | |
| { | |
| throw new FormatException("Invalid Base64 string."); | |
| } | |
| return Encoding.UTF8.GetString(span.Slice(0, bytesWritten)); |
There was a problem hiding this comment.
@sebastienros look at the helpers that was added. It's doing the same logic as this now. Do we need to pass neededLength in GetStream() and also GetSpan()? maybe only in GetSpan(), right?
| @@ -11,7 +11,15 @@ public class CommonGeneratorMethods : IGlobalMethodProvider | |||
| Name = "base64", | |||
| Method = serviceProvider => (Func<string, string>)(encoded => | |||
| { | |||
| return Encoding.UTF8.GetString(Convert.FromBase64String(encoded)); | |||
| var neededLength = GetByteArrayLengthFromBase64(encoded); | |||
| Span<byte> bytes = new byte[neededLength]; | |||
There was a problem hiding this comment.
This would be the same as the code on the left. But the code on the left is one line.
| using var gzip = new GZipStream(new MemoryStream(bytes), CompressionMode.Decompress); | ||
| var neededLength = GetByteArrayLengthFromBase64(encoded); | ||
| Span<byte> bytes = new byte[neededLength]; |
There was a problem hiding this comment.
Update with the GetSpan example
There was a problem hiding this comment.
I created a reusable helper for this logic.
| throw new FormatException("Invalid Base64 string."); | ||
| } | ||
| using var stream = MemoryStreamFactory.GetStream(); |
There was a problem hiding this comment.
And when using GetSpan this becomes unnecessary because the conversion is done directly on the stream.
| // The formula to calculate the number of bytes from the base64 string length is: | ||
| private static int GetByteArrayLengthFromBase64(string base64String) | ||
| { | ||
| return (base64String.Length * 3) / 4; |
There was a problem hiding this comment.
NB: it's important to do the multiplication first
|
|
||
| public static class Str | ||
| { | ||
| public static ReadOnlySpan<byte> FromBase64String(string base64) |
There was a problem hiding this comment.
Can't do that. I explained earlier.
|
|
||
| return output.ToArray(); | ||
| return (int)gZip.Length; |
There was a problem hiding this comment.
I don't think it's necessary or useful. The stream has a Length property.
We talked about need to know how much was written for the GetSpan case.
Fix #16946
@sebastienros I am carious to see is RecyclableMemoryStream will have an impact on the project. If you have time to do a comparison that would be awesome.