Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PipeWriter overloads to Json #101461

Merged
merged 18 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/libraries/NetCoreAppLibrary.props
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@
System.IO.MemoryMappedFiles;
System.IO.Pipes;
System.IO.Pipes.AccessControl;
System.IO.Pipelines;
System.IO.UnmanagedMemoryStream;
System.Linq;
System.Linq.Expressions;
Expand Down Expand Up @@ -220,7 +221,6 @@
Microsoft.Extensions.Options.DataAnnotations;
Microsoft.Extensions.Primitives;
System.Diagnostics.EventLog;
System.IO.Pipelines;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do these properties do, out of curiosity?

Copy link
Member

@ViktorHofer ViktorHofer Apr 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This property makes the assembly get included in the transport package for aspnetcore. Aspnetcore includes everything from that transport package in their shared framework.

Given that you move this assembly inbox into Microsoft.NETCore.App, this change is correct.

System.Security.Cryptography.Xml;
System.Threading.RateLimiting;
</AspNetCoreAppLibrary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@
<Compile Include="System.IO.Pipelines.cs" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)'">
<ProjectReference Include="$(LibrariesProjectRoot)System.Memory\ref\System.Memory.csproj" />
<ProjectReference Include="$(LibrariesProjectRoot)System.Runtime\ref\System.Runtime.csproj" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFrameworkIdentifier)' != '.NETCoreApp'">
<PackageReference Include="System.Memory" Version="$(SystemMemoryVersion)" />
<PackageReference Include="System.Buffers" Version="$(SystemBuffersVersion)" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,12 @@ System.IO.Pipelines.PipeReader</PackageDescription>
<PackageReference Include="System.Threading.Tasks.Extensions" Version="$(SystemThreadingTasksExtensionsVersion)" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)'">
eiriktsarpalis marked this conversation as resolved.
Show resolved Hide resolved
<Reference Include="System.Collections" />
<Reference Include="System.Memory" />
<Reference Include="System.Runtime" />
<Reference Include="System.Threading" />
<Reference Include="System.Threading.ThreadPool" />
</ItemGroup>

</Project>
9 changes: 9 additions & 0 deletions src/libraries/System.Text.Json/ref/System.Text.Json.cs
Original file line number Diff line number Diff line change
Expand Up @@ -307,8 +307,17 @@ public static void Serialize(System.Text.Json.Utf8JsonWriter writer, object? val
public static System.Threading.Tasks.Task SerializeAsync(System.IO.Stream utf8Json, object? value, System.Type inputType, System.Text.Json.Serialization.JsonSerializerContext context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
[System.Diagnostics.CodeAnalysis.RequiresDynamicCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.")]
[System.Diagnostics.CodeAnalysis.RequiresUnreferencedCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.")]
public static System.Threading.Tasks.Task SerializeAsync<TValue>(System.IO.Pipelines.PipeWriter utf8Json, TValue value, System.Text.Json.JsonSerializerOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
[System.Diagnostics.CodeAnalysis.RequiresDynamicCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.")]
[System.Diagnostics.CodeAnalysis.RequiresUnreferencedCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.")]
public static System.Threading.Tasks.Task SerializeAsync<TValue>(System.IO.Stream utf8Json, TValue value, System.Text.Json.JsonSerializerOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public static System.Threading.Tasks.Task SerializeAsync<TValue>(System.IO.Stream utf8Json, TValue value, System.Text.Json.Serialization.Metadata.JsonTypeInfo<TValue> jsonTypeInfo, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public static System.Threading.Tasks.Task SerializeAsync<TValue>(System.IO.Pipelines.PipeWriter utf8Json, TValue value, System.Text.Json.Serialization.Metadata.JsonTypeInfo<TValue> jsonTypeInfo, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public static System.Threading.Tasks.Task SerializeAsync(System.IO.Pipelines.PipeWriter utf8Json, object? value, System.Text.Json.Serialization.Metadata.JsonTypeInfo jsonTypeInfo, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
[System.Diagnostics.CodeAnalysis.RequiresDynamicCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.")]
[System.Diagnostics.CodeAnalysis.RequiresUnreferencedCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.")]
public static System.Threading.Tasks.Task SerializeAsync(System.IO.Pipelines.PipeWriter utf8Json, object? value, System.Type inputType, System.Text.Json.JsonSerializerOptions? options = null, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public static System.Threading.Tasks.Task SerializeAsync(System.IO.Pipelines.PipeWriter utf8Json, object? value, System.Type inputType, System.Text.Json.Serialization.JsonSerializerContext context, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
public static System.Text.Json.JsonDocument SerializeToDocument(object? value, System.Text.Json.Serialization.Metadata.JsonTypeInfo jsonTypeInfo) { throw null; }
[System.Diagnostics.CodeAnalysis.RequiresDynamicCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications.")]
[System.Diagnostics.CodeAnalysis.RequiresUnreferencedCodeAttribute("JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.")]
Expand Down
1 change: 1 addition & 0 deletions src/libraries/System.Text.Json/ref/System.Text.Json.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@

<ItemGroup>
<ProjectReference Include="$(LibrariesProjectRoot)System.Text.Encodings.Web\ref\System.Text.Encodings.Web.csproj" />
<ProjectReference Include="$(LibrariesProjectRoot)System.IO.Pipelines\ref\System.IO.Pipelines.csproj" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)'">
Expand Down
51 changes: 34 additions & 17 deletions src/libraries/System.Text.Json/src/Resources/Strings.resx
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
<?xml version="1.0" encoding="utf-8"?>
<root>
<!--
Microsoft ResX Schema
<!--
Microsoft ResX Schema

Version 2.0
The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes

The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes
associated with the data types.

Example:

... ado.net/XML headers & schema ...
<resheader name="resmimetype">text/microsoft-resx</resheader>
<resheader name="version">2.0</resheader>
Expand All @@ -22,29 +26,36 @@
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
<comment>This is a comment</comment>
</data>
There are any number of "resheader" rows that contain simple

There are any number of "resheader" rows that contain simple
name/value pairs.
Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the

Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the
mimetype set.
The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not

The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not
extensible. For a given mimetype the value must be set accordingly:
Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can

Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can
read any of the formats listed below.

mimetype: application/x-microsoft.net.object.binary.base64
value : The object must be serialized with
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
: and then encoded with base64 encoding.

mimetype: application/x-microsoft.net.object.soap.base64
value : The object must be serialized with
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
: and then encoded with base64 encoding.

mimetype: application/x-microsoft.net.object.bytearray.base64
value : The object must be serialized into a byte array
value : The object must be serialized into a byte array
: using a System.ComponentModel.TypeConverter
: and then encoded with base64 encoding.
-->
Expand Down Expand Up @@ -717,6 +728,12 @@
<data name="InvalidIndentSize" xml:space="preserve">
<value>Indentation size must be between {0} and {1}.</value>
</data>
<data name="PipeWriterCanceled" xml:space="preserve">
<value>PipeWriter.FlushAsync was canceled.</value>
eiriktsarpalis marked this conversation as resolved.
Show resolved Hide resolved
</data>
<data name="PipeWriterCompleted" xml:space="preserve">
<value>PipeWriter has been completed, nothing more can be written to it.</value>
</data>
<data name="InvalidNewLine" xml:space="preserve">
<value>New line can be only "\n" or "\r\n".</value>
</data>
Expand Down
3 changes: 3 additions & 0 deletions src/libraries/System.Text.Json/src/System.Text.Json.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ The System.Text.Json library is built-in as part of the shared framework in .NET
<Compile Include="System\Text\Json\Reader\Utf8JsonReader.TryGet.cs" />
<Compile Include="System\Text\Json\Serialization\Arguments.cs" />
<Compile Include="System\Text\Json\Serialization\ArgumentState.cs" />
<Compile Include="System\Text\Json\Serialization\AsyncSerializationBufferWriterContext.cs" />
<Compile Include="System\Text\Json\Serialization\Attributes\JsonObjectCreationHandlingAttribute.cs" />
<Compile Include="System\Text\Json\Serialization\Attributes\JsonConstructorAttribute.cs" />
<Compile Include="System\Text\Json\Serialization\Attributes\JsonConverterAttribute.cs" />
Expand Down Expand Up @@ -252,6 +253,7 @@ The System.Text.Json library is built-in as part of the shared framework in .NET
<Compile Include="System\Text\Json\Serialization\JsonSerializer.Write.HandleMetadata.cs" />
<Compile Include="System\Text\Json\Serialization\JsonSerializer.Write.Stream.cs" />
<Compile Include="System\Text\Json\Serialization\JsonSerializer.Write.String.cs" />
<Compile Include="System\Text\Json\Serialization\JsonSerializer.Write.Pipe.cs" />
<Compile Include="System\Text\Json\Serialization\JsonSerializer.Write.Utf8JsonWriter.cs" />
<Compile Include="System\Text\Json\Serialization\JsonSerializerOptions.Caching.cs" />
<Compile Include="System\Text\Json\Serialization\JsonSerializerOptions.Converters.cs" />
Expand Down Expand Up @@ -372,6 +374,7 @@ The System.Text.Json library is built-in as part of the shared framework in .NET
<!-- Application tfms (.NETCoreApp, .NETFramework) need to use the same or higher version of .NETStandard's dependencies. -->
<ItemGroup>
<ProjectReference Include="$(LibrariesProjectRoot)System.Text.Encodings.Web\src\System.Text.Encodings.Web.csproj" />
<ProjectReference Include="$(LibrariesProjectRoot)System.IO.Pipelines\src\System.IO.Pipelines.csproj" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)'">
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System.Buffers;
using System.IO;
using System.IO.Pipelines;
using System.Runtime.CompilerServices;
using System.Threading;
using System.Threading.Tasks;

namespace System.Text.Json.Serialization
{
// Common interface to help de-dupe code for different types that can do async serialization (Stream and PipeWriter)
internal interface IAsyncSerializationBufferWriterContext : IDisposable
{
int FlushThreshold { get; }

ValueTask FlushAsync(CancellationToken cancellationToken);

public IBufferWriter<byte> BufferWriter { get; }
}

internal readonly struct AsyncSerializationStreamContext : IAsyncSerializationBufferWriterContext
{
private readonly Stream _stream;
private readonly JsonSerializerOptions _options;
private readonly PooledByteBufferWriter _bufferWriter;

public AsyncSerializationStreamContext(Stream stream, JsonSerializerOptions options)
{
_stream = stream;
_options = options;
_bufferWriter = new PooledByteBufferWriter(_options.DefaultBufferSize);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public async ValueTask FlushAsync(CancellationToken cancellationToken)
{
await _bufferWriter.WriteToStreamAsync(_stream, cancellationToken).ConfigureAwait(false);
_bufferWriter.Clear();
}

public int FlushThreshold => (int)(_options.DefaultBufferSize * JsonSerializer.FlushThreshold);

public IBufferWriter<byte> BufferWriter => _bufferWriter;

public void Dispose()
{
_bufferWriter.Dispose();
}
}

internal readonly struct AsyncSerializationPipeContext : IAsyncSerializationBufferWriterContext
{
private readonly PipeWriter _pipe;

public AsyncSerializationPipeContext(PipeWriter pipe)
{
_pipe = pipe;
}

public int FlushThreshold => (int)((4 * PipeOptions.Default.MinimumSegmentSize) * JsonSerializer.FlushThreshold);

public IBufferWriter<byte> BufferWriter => _pipe;

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public async ValueTask FlushAsync(CancellationToken cancellationToken)
{
FlushResult result = await _pipe.FlushAsync(cancellationToken).ConfigureAwait(false);
if (result.IsCanceled || result.IsCompleted)
{
if (result.IsCanceled)
{
ThrowHelper.ThrowOperationCanceledException_PipeWriteCanceled();
}

ThrowHelper.ThrowOperationCanceledException_PipeWriteCompleted();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to throw in this case? I think this is going to end up throwing a lot of unwanted exceptions.

For normal HTTP/1.1 response bodies, this just means that the client has closed the connection. BodyWriter.FlushAsync with return a completed flush result, but Body.FlushAsync would not throw because it doesn't want to create a bunch of unnecessary noise in applications writing small responses. Things that actually care about connection close like the static files middleware subscribe to RequestAborted.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right @halter73. Let's remove this exception in a follow up PR.

}
}
eiriktsarpalis marked this conversation as resolved.
Show resolved Hide resolved

public void Dispose() { }
}
}
Loading
Loading