Using `StringValues` with `JsonSerializerContext` produces warnings with `7.0.0-preview*` #74652

austindrenski · 2022-08-26T14:48:21Z

Description

JsonSerializerContext produces warnings when models contain StringValues due to nullable annotation mismatch.

Reproduction Steps

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
    <OutputType>exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Primitives" Version="7.0.0-preview.7.22375.6" />
    <PackageReference Include="System.Text.Json" Version="7.0.0-preview.7.22375.6" />
  </ItemGroup>

</Project>

using System;
using System.Text.Json.Serialization;
using Microsoft.Extensions.Primitives;

Console.WriteLine("Hello, world!");

[JsonSerializable(typeof(SomeClass))]
[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Default)]
public sealed partial class SomeJsonSerializerContext : JsonSerializerContext
{
}

public sealed class SomeClass
{
    public StringValues SomeStringValues { get; set; }
}

$ dotnet build
MSBuild version 17.4.0-preview-22368-02+c8492483a for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
C:\Program Files\dotnet\sdk\7.0.100-preview.7.22377.5\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\system-text-json-source-generator-fails-on-stringvalues.csproj]
C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\SomeJsonSerializerContext.StringValues.g.cs(31,32): warning CS8631: The type 'Microsoft.Extensions.Primitives.StringValues' cannot be used as type parameter 'TCollection' in the generic type or method 'JsonMetadataServices.CreateIListInfo<TCollection, TElement>(JsonSerializerOptions, JsonCollectionInfoValues<TCollection>)'. Nullability of type argument 'Microsoft.Extensions.Primitives.StringValues' doesn't match constraint type 'System.Collections.Generic.IList<string>'. [C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\system-text-json-source-generator-fails-on-stringvalues.csproj]
  system-text-json-source-generator-fails-on-stringvalues -> C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\bin\Debug\net7.0\system-text-json-source-generator-fails-on-stringvalues.dll

Build succeeded.

C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\SomeJsonSerializerContext.StringValues.g.cs(31,32): warning CS8631: The type 'Microsoft.Extensions.Primitives.StringValues' cannot be used as type parameter 'TCollection' in the generic type or method 'JsonMetadataServices.CreateIListInfo<TCollection, TElement>(JsonSerializerOptions, JsonCollectionInfoValues<TCollection>)'. Nullability of type argument 'Microsoft.Extensions.Primitives.StringValues' doesn't match constraint type 'System.Collections.Generic.IList<string>'. [C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\system-text-json-source-generator-fails-on-stringvalues.csproj]
    1 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.41

Expected behavior

Using StringValues in conjunction with JsonSerializerContext does not produce warnings.

Actual behavior

Using StringValues in conjunction with JsonSerializerContext produces warnings.

warning CS8631: The type 'Microsoft.Extensions.Primitives.StringValues' cannot be used as type parameter 'TCollection' in the generic type or method 'JsonMetadataServices.CreateIListInfo<TCollection, TElement>(JsonSerializerOptions, JsonCollectionInfoValues)'. Nullability of type argument 'Microsoft.Extensions.Primitives.StringValues' doesn't match constraint type 'System.Collections.Generic.IList'.

Regression?

This works when using version 6.0.0 of Microsoft.Extensions.Primitives because the interfaces for StringValues were updated with nullable annotations for 7.0.0-preview*.

Known Workarounds

Suppress CS8631
Stay on version 6.0.0 of Microsoft.Extensions.Primitives

Configuration

$ dotnet --info
.NET SDK:
 Version:   7.0.100-preview.7.22377.5
 Commit:    ba310d9309

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22000
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.100-preview.7.22377.5\

Host:
  Version:      7.0.0-preview.7.22375.6
  Architecture: x64
  Commit:       eecb028078

Other information

Not sure where the fix belongs, but here's where the warning bubbles from:

runtime/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonMetadataServices.Collections.cs

Lines 195 to 198 in 536f34d

 public static JsonTypeInfo<TCollection> CreateIListInfo<TCollection, TElement>( 

 JsonSerializerOptions options, 

 JsonCollectionInfoValues<TCollection> collectionInfo) 

 where TCollection : IList<TElement>

The text was updated successfully, but these errors were encountered:

ghost · 2022-08-26T14:48:31Z

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

System.Text.Json.JsonSerializerContext produces warnings when models contain StringValues due to nullable annotation mismatch.

Reproduction Steps

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
    <OutputType>exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Primitives" Version="7.0.0-preview.7.22375.6" />
    <PackageReference Include="System.Text.Json" Version="7.0.0-preview.7.22375.6" />
  </ItemGroup>

</Project>

using System;
using System.Text.Json.Serialization;
using Microsoft.Extensions.Primitives;

Console.WriteLine("Hello, world!");

[JsonSerializable(typeof(SomeClass))]
[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Default)]
public sealed partial class SomeJsonSerializerContext : JsonSerializerContext
{
}

public sealed class SomeClass
{
    public StringValues SomeStringValues { get; set; }
}

$ dotnet build
MSBuild version 17.4.0-preview-22368-02+c8492483a for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
C:\Program Files\dotnet\sdk\7.0.100-preview.7.22377.5\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\system-text-json-source-generator-fails-on-stringvalues.csproj]
C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\SomeJsonSerializerContext.StringValues.g.cs(31,32): warning CS8631: The type 'Microsoft.Extensions.Primitives.StringValues' cannot be used as type parameter 'TCollection' in the generic type or method 'JsonMetadataServices.CreateIListInfo<TCollection, TElement>(JsonSerializerOptions, JsonCollectionInfoValues<TCollection>)'. Nullability of type argument 'Microsoft.Extensions.Primitives.StringValues' doesn't match constraint type 'System.Collections.Generic.IList<string>'. [C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\system-text-json-source-generator-fails-on-stringvalues.csproj]
  system-text-json-source-generator-fails-on-stringvalues -> C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\bin\Debug\net7.0\system-text-json-source-generator-fails-on-stringvalues.dll

Build succeeded.

C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\System.Text.Json.SourceGeneration\System.Text.Json.SourceGeneration.JsonSourceGenerator\SomeJsonSerializerContext.StringValues.g.cs(31,32): warning CS8631: The type 'Microsoft.Extensions.Primitives.StringValues' cannot be used as type parameter 'TCollection' in the generic type or method 'JsonMetadataServices.CreateIListInfo<TCollection, TElement>(JsonSerializerOptions, JsonCollectionInfoValues<TCollection>)'. Nullability of type argument 'Microsoft.Extensions.Primitives.StringValues' doesn't match constraint type 'System.Collections.Generic.IList<string>'. [C:\Users\adren\rider\system-text-json-source-generator-fails-on-stringvalues\system-text-json-source-generator-fails-on-stringvalues.csproj]
    1 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.41

Expected behavior

Using StringValues in conjunction with JsonSerializerContext does not produce warnings.

Actual behavior

Using StringValues in conjunction with JsonSerializerContext produces warnings.

warning CS8631: The type 'Microsoft.Extensions.Primitives.StringValues' cannot be used as type parameter 'TCollection' in the generic type or method 'JsonMetadataServices.CreateIListInfo<TCollection, TElement>(JsonSerializerOptions, JsonCollectionInfoValues)'. Nullability of type argument 'Microsoft.Extensions.Primitives.StringValues' doesn't match constraint type 'System.Collections.Generic.IList'.

Regression?

This works when using version 6.0.0 of Microsoft.Extensions.Primitives because the interfaces for StringValues were updated with nullable annotations for 7.0.0-preview*.

Known Workarounds

Suppress CS8631
Stay on version 6.0.0 of Microsoft.Extensions.Primitives

Configuration

$ dotnet --info
.NET SDK:
 Version:   7.0.100-preview.7.22377.5
 Commit:    ba310d9309

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22000
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.100-preview.7.22377.5\

Host:
  Version:      7.0.0-preview.7.22375.6
  Architecture: x64
  Commit:       eecb028078

Other information

Not sure where the fix belongs, but here's where the warning bubbles from:

runtime/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonMetadataServices.Collections.cs

Lines 195 to 198 in 536f34d

 public static JsonTypeInfo<TCollection> CreateIListInfo<TCollection, TElement>( 

 JsonSerializerOptions options, 

 JsonCollectionInfoValues<TCollection> collectionInfo) 

 where TCollection : IList<TElement>

Author:	austindrenski
Assignees:	-
Labels:	`area-System.Text.Json`
Milestone:	-

eiriktsarpalis · 2022-08-26T16:28:37Z

Related to #61734. I don't believe this is a regression in .NET 7, the warnings have surfaced because of updates to the nullability annotation of StringValues.

austindrenski · 2022-08-26T20:24:32Z

Related to #61734. I don't believe this is a regression in .NET 7, the warnings have surfaced because of updates to the nullability annotation of StringValues.

@eiriktsarpalis I agree this isn't an STJ regression in .NET 7 (traces back to #55566 which shipped with .NET 6), but I was surprised to run into it using a combination of System.Text.Json + Microsoft.Extensions.Primitives (i.e. no shade, just the same repo, so had assumed there would have been a test case somewhere).

Given RC1 is right around the corner, I'm assuming this won't meet the bar for shipping with 7.0.0, but since it looks like it impacts any IList<T?> where T : class, would it meet the bar for an STJ 7.0.1 release?

Investigation stuff

Collecting snippets here to see if this is something I can contribute to, or if it's out of my depth.

Here's the generated source causing the issue from SomeJsonSerializerContext.StringValues.g.cs:

// <auto-generated/>
#nullable enable

// Suppress warnings about [Obsolete] member usage in generated code.
#pragma warning disable CS0618
    public sealed partial class SomeJsonSerializerContext
    {
        private global::System.Text.Json.Serialization.Metadata.JsonTypeInfo<global::Microsoft.Extensions.Primitives.StringValues>? _StringValues;
        public global::System.Text.Json.Serialization.Metadata.JsonTypeInfo<global::Microsoft.Extensions.Primitives.StringValues> StringValues
        {
            get => _StringValues ??= Create_StringValues(Options);
        }
        
        private static global::System.Text.Json.Serialization.Metadata.JsonTypeInfo<global::Microsoft.Extensions.Primitives.StringValues> Create_StringValues(global::System.Text.Json.JsonSerializerOptions options)
        {
            global::System.Text.Json.Serialization.Metadata.JsonTypeInfo<global::Microsoft.Extensions.Primitives.StringValues>? jsonTypeInfo = null;
            global::System.Text.Json.Serialization.JsonConverter? customConverter;
            if (options.Converters.Count > 0 && (customConverter = GetRuntimeProvidedCustomConverter(options, typeof(global::Microsoft.Extensions.Primitives.StringValues))) != null)
            {
                jsonTypeInfo = global::System.Text.Json.Serialization.Metadata.JsonMetadataServices.CreateValueInfo<global::Microsoft.Extensions.Primitives.StringValues>(options, customConverter);
            }
            else
            {
                global::System.Text.Json.Serialization.Metadata.JsonCollectionInfoValues<global::Microsoft.Extensions.Primitives.StringValues> info = new global::System.Text.Json.Serialization.Metadata.JsonCollectionInfoValues<global::Microsoft.Extensions.Primitives.StringValues>()
                {
                    ObjectCreator = () => new global::Microsoft.Extensions.Primitives.StringValues(),
                    NumberHandling = default,
                    SerializeHandler = StringValuesSerializeHandler
                };
        
                jsonTypeInfo = global::System.Text.Json.Serialization.Metadata.JsonMetadataServices.CreateIListInfo<global::Microsoft.Extensions.Primitives.StringValues, global::System.String>(options, info);
        
            }
        
            return jsonTypeInfo;
        }
        
        
        private static void StringValuesSerializeHandler(global::System.Text.Json.Utf8JsonWriter writer, global::Microsoft.Extensions.Primitives.StringValues value)
        {
            
            writer.WriteStartArray();
        
            for (int i = 0; i < value.Count; i++)
            {
                writer.WriteStringValue(value[i]);
            }
        
            writer.WriteEndArray();
        }
    }

where the offending line is:

                jsonTypeInfo = global::System.Text.Json.Serialization.Metadata.JsonMetadataServices.CreateIListInfo<global::Microsoft.Extensions.Primitives.StringValues, global::System.String>(options, info);

and the literal patch would be:

-               jsonTypeInfo = global::System.Text.Json.Serialization.Metadata.JsonMetadataServices.CreateIListInfo<global::Microsoft.Extensions.Primitives.StringValues, global::System.String>(options, info);
+               jsonTypeInfo = global::System.Text.Json.Serialization.Metadata.JsonMetadataServices.CreateIListInfo<global::Microsoft.Extensions.Primitives.StringValues, global::System.String?>(options, info);

Back tracking through the generator source code:

runtime/src/libraries/System.Text.Json/gen/JsonSourceGenerator.Emitter.cs

Lines 480 to 488 in ecb3038

 string metadataInitSource = @$"{JsonCollectionInfoValuesTypeRef}<{typeRef}> {InfoVarName} = new {JsonCollectionInfoValuesTypeRef}<{typeRef}>() 

  {{ 

  {ObjectCreatorPropName} = {objectCreatorValue}, 

  {NumberHandlingPropName} = {numberHandlingArg}, 

  {SerializeHandlerPropName} = {serializeHandlerValue} 

  }}; 

  {JsonTypeInfoReturnValueLocalVariableName} = {collectionTypeInfoValue}; 

 ";

runtime/src/libraries/System.Text.Json/gen/JsonSourceGenerator.Emitter.cs

Lines 475 to 477 in ecb3038

default:

collectionTypeInfoValue = $"{enumerableInfoCreationPrefix})";

break;
runtime/src/libraries/System.Text.Json/gen/JsonSourceGenerator.Emitter.cs

Line 441 in ecb3038

string enumerableInfoCreationPrefix = $"{collectionInfoCreationPrefix}{typeRef}, {valueTypeCompilableName}>({OptionsLocalVariableName}, {InfoVarName}";
runtime/src/libraries/System.Text.Json/gen/JsonSourceGenerator.Emitter.cs

Line 389 in ecb3038

string valueTypeCompilableName = collectionValueTypeMetadata.TypeRef;
runtime/src/libraries/System.Text.Json/gen/JsonSourceGenerator.Emitter.cs

Line 387 in ecb3038

TypeGenerationSpec? collectionValueTypeMetadata = typeGenerationSpec.CollectionValueTypeMetadata;
runtime/src/libraries/System.Text.Json/gen/TypeGenerationSpec.cs

Line 67 in ecb3038

public TypeGenerationSpec? CollectionValueTypeMetadata { get; private set; }
runtime/src/libraries/System.Text.Json/gen/TypeGenerationSpec.cs

Lines 17 to 20 in ecb3038

/// <summary>

/// Fully qualified assembly name, prefixed with "global::", e.g. global::System.Numerics.BigInteger.

/// </summary>

public string TypeRef { get; private set; }
runtime/src/libraries/System.Text.Json/gen/TypeGenerationSpec.cs

Line 139 in ecb3038

TypeRef = type.GetCompilableName();

runtime/src/libraries/System.Text.Json/gen/Reflection/TypeExtensions.cs

Lines 10 to 72 in ecb3038

 public static string GetCompilableName(this Type type) 

 { 

 if (type.IsArray) 

 { 

 return GetCompilableName(type.GetElementType()) + "[]"; 

 } 

 if (type.IsGenericParameter) 

 { 

 return type.Name; 

 } 

 StringBuilder sb = new(); 

 sb.Append("global::"); 

 string @namespace = type.Namespace; 

 if (!string.IsNullOrEmpty(@namespace) && @namespace != JsonConstants.GlobalNamespaceValue) 

 { 

 sb.Append(@namespace); 

 sb.Append('.'); 

 } 

 int argumentIndex = 0; 

 AppendTypeChain(sb, type, type.GetGenericArguments(), ref argumentIndex); 

 return sb.ToString(); 

 static void AppendTypeChain(StringBuilder sb, Type type, Type[] genericArguments, ref int argumentIndex) 

 { 

 Type declaringType = type.DeclaringType; 

 if (declaringType != null) 

 { 

 AppendTypeChain(sb, declaringType, genericArguments, ref argumentIndex); 

 sb.Append('.'); 

 } 

 int backTickIndex = type.Name.IndexOf('`'); 

 if (backTickIndex == -1) 

 { 

 sb.Append(type.Name); 

 } 

 else 

 { 

 sb.Append(type.Name, 0, backTickIndex); 

 sb.Append('<'); 

 int startIndex = argumentIndex; 

 argumentIndex = type.GetGenericArguments().Length; 

 for (int i = startIndex; i < argumentIndex; i++) 

 { 

 if (i != startIndex) 

 { 

 sb.Append(", "); 

 } 

 sb.Append(GetCompilableName(genericArguments[i])); 

 } 

 sb.Append('>'); 

 } 

 } 

 }

So it seems like the patch would probably involve making TypeGenerationSpec (more) aware of the annotated nullability of collection types being passed into it. Or maybe preserving the exact annotations when the type is an implemented interface (i.e. StringValues : IList<string?>).

Not sure how much work that would involve, but going to keep poking at this until/unless someone with more experience wants to jump in.

eiriktsarpalis · 2022-08-26T20:41:51Z

Honestly, I don't think this would meet the bar for a .NET 7 fix at this stage. Realistically the only safe intervention might be to introduce a #nullable disable warnings declaration in all source generated files cc @layomia @krwq

krwq · 2022-08-29T06:00:33Z

Given though technically it isn't directly our regression it is a regression from the overall product perspective. IMO we should at least bar check this and I agree the appropriate fix for 7.0 (if any) seems like disabling warnings for the generated code since we don't know all the edge cases

eiriktsarpalis · 2022-08-29T17:13:41Z

Let's try to mitigate the problem (and the related issue in #61734) by disabling nullability warnings in .NET 7.

eiriktsarpalis · 2022-08-31T08:32:52Z

See #74801 (comment). We should keep this one open.

dotnet-issue-labeler bot added the area-System.Text.Json label Aug 26, 2022

ghost added the untriaged New issue has not been triaged by the area owner label Aug 26, 2022

eiriktsarpalis removed the untriaged New issue has not been triaged by the area owner label Aug 26, 2022

eiriktsarpalis added this to the 8.0.0 milestone Aug 26, 2022

eiriktsarpalis added the enhancement Product code improvement that does NOT require public API changes/additions label Aug 26, 2022

eiriktsarpalis modified the milestones: 8.0.0, 7.0.0 Aug 29, 2022

eiriktsarpalis assigned krwq Aug 29, 2022

krwq mentioned this issue Aug 30, 2022

Disable nullability warnings in JSON source generator #74801

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 30, 2022

krwq closed this as completed in #74801 Aug 31, 2022

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 31, 2022

eiriktsarpalis reopened this Aug 31, 2022

eiriktsarpalis modified the milestones: 7.0.0, Future Aug 31, 2022

krwq mentioned this issue Aug 31, 2022

[release/7.0] Disable nullability warnings in JSON source generator #74861

Merged

eiriktsarpalis added the source-generator Indicates an issue with a source generator feature label Sep 2, 2022

krwq removed their assignment Sep 7, 2022

eiriktsarpalis mentioned this issue Dec 2, 2022

CS8631 in System.Text.Json source generated code #78903

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using `StringValues` with `JsonSerializerContext` produces warnings with `7.0.0-preview*` #74652

Using `StringValues` with `JsonSerializerContext` produces warnings with `7.0.0-preview*` #74652

austindrenski commented Aug 26, 2022 •

edited

Loading

ghost commented Aug 26, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

eiriktsarpalis commented Aug 26, 2022 •

edited

Loading

austindrenski commented Aug 26, 2022 •

edited

Loading

eiriktsarpalis commented Aug 26, 2022

krwq commented Aug 29, 2022 •

edited

Loading

eiriktsarpalis commented Aug 29, 2022

eiriktsarpalis commented Aug 31, 2022

Using StringValues with JsonSerializerContext produces warnings with 7.0.0-preview* #74652

Using StringValues with JsonSerializerContext produces warnings with 7.0.0-preview* #74652

Comments

austindrenski commented Aug 26, 2022 • edited Loading

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

ghost commented Aug 26, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

eiriktsarpalis commented Aug 26, 2022 • edited Loading

austindrenski commented Aug 26, 2022 • edited Loading

Investigation stuff

eiriktsarpalis commented Aug 26, 2022

krwq commented Aug 29, 2022 • edited Loading

eiriktsarpalis commented Aug 29, 2022

eiriktsarpalis commented Aug 31, 2022

Using `StringValues` with `JsonSerializerContext` produces warnings with `7.0.0-preview*` #74652

Using `StringValues` with `JsonSerializerContext` produces warnings with `7.0.0-preview*` #74652

austindrenski commented Aug 26, 2022 •

edited

Loading

eiriktsarpalis commented Aug 26, 2022 •

edited

Loading

austindrenski commented Aug 26, 2022 •

edited

Loading

krwq commented Aug 29, 2022 •

edited

Loading