[wasm] Perf benchmark failures - `System.Text.Json` serializer related failures #88610

radical · 2023-07-10T16:59:32Z

Failing for AOT, and interpereter.

Benchmark: Json_ToStream<Location>.SystemTextJson_Reflection:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NotSupportedException: JsonTypeInfo metadata for type 'MicroBenchmarks.Serializers.Location' was not provided by TypeInfoResolver of type '[]'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.
   at BenchmarkDotNet.Autogenerated.Runnable_7.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Benchmark ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.InvalidOperationException: JsonSerializerOptions instance must specify a TypeInfoResolver setting before being marked as read-only.
   at System.Text.Json.Serialization.Tests.ColdStartSerialization`1[[MicroBenchmarks.Serializers.SimpleStructWithProperties, MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null]].Setup()
   at BenchmarkDotNet.Autogenerated.Runnable_142.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Changes 60799cc...53a10ea ([20230707.3](https://dev.azure.com/dnceng/internal/_build/results?buildId=2216705) to https://dev.azure.com/dnceng/internal/_build/results?buildId=2217531&view=results). The pipeline had other failures earlier, so the starting hash could be a little off.

Sdk version changed from 8.0.100-preview.7.23357.5 to 8.0.100-preview.7.23359.2.
.. which corresponds to dotnet/sdk@f8336e3...cfa4244

dotnet/performance commit used for both was dotnet/performance@a719dec

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

The text was updated successfully, but these errors were encountered:

ghost · 2023-07-10T16:59:38Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Failing for AOT, and interpereter.

Benchmark: Json_ToStream<Location>.SystemTextJson_Reflection:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NotSupportedException: JsonTypeInfo metadata for type 'MicroBenchmarks.Serializers.Location' was not provided by TypeInfoResolver of type '[]'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.
   at BenchmarkDotNet.Autogenerated.Runnable_7.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Benchmark ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.InvalidOperationException: JsonSerializerOptions instance must specify a TypeInfoResolver setting before being marked as read-only.
   at System.Text.Json.Serialization.Tests.ColdStartSerialization`1[[MicroBenchmarks.Serializers.SimpleStructWithProperties, MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null]].Setup()
   at BenchmarkDotNet.Autogenerated.Runnable_142.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Changes 60799cc...53a10ea ([20230707.3](https://dev.azure.com/dnceng/internal/_build/results?buildId=2216705) to https://dev.azure.com/dnceng/internal/_build/results?buildId=2217531&view=results). The pipeline had other failures earlier, so the starting hash could be a little off.

Sdk version changed from 8.0.100-preview.7.23357.5 to 8.0.100-preview.7.23359.2.
.. which corresponds to dotnet/sdk@f8336e3...cfa4244

dotnet/performance commit used for both was dotnet/performance@a719dec

Author:	radical
Assignees:	-
Labels:	`arch-wasm`, `blocking-clean-ci`, `perf-pipeline`
Milestone:	-

radical · 2023-07-10T17:01:26Z

@eiriktsarpalis Any idea what might be wrong here?

eiriktsarpalis · 2023-07-10T17:19:25Z

Seems related to #88480. These benchmarks target the reflection serializer and thus should never be running in trimmed applications or AOT. Were they even working in AOT before the feature switch got turned on?

lewing · 2023-07-10T17:23:34Z

Seems related to #88480. These benchmarks target the reflection serializer and thus should never be running in trimmed applications or AOT. Were they even working in AOT before the feature switch got turned on?

why did you merge that pr on red?

lewing · 2023-07-10T17:27:36Z

opened #88612

radical · 2023-07-10T17:28:16Z

ColdStartSerialization - this one seems to have been around since April, and I haven't seen it fail before this.

lewing · 2023-07-10T17:34:41Z

yeah I was reading across too many failures apologies for the noise

eiriktsarpalis · 2023-07-10T17:42:22Z

@radical I just created this PR that should fix the issue. Can we trigger CI against it to validate?

ghost · 2023-07-10T18:03:28Z

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Failing for AOT, and interpereter.

Benchmark: Json_ToStream<Location>.SystemTextJson_Reflection:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NotSupportedException: JsonTypeInfo metadata for type 'MicroBenchmarks.Serializers.Location' was not provided by TypeInfoResolver of type '[]'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.
   at BenchmarkDotNet.Autogenerated.Runnable_7.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Benchmark ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions:

console.info: Initializing dotnet version 8.0.0-ci commit hash 36481bfd079093fe7d29ef024b1fb801e6c787fe
...
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.InvalidOperationException: JsonSerializerOptions instance must specify a TypeInfoResolver setting before being marked as read-only.
   at System.Text.Json.Serialization.Tests.ColdStartSerialization`1[[MicroBenchmarks.Serializers.SimpleStructWithProperties, MicroBenchmarks, Version=42.42.42.42, Culture=neutral, PublicKeyToken=null]].Setup()
   at BenchmarkDotNet.Autogenerated.Runnable_142.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object , IntPtr* , BindingFlags )
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
test-main.js exiting dba61141-3925-4bba-8dea-e01f32679898.dll --benchmarkName with result -1
MONO_WASM: {"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
console.info: WASM EXIT -1

Changes 60799cc...53a10ea ([20230707.3](https://dev.azure.com/dnceng/internal/_build/results?buildId=2216705) to https://dev.azure.com/dnceng/internal/_build/results?buildId=2217531&view=results). The pipeline had other failures earlier, so the starting hash could be a little off.

Sdk version changed from 8.0.100-preview.7.23357.5 to 8.0.100-preview.7.23359.2.
.. which corresponds to dotnet/sdk@f8336e3...cfa4244

dotnet/performance commit used for both was dotnet/performance@a719dec

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Author:	radical
Assignees:	-
Labels:	`arch-wasm`, `area-System.Text.Json`, `blocking-clean-ci`, `untriaged`, `perf-pipeline`, `needs-area-label`
Milestone:	-

Original issue: dotnet/runtime#88610

…3115) * Explicitly enable JsonSerializerIsReflectionEnabledByDefault * [wasm] Workaround STJ based tests using reflection Original issue: dotnet/runtime#88610 * Revert "Explicitly enable JsonSerializerIsReflectionEnabledByDefault" This reverts commit 2b7a6c8. --------- Co-authored-by: Ankit Jain <radical@gmail.com>

radical added arch-wasm WebAssembly architecture blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines labels Jul 10, 2023

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jul 10, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Jul 10, 2023

eiriktsarpalis mentioned this issue Jul 10, 2023

[wasm] Explicitly enable JsonSerializerIsReflectionEnabledByDefault dotnet/performance#3115

Merged

radical added the area-System.Text.Json label Jul 10, 2023

radical added this to the 8.0.0 milestone Jul 10, 2023

ghost removed the untriaged New issue has not been triaged by the area owner label Jul 10, 2023

radical removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jul 10, 2023

radical added a commit to eiriktsarpalis/performance that referenced this issue Jul 10, 2023

[wasm] Workaround STJ based tests using reflection

86f79ef

Original issue: dotnet/runtime#88610

radical changed the title ~~[wasm] Perf benchmark failures~~ [wasm] Perf benchmark failures - System.Text.Json serializer related failures Jul 10, 2023

lewing closed this as completed in dotnet/performance#3115 Jul 11, 2023

ghost locked as resolved and limited conversation to collaborators Aug 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wasm] Perf benchmark failures - `System.Text.Json` serializer related failures #88610

[wasm] Perf benchmark failures - `System.Text.Json` serializer related failures #88610

radical commented Jul 10, 2023 •

edited by build-analysis bot

Loading

ghost commented Jul 10, 2023

radical commented Jul 10, 2023

eiriktsarpalis commented Jul 10, 2023

lewing commented Jul 10, 2023

lewing commented Jul 10, 2023

radical commented Jul 10, 2023

lewing commented Jul 10, 2023

eiriktsarpalis commented Jul 10, 2023

ghost commented Jul 10, 2023

Known Issue Error Message

Report

Summary

[wasm] Perf benchmark failures - System.Text.Json serializer related failures #88610

[wasm] Perf benchmark failures - System.Text.Json serializer related failures #88610

Comments

radical commented Jul 10, 2023 • edited by build-analysis bot Loading

Known Issue Error Message

Report

Summary

ghost commented Jul 10, 2023

radical commented Jul 10, 2023

eiriktsarpalis commented Jul 10, 2023

lewing commented Jul 10, 2023

lewing commented Jul 10, 2023

radical commented Jul 10, 2023

lewing commented Jul 10, 2023

eiriktsarpalis commented Jul 10, 2023

ghost commented Jul 10, 2023

Known Issue Error Message

Report

Summary

[wasm] Perf benchmark failures - `System.Text.Json` serializer related failures #88610

[wasm] Perf benchmark failures - `System.Text.Json` serializer related failures #88610

radical commented Jul 10, 2023 •

edited by build-analysis bot

Loading