[dotnet-sdk-9.0.100-preview.3.24123.6] Windows SDK thrown "Non-blittable generic types cannot be marshaled" exception. #98977

Junjun-zhao · 2024-02-27T07:10:02Z

Description

Description:
When running 3rd party application with the latest .NET 9 build, applications cannot launch, or some functions don't work anymore. After investigating, we found the issue is caused by Windows sdk, which is thrown "Non-blittable generic types cannot be marshaled" exception.

Application Name: NatTypeTester, MyMoneyNet, FlowLauncher, ModernFlyouts, TextGrab, lively,CountdownNumberPuzzleSolver
OS: Windows 10 21H2
CPU: X64
.NET Build Number: dotnet-sdk-9.0.100-preview.3.24123.6
App & Source Location checking at: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1977811

Github Link:
https://github.com/HMBSbige/NatTypeTester
https://github.com/MoneyTools/MyMoney.Net
https://github.com/Flow-Launcher/Flow.Launcher

Reproduction Steps

Minimal Repro steps (Demo attached):MarshalDirectiveExceptionDemo.zip
The machine has dotnet-sdk-9.0.100-preview.3.24123.6 installed.
1.Create a new Console project.
2.Update TargetFramework in .csproject to "net9.0-windows10.0.22621.0", which is look like this:
<TargetFramework>net9.0-windows10.0.22621.0</TargetFramework>
3.Add following code into Program.cs file:

using Windows.Globalization;
using Windows.Media.Ocr;

List<Language> possibleOCRLanguages = OcrEngine.AvailableRecognizerLanguages.ToList();

// See https://aka.ms/new-console-template for more information
Console.WriteLine("Hello, World!");

Launch the app.

Expected Result:

Expected behavior

App launch successfully.

Actual behavior

Actual Result:
Launch failed with error:

System.Runtime.InteropServices.MarshalDirectiveException
  HResult=0x80131535
  Message=Non-blittable generic types cannot be marshaled.
  Source=WinRT.Runtime
  StackTrace:
   at ABI.System.Collections.Generic.IEnumerator`1.get_HasCurrent()
   at ABI.System.Collections.Generic.IEnumerator`1.FromAbiHelper.MoveNext()
   at ABI.System.Collections.Generic.IEnumerator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.<Main>$(String[] args) in E:\Demos\MarshalDirectiveExceptionDemo\MarshalDirectiveExceptionDemo\Program.cs:line 4

Regression?

Yes

Verify Scenarios:
1). Windows 10 21H2 AMD64 + dotnet-sdk-8.0.101: Pass
2). Windows 10 21H2 AMD64 + dotnet-sdk-9.0.100-preview.2.24119.3: Pass
3). Windows 10 21H2 AMD64 + dotnet-sdk-9.0.100-preview.3.24123.6: Fail
4). Windows 10 21H2 AMD64 + dotnet-sdk-9.0.100-preview.3.24126.3: Fail

Known Workarounds

No response

Configuration

Dotnet Info:
.NET SDK:
Version: 9.0.100-preview.3.24123.6
Commit: 884fddb17a
Workload version: 9.0.100-manifests.06e7bcac
MSBuild version: 17.10.0-preview-24122-01+d4cb14fe4

Runtime Environment:
OS Name: Windows
OS Version: 10.0.19045
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\9.0.100-preview.3.24123.6\

.NET workloads installed:
[wasm-tools]
Installation Source: VS 17.8.34601.278
Manifest Version: 9.0.0-preview.2.24123.1/9.0.100-preview.2
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100-preview.2\microsoft.net.workload.mono.toolchain.current\9.0.0-preview.2.24123.1\WorkloadManifest.json
Install Type: Msi

Host:
Version: 9.0.0-preview.2.24123.1
Architecture: x64
Commit: 99b7601

.NET SDKs installed:
9.0.100-preview.3.24123.6 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 9.0.0-preview.2.24121.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 9.0.0-preview.2.24123.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 9.0.0-preview.2.24122.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other information

@dotnet-actwx-bot @dotnet/compat

The text was updated successfully, but these errors were encountered:

ghost · 2024-02-27T15:39:30Z

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

Description:
When running 3rd party application with the latest .NET 9 build, applications cannot launch, or some functions don't work anymore. After investigating, we found the issue is caused by Windows sdk, which is thrown "Non-blittable generic types cannot be marshaled" exception.

Application Name: NatTypeTester, MyMoneyNet, FlowLauncher, ModernFlyouts, TextGrab, lively,CountdownNumberPuzzleSolver
OS: Windows 10 21H2
CPU: X64
.NET Build Number: dotnet-sdk-9.0.100-preview.3.24123.6
App & Source Location checking at: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1977811

Github Link:
https://github.com/HMBSbige/NatTypeTester
https://github.com/MoneyTools/MyMoney.Net
https://github.com/Flow-Launcher/Flow.Launcher

Reproduction Steps

Minimal Repro steps (Demo attached):MarshalDirectiveExceptionDemo.zip
The machine has dotnet-sdk-9.0.100-preview.3.24123.6 installed.
1.Create a new Console project.
2.Update TargetFramework in .csproject to "net9.0-windows10.0.22621.0", which is look like this:
<TargetFramework>net9.0-windows10.0.22621.0</TargetFramework>
3.Add following code into Program.cs file:

using Windows.Globalization;
using Windows.Media.Ocr;

List<Language> possibleOCRLanguages = OcrEngine.AvailableRecognizerLanguages.ToList();

// See https://aka.ms/new-console-template for more information
Console.WriteLine("Hello, World!");

Launch the app.

Expected Result:

Expected behavior

App launch successfully.

Actual behavior

Actual Result:
Launch failed with error:

System.Runtime.InteropServices.MarshalDirectiveException
  HResult=0x80131535
  Message=Non-blittable generic types cannot be marshaled.
  Source=WinRT.Runtime
  StackTrace:
   at ABI.System.Collections.Generic.IEnumerator`1.get_HasCurrent()
   at ABI.System.Collections.Generic.IEnumerator`1.FromAbiHelper.MoveNext()
   at ABI.System.Collections.Generic.IEnumerator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.<Main>$(String[] args) in E:\Demos\MarshalDirectiveExceptionDemo\MarshalDirectiveExceptionDemo\Program.cs:line 4

Regression?

Yes

Verify Scenarios:
1). Windows 10 21H2 AMD64 + dotnet-sdk-8.0.101: Pass
2). Windows 10 21H2 AMD64 + dotnet-sdk-9.0.100-preview.2.24119.3: Pass
3). Windows 10 21H2 AMD64 + dotnet-sdk-9.0.100-preview.3.24123.6: Fail
4). Windows 10 21H2 AMD64 + dotnet-sdk-9.0.100-preview.3.24126.3: Fail

Known Workarounds

No response

Configuration

Dotnet Info:
.NET SDK:
Version: 9.0.100-preview.3.24123.6
Commit: 884fddb17a
Workload version: 9.0.100-manifests.06e7bcac
MSBuild version: 17.10.0-preview-24122-01+d4cb14fe4

Runtime Environment:
OS Name: Windows
OS Version: 10.0.19045
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\9.0.100-preview.3.24123.6\

.NET workloads installed:
[wasm-tools]
Installation Source: VS 17.8.34601.278
Manifest Version: 9.0.0-preview.2.24123.1/9.0.100-preview.2
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100-preview.2\microsoft.net.workload.mono.toolchain.current\9.0.0-preview.2.24123.1\WorkloadManifest.json
Install Type: Msi

Host:
Version: 9.0.0-preview.2.24123.1
Architecture: x64
Commit: 99b7601

.NET SDKs installed:
9.0.100-preview.3.24123.6 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 9.0.0-preview.2.24121.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 9.0.0-preview.2.24123.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 9.0.0-preview.2.24122.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other information

@dotnet-actwx-bot @dotnet/compat

Author:	Junjun-zhao
Assignees:	-
Labels:	`area-System.Runtime.InteropServices`, `untriaged`
Milestone:	-

AaronRobinsonMSFT · 2024-02-27T17:58:33Z

This regression seems to have been introduced by #97079.

AaronRobinsonMSFT · 2024-02-27T18:03:58Z

@jkotas @jkoritzinsky I've not dug into the specifics of this yet. Does anything jump out to immediately?

/cc @manodasanW @hez2010

The failing stack is:

0:000> k 16
 # Child-SP          RetAddr               Call Site
00 0000005d`2657d6d0 00007ffc`70fb03c7     KERNELBASE!RaiseException+0x6c [minkernel\kernelbase\xcpt.c @ 936] 
01 0000005d`2657d7b0 00007ffc`71063c89     coreclr!_CxxThrowException+0x97 [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\eh\throw.cpp @ 82] 
02 0000005d`2657d810 00007ffc`70ef3637     coreclr!RealCOMPlusThrow+0x13d [D:\a\_work\1\s\src\coreclr\vm\excep.cpp @ 11623] 
03 0000005d`2657d880 00007ffc`70ef2e3a     coreclr!`anonymous namespace'::CreateInteropILStub+0x737 [D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp @ 5051] 
04 0000005d`2657de80 00007ffc`70f61d73     coreclr!NDirect::CreateCLRToNativeILStub+0x166 [D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp @ 5209] 
05 0000005d`2657df40 00007ffc`70f61ef1     coreclr!GetILStubForCalli+0x3ab [D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp @ 6087] 
06 0000005d`2657e190 00007ffc`70faab1e     coreclr!GenericPInvokeCalliStubWorker+0x91 [D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp @ 5975] 
07 0000005d`2657e200 00007ffc`1144e822     coreclr!GenericPInvokeCalliGenILStub+0x5e [D:\a\_work\1\s\src\coreclr\vm\amd64\PInvokeStubs.asm @ 68] 
08 0000005d`2657e2b0 00007ffc`1144e5aa     WinRT_Runtime!ABI.System.Collections.Generic.IEnumerator<Windows.Globalization.Language>.get_HasCurrent+0xd2 [D:\a\1\s\src\WinRT.Runtime\Projections\IEnumerable.net5.cs @ 705] 
09 0000005d`2657e360 00007ffc`1144e4ce     WinRT_Runtime!ABI.System.Collections.Generic.IEnumerator<Windows.Globalization.Language>.FromAbiHelper.MoveNext+0xba [D:\a\1\s\src\WinRT.Runtime\Projections\IEnumerable.net5.cs @ 348] 
0a 0000005d`2657e410 00007ffc`60936254     WinRT_Runtime!ABI.System.Collections.Generic.IEnumerator<System.__Canon>.MoveNext+0x1e [D:\a\1\s\src\WinRT.Runtime\Projections\IEnumerable.net5.cs @ 710] 
0b 0000005d`2657e440 00007ffc`d90bf3fe     System_Private_CoreLib!System.Collections.Generic.List<Windows.Globalization.Language>..ctor+0x124 [_/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/List.cs @ 85] 
0c 0000005d`2657e4a0 00007ffc`11434fd9     System_Linq!System.Linq.Enumerable.ToList<Windows.Globalization.Language>+0x6e
0d 0000005d`2657e4f0 00007ffc`70fa9b63     ConsoleApp1!Program.<Main>$+0x49 [E:\Demos\MarshalDirectiveExceptionDemo\ConsoleApp1\Program.cs @ 4] 
0e 0000005d`2657e540 00007ffc`70ecf1ae     coreclr!CallDescrWorkerInternal+0x83 [D:\a\_work\1\s\src\coreclr\vm\amd64\CallDescrWorkerAMD64.asm @ 100] 
0f 0000005d`2657e580 00007ffc`70f632a4     coreclr!MethodDescCallSite::CallTargetWorker+0x21a [D:\a\_work\1\s\src\coreclr\vm\callhelpers.cpp @ 563] 
10 (Inline Function) --------`--------     coreclr!MethodDescCallSite::Call+0xb [D:\a\_work\1\s\src\coreclr\vm\callhelpers.h @ 458] 
11 0000005d`2657e6c0 00007ffc`70f62e26     coreclr!RunMainInternal+0x11c [D:\a\_work\1\s\src\coreclr\vm\assembly.cpp @ 1249] 
12 0000005d`2657e7e0 00007ffc`70f6251e     coreclr!RunMain+0xd2 [D:\a\_work\1\s\src\coreclr\vm\assembly.cpp @ 1320] 
13 0000005d`2657e890 00007ffc`70f29abf     coreclr!Assembly::ExecuteMainMethod+0x1ae [D:\a\_work\1\s\src\coreclr\vm\assembly.cpp @ 1449] 
14 0000005d`2657eb60 00007ffc`70f24898     coreclr!CorHost2::ExecuteAssembly+0x21f [D:\a\_work\1\s\src\coreclr\vm\corhost.cpp @ 349] 
15 0000005d`2657ece0 00007ffc`e118f106     coreclr!coreclr_execute_assembly+0xd8 [D:\a\_work\1\s\src\coreclr\dlls\mscoree\exports.cpp @ 494]

hez2010 · 2024-02-27T18:17:21Z

We added a check here to block any calli which has a generic type context and requires runtime marshalling:

runtime/src/coreclr/vm/dllimport.cpp

Line 5049 in 151982a

    
           && NDirect::MarshalingRequired(NULL, pStubMD->GetSig(), pSigDesc->m_pModule, &pSigDesc->m_typeContext))

Looking at the code ABI.System.Collections.Generic.IEnumerator`1.HasCurrent in WinRT.Runtime:

public unsafe bool HasCurrent
{
	get
	{
		byte b = 0;
		ExceptionHelpers.ThrowExceptionForHR(_obj.Vftbl.Get_HasCurrent_1(ThisPtr, out b));
		return b != 0;
	}
}

The Get_HasCurrent_1 is a calli with signature delegate* unmanaged[Stdcall]<System.IntPtr, out byte, int> under ABI.System.Collections.Generic.IEnumerator<T>.Vftbl.Get_HasCurrent_1, so it's a calli under a generic context.

Seems that the check here is too conservative.

There're three possible fixes:

Make the check more precise so that it only throws exception for generic type that requires runtime marshalling
Remove the check as we are already not checking this in NativeAOT either.
Update CsWinRT to change the signature to delegate* unmanaged[Stdcall]<System.IntPtr, byte*, int> so that it's blittable

jkotas · 2024-02-27T18:44:47Z

delegate* unmanaged[Stdcall]<System.IntPtr, out byte, int>

There is no generic type in this signature. Why is this check failing on it in the first place? I agree that this check needs fixing.

Update CsWinRT to change the signature to delegate* unmanaged[Stdcall]<System.IntPtr, byte*, int> so that it's blittable

It would be a good idea in any case. The marshalling is unnecessary and just a pure overhead.

jkotas · 2024-02-27T18:48:38Z

There is no generic type in this signature. Why is this check failing on it in the first place? I agree that this check needs fixing.

We may want to strip the generic context when we are creating the calli cookie. Including the generic context in the calli cookie when it is not actually used by the signature prevents sharing the calli cookies for no good reason. It would not be surprised if it shows up as a perf regression in CsWinRT.

hez2010 · 2024-02-27T18:54:47Z

Why is this check failing on it in the first place?

The check failing because the out byte requires runtime marshalling. We do the check if the calli is called under a generic context, regardless of whether the generic type is actually used in the calli signature. Previously we didn't do this check so the out byte is not being observed as an issue.

jkotas · 2024-02-27T18:57:49Z

I agree with your analysis. I think we should strip the unused generic context early when creating the calli cookie: #98977 (comment) . Would you like to submit a PR with the fix?

hez2010 · 2024-02-27T19:04:40Z

Would you like to submit a PR with the fix?

Yes, but I'm afraid I'm not able to work on this until the next week. If it's urgent feel free to pick it up :)

But I think stripping the unused generic context early is not a correct fix, considering this valid case:

RefFoo<int>.Call<int>((delegate* unmanaged<ref int, int, int>)(void*)(delegate* unmanaged<int*, int, int>)&TestPIII, ref x, x));

[UnmanagedCallersOnly]
static unsafe int TestPIII(int* arg1, int arg2) => arg2;

class RefFoo<T>
{
    public unsafe static U Call<U>(void* pfn, ref int arg1, T arg2)
    {
        return ((delegate* unmanaged<ref int, T, U>)pfn)(ref arg1, arg2);
    }
}

All generic context is used but it will fail due to the ref int.

Another fix is to check generic arguments only: hez2010@4b56160

But checking generic arguments only seems weird.

I think we can just remove the check completely as what NativeAOT does today.
Or we can make the check more conservative to check calli without a generic context as well, because we are already disallowing runtime marshalling for UnmanagedCallerOnly and reporting a compile error in C# (would requires fixes in CsWinRT).

@jkotas What do you think?

jkotas · 2024-02-27T20:56:06Z

All generic context is used but it will fail due to the ref int.

I think it is the desired behavior. We have said that we are not going to expand the scenarios where the built-in runtime marshaling kicks in. You example requires built-in runtime marshalling and it did not work before the change, so it should continue to not work after the change.

Fixes dotnet#98977

* Strip unused generic type context for VASigCookie Fixes #98977 --------- Co-authored-by: Aaron Robinson <arobins@microsoft.com>

Junjun-zhao · 2024-03-01T08:41:24Z

Verified this issue and it has been fixed on dotnet-sdk-9.0.100-preview.2.24129.6

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Feb 27, 2024

ghost added the untriaged New issue has not been triaged by the area owner label Feb 27, 2024

vcsjones added area-System.Runtime.InteropServices and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Feb 27, 2024

AaronRobinsonMSFT removed the untriaged New issue has not been triaged by the area owner label Feb 27, 2024

AaronRobinsonMSFT added this to the 9.0.0 milestone Feb 27, 2024

jkotas added a commit to jkotas/runtime that referenced this issue Feb 28, 2024

Strip unused generic type context for VASigCookie

f4b713e

Fixes dotnet#98977

jkotas mentioned this issue Feb 28, 2024

Strip unused generic type context for VASigCookie #99026

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Feb 28, 2024

jkotas added a commit to jkotas/runtime that referenced this issue Feb 28, 2024

Strip unused generic type context for VASigCookie

7b6159f

Fixes dotnet#98977

jkotas closed this as completed in #99026 Mar 1, 2024

jkotas added a commit that referenced this issue Mar 1, 2024

Strip unused generic type context for VASigCookie (#99026)

b00f084

* Strip unused generic type context for VASigCookie Fixes #98977 --------- Co-authored-by: Aaron Robinson <arobins@microsoft.com>

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Mar 1, 2024

github-actions bot locked and limited conversation to collaborators Apr 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dotnet-sdk-9.0.100-preview.3.24123.6] Windows SDK thrown "Non-blittable generic types cannot be marshaled" exception. #98977

[dotnet-sdk-9.0.100-preview.3.24123.6] Windows SDK thrown "Non-blittable generic types cannot be marshaled" exception. #98977

Junjun-zhao commented Feb 27, 2024

ghost commented Feb 27, 2024

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

AaronRobinsonMSFT commented Feb 27, 2024

AaronRobinsonMSFT commented Feb 27, 2024

hez2010 commented Feb 27, 2024 •

edited

Loading

jkotas commented Feb 27, 2024

jkotas commented Feb 27, 2024

hez2010 commented Feb 27, 2024 •

edited

Loading

jkotas commented Feb 27, 2024

hez2010 commented Feb 27, 2024 •

edited

Loading

jkotas commented Feb 27, 2024 •

edited

Loading

Junjun-zhao commented Mar 1, 2024

[dotnet-sdk-9.0.100-preview.3.24123.6] Windows SDK thrown "Non-blittable generic types cannot be marshaled" exception. #98977

[dotnet-sdk-9.0.100-preview.3.24123.6] Windows SDK thrown "Non-blittable generic types cannot be marshaled" exception. #98977

Comments

Junjun-zhao commented Feb 27, 2024

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

ghost commented Feb 27, 2024

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

AaronRobinsonMSFT commented Feb 27, 2024

AaronRobinsonMSFT commented Feb 27, 2024

hez2010 commented Feb 27, 2024 • edited Loading

jkotas commented Feb 27, 2024

jkotas commented Feb 27, 2024

hez2010 commented Feb 27, 2024 • edited Loading

jkotas commented Feb 27, 2024

hez2010 commented Feb 27, 2024 • edited Loading

jkotas commented Feb 27, 2024 • edited Loading

Junjun-zhao commented Mar 1, 2024

hez2010 commented Feb 27, 2024 •

edited

Loading

hez2010 commented Feb 27, 2024 •

edited

Loading

hez2010 commented Feb 27, 2024 •

edited

Loading

jkotas commented Feb 27, 2024 •

edited

Loading