[CoreCLR and native AOT] UnsafeAccessorAttribute supports generic parameters (#99468)

* Generic type support

Instance and static fields on generic types
Re-enable and add to field tests
Generic types with non-generic methods.
Generic static methods

* Native AOT support

* Add design document
Add strict check for precisely matched Generic constraints

Author: AaronRobinsonMSFT

docs/design/features/unsafeaccessors.md

@@ -0,0 +1,137 @@
+# `UnsafeAccessorAttribute`
+
+## Background and motivation
+
+Number of existing .NET serializers depend on skipping member visibility checks for data serialization. Examples include System.Text.Json or EF Core. In order to skip the visibility checks, the serializers typically use dynamically emitted code (Reflection.Emit or Linq.Expressions) and classic reflection APIs as slow fallback. Neither of these two options are great for source generated serializers and native AOT compilation. This API proposal introduces a first class zero-overhead mechanism for skipping visibility checks.
+
+## Semantics
+
+This attribute will be applied to an `extern static` method. The implementation of the `extern static` method annotated with this attribute will be provided by the runtime based on the information in the attribute and the signature of the method that the attribute is applied to. The runtime will try to find the matching method or field and forward the call to it. If the matching method or field is not found, the body of the `extern static` method will throw `MissingFieldException` or `MissingMethodException`.
+
+For `Method`, `StaticMethod`, `Field`, and `StaticField`, the type of the first argument of the annotated `extern static` method identifies the owning type. Only the specific type defined will be examined for inaccessible members. The type hierarchy is not walked looking for a match.
+
+The value of the first argument is treated as `this` pointer for instance fields and methods.
+
+The first argument must be passed as `ref` for instance fields and methods on structs.
+
+The value of the first argument is not used by the implementation for static fields and methods.
+
+The return value for an accessor to a field can be `ref` if setting of the field is desired.
+
+Constructors can be accessed using Constructor or Method.
+
+The return type is considered for the signature match. Modreqs and modopts are initially not considered for the signature match. However, if an ambiguity exists ignoring modreqs and modopts, a precise match is attempted. If an ambiguity still exists, `AmbiguousMatchException` is thrown.
+
+By default, the attributed method's name dictates the name of the method/field. This can cause confusion in some cases since language abstractions, like C# local functions, generate mangled IL names. The solution to this is to use the `nameof` mechanism and define the `Name` property.
+
+Scenarios involving generics may require creating new generic types to contain the `extern static` method definition. The decision was made to require all `ELEMENT_TYPE_VAR` and `ELEMENT_TYPE_MVAR` instances to match identically type and generic parameter index. This means if the target method for access uses an `ELEMENT_TYPE_VAR`, the `extern static` method must also use an `ELEMENT_TYPE_VAR`. For example:
+
+```csharp
+class C<T>
+{
+    T M<U>(U u) => default;
+}
+
+class Accessor<V>
+{
+    // Correct - V is an ELEMENT_TYPE_VAR and W is ELEMENT_TYPE_VAR,
+    //           respectively the same as T and U in the definition of C<T>::M<U>().
+    [UnsafeAccessor(UnsafeAccessorKind.Method, Name = "M")]
+    extern static void CallM<W>(C<V> c, W w);
+
+    // Incorrect - Since Y must be an ELEMENT_TYPE_VAR, but is ELEMENT_TYPE_MVAR below.
+    // [UnsafeAccessor(UnsafeAccessorKind.Method, Name = "M")]
+    // extern static void CallM<Y, Z>(C<Y> c, Z z);
+}
+```
+
+Methods with the `UnsafeAccessorAttribute` that access members with generic parameters are expected to have the same declared constraints with the target member. Failure to do so results in unspecified behavior. For example:
+
+```csharp
+class C<T>
+{
+    T M<U>(U u) where U: Base => default;
+}
+
+class Accessor<V>
+{
+    // Correct - Constraints match the target member.
+    [UnsafeAccessor(UnsafeAccessorKind.Method, Name = "M")]
+    extern static void CallM<W>(C<V> c, W w) where W: Base;
+
+    // Incorrect - Constraints do not match target member.
+    // [UnsafeAccessor(UnsafeAccessorKind.Method, Name = "M")]
+    // extern static void CallM<W>(C<V> c, W w);
+}
+```
+
+## API
+
+```csharp
+namespace System.Runtime.CompilerServices;
+
+[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
+public class UnsafeAccessorAttribute : Attribute
+{
+    public UnsafeAccessorAttribute(UnsafeAccessorKind kind);
+
+    public UnsafeAccessorKind Kind { get; }
+
+    // The name defaults to the annotated method name if not specified.
+    // The name must be null for constructors
+    public string? Name { get; set; }
+}
+
+public enum UnsafeAccessorKind
+{
+    Constructor, // call instance constructor (`newobj` in IL)
+    Method, // call instance method (`callvirt` in IL)
+    StaticMethod, // call static method (`call` in IL)
+    Field, // address of instance field (`ldflda` in IL)
+    StaticField // address of static field (`ldsflda` in IL)
+};
+```
+
+## API Usage
+
+```csharp
+class UserData
+{
+    private UserData() { }
+    public string Name { get; set; }
+}
+
+[UnsafeAccessor(UnsafeAccessorKind.Constructor)]
+extern static UserData CallPrivateConstructor();
+
+// This API allows accessing backing fields for auto-implemented properties with unspeakable names.
+[UnsafeAccessor(UnsafeAccessorKind.Field, Name = "<Name>k__BackingField")]
+extern static ref string GetName(UserData userData);
+
+UserData ud = CallPrivateConstructor();
+GetName(ud) = "Joe";
+```
+
+Using generics
+
+```csharp
+class UserData<T>
+{
+    private T _field;
+    private UserData(T t) { _field = t; }
+    private U ConvertFieldToT<U>() => (U)_field;
+}
+
+// The Accessors class provides the generic Type parameter for the method definitions.
+class Accessors<V>
+{
+    [UnsafeAccessor(UnsafeAccessorKind.Constructor)]
+    extern static UserData<V> CallPrivateConstructor(V v);
+
+    [UnsafeAccessor(UnsafeAccessorKind.Method, Name = "ConvertFieldToT")]
+    extern static U CallConvertFieldToT<U>(UserData<V> userData);
+}
+
+UserData<string> ud = Accessors<string>.CallPrivateConstructor("Joe");
+Accessors<string>.CallPrivateConstructor<object>(ud);
+```

src/coreclr/tools/Common/TypeSystem/IL/Stubs/ILEmitter.cs

@@ -686,27 +686,27 @@ private ILToken NewToken(object value, int tokenType)
 
         public ILToken NewToken(TypeDesc value)
         {
-            return NewToken(value, 0x01000000);
+            return NewToken(value, 0x01000000); // mdtTypeRef
         }
 
         public ILToken NewToken(MethodDesc value)
         {
-            return NewToken(value, 0x0a000000);
+            return NewToken(value, 0x0a000000); // mdtMemberRef
         }
 
         public ILToken NewToken(FieldDesc value)
         {
-            return NewToken(value, 0x0a000000);
+            return NewToken(value, 0x0a000000); // mdtMemberRef
         }
 
         public ILToken NewToken(string value)
         {
-            return NewToken(value, 0x70000000);
+            return NewToken(value, 0x70000000); // mdtString
         }
 
         public ILToken NewToken(MethodSignature value)
         {
-            return NewToken(value, 0x11000000);
+            return NewToken(value, 0x11000000); // mdtSignature
         }
 
         public ILLocalVariable NewLocal(TypeDesc localType, bool isPinned = false)

src/coreclr/tools/Common/TypeSystem/IL/UnsafeAccessors.cs

@@ -29,12 +29,6 @@ public static MethodIL TryGetIL(EcmaMethod method)
                 return GenerateAccessorBadImageFailure(method);
             }
 
-            // Block generic support early
-            if (method.HasInstantiation || method.OwningType.HasInstantiation)
-            {
-                return GenerateAccessorBadImageFailure(method);
-            }
-
             if (!TryParseUnsafeAccessorAttribute(method, decodedAttribute.Value, out UnsafeAccessorKind kind, out string name))
             {
                 return GenerateAccessorBadImageFailure(method);
@@ -54,7 +48,7 @@ public static MethodIL TryGetIL(EcmaMethod method)
                 firstArgType = sig[0];
             }
 
-            bool isAmbiguous = false;
+            SetTargetResult result;
 
             // Using the kind type, perform the following:
             //  1) Validate the basic type information from the signature.
@@ -77,9 +71,10 @@ public static MethodIL TryGetIL(EcmaMethod method)
                     }
 
                     const string ctorName = ".ctor";
-                    if (!TrySetTargetMethod(ref context, ctorName, out isAmbiguous))
+                    result = TrySetTargetMethod(ref context, ctorName);
+                    if (result is not SetTargetResult.Success)
                     {
-                        return GenerateAccessorSpecificFailure(ref context, ctorName, isAmbiguous);
+                        return GenerateAccessorSpecificFailure(ref context, ctorName, result);
                     }
                     break;
                 case UnsafeAccessorKind.Method:
@@ -105,9 +100,10 @@ public static MethodIL TryGetIL(EcmaMethod method)
                     }
 
                     context.IsTargetStatic = kind == UnsafeAccessorKind.StaticMethod;
-                    if (!TrySetTargetMethod(ref context, name, out isAmbiguous))
+                    result = TrySetTargetMethod(ref context, name);
+                    if (result is not SetTargetResult.Success)
                     {
-                        return GenerateAccessorSpecificFailure(ref context, name, isAmbiguous);
+                        return GenerateAccessorSpecificFailure(ref context, name, result);
                     }
                     break;
 
@@ -136,9 +132,10 @@ public static MethodIL TryGetIL(EcmaMethod method)
                     }
 
                     context.IsTargetStatic = kind == UnsafeAccessorKind.StaticField;
-                    if (!TrySetTargetField(ref context, name, ((ParameterizedType)retType).GetParameterType()))
+                    result = TrySetTargetField(ref context, name, ((ParameterizedType)retType).GetParameterType());
+                    if (result is not SetTargetResult.Success)
                     {
-                        return GenerateAccessorSpecificFailure(ref context, name, isAmbiguous);
+                        return GenerateAccessorSpecificFailure(ref context, name, result);
                     }
                     break;
 
@@ -232,6 +229,12 @@ private static bool ValidateTargetType(TypeDesc targetTypeMaybe, out TypeDesc va
                 targetType = null;
             }
 
+            // We do not support signature variables as a target (for example, VAR and MVAR).
+            if (targetType is SignatureVariable)
+            {
+                targetType = null;
+            }
+
             validated = targetType;
             return validated != null;
         }
@@ -366,7 +369,45 @@ private static bool DoesMethodMatchUnsafeAccessorDeclaration(ref GenerationConte
             return true;
         }
 
-        private static bool TrySetTargetMethod(ref GenerationContext context, string name, out bool isAmbiguous, bool ignoreCustomModifiers = true)
+        private static bool VerifyDeclarationSatisfiesTargetConstraints(MethodDesc declaration, TypeDesc targetType, MethodDesc targetMethod)
+        {
+            Debug.Assert(declaration != null);
+            Debug.Assert(targetType != null);
+            Debug.Assert(targetMethod != null);
+
+            if (targetType.HasInstantiation)
+            {
+                Instantiation declClassInst = declaration.OwningType.Instantiation;
+                var instType = targetType.Context.GetInstantiatedType((MetadataType)targetType.GetTypeDefinition(), declClassInst);
+                if (!instType.CheckConstraints())
+                {
+                    return false;
+                }
+
+                targetMethod = instType.FindMethodOnExactTypeWithMatchingTypicalMethod(targetMethod);
+            }
+
+            if (targetMethod.HasInstantiation)
+            {
+                Instantiation declMethodInst = declaration.Instantiation;
+                var instMethod = targetType.Context.GetInstantiatedMethod(targetMethod, declMethodInst);
+                if (!instMethod.CheckConstraints())
+                {
+                    return false;
+                }
+            }
+            return true;
+        }
+
+        private enum SetTargetResult
+        {
+            Success,
+            Missing,
+            Ambiguous,
+            Invalid,
+        }
+
+        private static SetTargetResult TrySetTargetMethod(ref GenerationContext context, string name, bool ignoreCustomModifiers = true)
         {
             TypeDesc targetType = context.TargetType;
 
@@ -399,23 +440,39 @@ private static bool TrySetTargetMethod(ref GenerationContext context, string nam
                         // We have detected ambiguity when ignoring custom modifiers.
                         // Start over, but look for a match requiring custom modifiers
                         // to match precisely.
-                        if (TrySetTargetMethod(ref context, name, out isAmbiguous, ignoreCustomModifiers: false))
-                            return true;
+                        if (SetTargetResult.Success == TrySetTargetMethod(ref context, name, ignoreCustomModifiers: false))
+                            return SetTargetResult.Success;
                     }
-
-                    isAmbiguous = true;
-                    return false;
+                    return SetTargetResult.Ambiguous;
                 }
 
                 targetMaybe = md;
             }
 
-            isAmbiguous = false;
+            if (targetMaybe != null)
+            {
+                if (!VerifyDeclarationSatisfiesTargetConstraints(context.Declaration, targetType, targetMaybe))
+                {
+                    return SetTargetResult.Invalid;
+                }
+
+                if (targetMaybe.HasInstantiation)
+                {
+                    TypeDesc[] methodInstantiation = new TypeDesc[targetMaybe.Instantiation.Length];
+                    for (int i = 0; i < methodInstantiation.Length; ++i)
+                    {
+                        methodInstantiation[i] = targetMaybe.Context.GetSignatureVariable(i, true);
+                    }
+                    targetMaybe = targetMaybe.Context.GetInstantiatedMethod(targetMaybe, new Instantiation(methodInstantiation));
+                }
+                Debug.Assert(targetMaybe is not null);
+            }
+
             context.TargetMethod = targetMaybe;
-            return context.TargetMethod != null;
+            return context.TargetMethod != null ? SetTargetResult.Success : SetTargetResult.Missing;
         }
 
-        private static bool TrySetTargetField(ref GenerationContext context, string name, TypeDesc fieldType)
+        private static SetTargetResult TrySetTargetField(ref GenerationContext context, string name, TypeDesc fieldType)
         {
             TypeDesc targetType = context.TargetType;
 
@@ -431,10 +488,10 @@ private static bool TrySetTargetField(ref GenerationContext context, string name
                     && fieldType == fd.FieldType)
                 {
                     context.TargetField = fd;
-                    return true;
+                    return SetTargetResult.Success;
                 }
             }
-            return false;
+            return SetTargetResult.Missing;
         }
 
         private static MethodIL GenerateAccessor(ref GenerationContext context)
@@ -486,7 +543,7 @@ private static MethodIL GenerateAccessor(ref GenerationContext context)
             return emit.Link(context.Declaration);
         }
 
-        private static MethodIL GenerateAccessorSpecificFailure(ref GenerationContext context, string name, bool ambiguous)
+        private static MethodIL GenerateAccessorSpecificFailure(ref GenerationContext context, string name, SetTargetResult result)
         {
             ILEmitter emit = new ILEmitter();
             ILCodeStream codeStream = emit.NewCodeStream();
@@ -496,14 +553,19 @@ private static MethodIL GenerateAccessorSpecificFailure(ref GenerationContext co
 
             MethodDesc thrower;
             TypeSystemContext typeSysContext = context.Declaration.Context;
-            if (ambiguous)
+            if (result is SetTargetResult.Ambiguous)
             {
                 codeStream.EmitLdc((int)ExceptionStringID.AmbiguousMatchUnsafeAccessor);
                 thrower = typeSysContext.GetHelperEntryPoint("ThrowHelpers", "ThrowAmbiguousMatchException");
             }
+            else if (result is SetTargetResult.Invalid)
+            {
+                codeStream.EmitLdc((int)ExceptionStringID.InvalidProgramDefault);
+                thrower = typeSysContext.GetHelperEntryPoint("ThrowHelpers", "ThrowInvalidProgramException");
+            }
             else
             {
-
+                Debug.Assert(result is SetTargetResult.Missing);
                 ExceptionStringID id;
                 if (context.Kind == UnsafeAccessorKind.Field || context.Kind == UnsafeAccessorKind.StaticField)
                 {