Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NativeAOT: Dependency analysis support for generic method reflection #81204

Closed
hez2010 opened this issue Oct 8, 2021 · 10 comments · Fixed by #100858
Closed

NativeAOT: Dependency analysis support for generic method reflection #81204

hez2010 opened this issue Oct 8, 2021 · 10 comments · Fixed by #100858
Labels
area-NativeAOT-coreclr in-pr There is an active PR which will close this issue when it is merged
Milestone

Comments

@hez2010
Copy link
Contributor

hez2010 commented Oct 8, 2021

I hope ILCompiler dependency analysis can take generic pattern into account so that we don't need rd.xml for this case any longer:

// Here we have System.Array.IndexOf<System.Int32>(System.Int32[],System.Int32)
var method = typeof(Array).GetTypeInfo().GetMethod(nameof(Array.IndexOf), 1, BindingFlags.Public | BindingFlags.Static | BindingFlags.DeclaredOnly, null, CallingConventions.Any, new[] { Type.MakeGenericMethodParameter(0).MakeArrayType(), Type.MakeGenericMethodParameter(0) }, null)!.MakeGenericMethod(typeof(int));
var array = new int[] { 3, 2, 1, 0 };
Console.WriteLine(method.Invoke(null, new object?[] { array, 0 }));

Current behavior without a rd.xml:

Unhandled Exception: EETypeRva:0x00306AF8(System.Reflection.MissingRuntimeArtifactException): MakeGenericMethod() cannot create this generic method instantiation because no code was generated for it: 'System.Array.IndexOf<System.Int32>(System.Int32[],System.Int32)'.
@jkotas
Copy link
Member

jkotas commented Oct 8, 2021

cc @MichalStrehovsky

@MichalStrehovsky
Copy link
Member

It could but we haven't put the effort into it because I didn't think this pattern would be common. We would need to know the exact method and the exact instantiation arguments. Whenever I see this pattern, the instantiation arguments are usually statically unknown.

If everything is statically known, one is usually better off e.g. constructing a delegate.

Do you have a real world example of this?

@hez2010
Copy link
Contributor Author

hez2010 commented Oct 9, 2021

efcore is using this pattern heavily to get generic method.

@MichalStrehovsky
Copy link
Member

Do you have an example? I did a quick search for MakeGenericMethod.*typeof regex but I haven't seen a spot that would look like this. We either don't know the left hand side (the method... for reasons such as reading it from a field, or using LINQ (!!!) to find the interesting method), or we don't know the right hand side (the arguments).

@hez2010
Copy link
Contributor Author

hez2010 commented Oct 9, 2021

An example: https://github.com/npgsql/efcore.pg/pull/2028/files#r725139551
I added a comment there to suggest replacing LINQs with pure reflection APIs.

@MichalStrehovsky
Copy link
Member

That one doesn't seem to be calling MakeGenericMethod though.

In general if the exact generic method is known, it's probably more efficient to get a delegate to it and if one really needs a MethodInfo, call Delegate.GetMethodInfo. I wish C# had a methodof.

@JamesNK
Copy link
Member

JamesNK commented Jan 26, 2023

Example of where this would be useful:

internal static class ResultsOfTHelper
{
    [UnconditionalSuppressMessage("AOT", "IL3050", Justification = "MakeGenericType is safe because the target can be statically analyzed during compile.")]
    public static void PopulateMetadataIfTargetIsIEndpointMetadataProvider<TTarget>(MethodInfo method, EndpointBuilder builder)
    {
        if (typeof(IEndpointMetadataProvider).IsAssignableFrom(typeof(TTarget)))
        {
            var methodInfo = typeof(MetadataPopulator<>).MakeGenericType(typeof(TTarget)).GetMethod("PopulateMetadata")!;
            methodInfo.Invoke(null, new object[] { method, builder });
        }
    }

    internal static class MetadataPopulator<TTarget> where TTarget : IEndpointMetadataProvider
    {
        public static void PopulateMetadata(MethodInfo method, EndpointBuilder builder)
        {
            TTarget.PopulateMetadata(method, builder);
        }
    }
}

It currently fails when TTarget is a value type.

ASP.NET Core issue to use future improvement: dotnet/aspnetcore#46267

@hez2010
Copy link
Contributor Author

hez2010 commented Jan 26, 2023

@MichalStrehovsky Even everything is statically known, we may still have to use MakeGenericType/MakeGenericMethod due to some generic constraints cannot be satisfied at the compile time but only at the runtime.
The current C# lacks the ability of pattern matching on generic parameters/generic method overloads on constraints, see dotnet/csharplang#6308.

@MichalStrehovsky
Copy link
Member

Yup, that's why I brought this over from runtimelab. The linked discussion in the aspnetcore repo is about bridging constraints

@MichalStrehovsky MichalStrehovsky added this to the 8.0.0 milestone Feb 14, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Feb 14, 2023
@MichalStrehovsky
Copy link
Member

We don't have a good spot to do this in the compiler. Dataflow analysis runs on top of uninstantiated IL so we can't do this there.

This would ideally be done in RyuJIT (intrinsic expansion of typeof(Foo<>).MakeGenericType(typeof(T)) to typeof(Foo<T>), provided constraints are met, etc). If we do it in RyuJIT, we also need to do it in IL Scanner.

Another closest spot where we could do this is the constant propagation (that rewrites IL) - this would be a single spot on the managed compiler side and it would not require RyuJIT work. However, we don't have facilities to easily do it here. The result would not be great.

I'm moving this to 9.0. This would be better done if we lay some foundational groundwork first.

@MichalStrehovsky MichalStrehovsky modified the milestones: 8.0.0, 9.0.0 Jul 31, 2023
MichalStrehovsky added a commit to MichalStrehovsky/runtime that referenced this issue Feb 28, 2024
Contributes to dotnet#81204.

With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

* Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
* Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy.
* Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.
MichalStrehovsky added a commit that referenced this issue Mar 6, 2024
Contributes to #81204.

With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

* Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
* Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy.
* Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.
MichalStrehovsky added a commit to MichalStrehovsky/runtime that referenced this issue Apr 10, 2024
Follow up to dotnet#99037.
Resolves dotnet#81204.

When `MakeGenericXXX` API is used in the "bridging constraints" pattern and all types/members involved are statically known, don't emit warning from the Roslyn analyzer since ILC will do the right thing and ensure this works.
@dotnet-policy-service dotnet-policy-service bot added the in-pr There is an active PR which will close this issue when it is merged label Apr 10, 2024
MichalStrehovsky added a commit that referenced this issue Apr 12, 2024
Follow up to #99037.
Resolves #81204.

When `MakeGenericXXX` API is used in the "bridging constraints" pattern and all types/members involved are statically known, don't emit warning from the Roslyn analyzer since ILC will do the right thing and ensure this works.
matouskozak pushed a commit to matouskozak/runtime that referenced this issue Apr 30, 2024
…t#100858)

Follow up to dotnet#99037.
Resolves dotnet#81204.

When `MakeGenericXXX` API is used in the "bridging constraints" pattern and all types/members involved are statically known, don't emit warning from the Roslyn analyzer since ILC will do the right thing and ensure this works.
@github-actions github-actions bot locked and limited conversation to collaborators May 13, 2024
Ruihan-Yin pushed a commit to Ruihan-Yin/runtime that referenced this issue May 30, 2024
…t#100858)

Follow up to dotnet#99037.
Resolves dotnet#81204.

When `MakeGenericXXX` API is used in the "bridging constraints" pattern and all types/members involved are statically known, don't emit warning from the Roslyn analyzer since ILC will do the right thing and ensure this works.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-NativeAOT-coreclr in-pr There is an active PR which will close this issue when it is merged
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants