Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RequiresDynamicCode analyzer should understand RuntimeFeature.IsDynamicCodeSupported #2715

Closed
MichalStrehovsky opened this issue Mar 30, 2022 · 13 comments · Fixed by dotnet/runtime#94123
Assignees
Milestone

Comments

@MichalStrehovsky
Copy link
Member

Even though M1 below calls a method annotated RequiresDynamicCode, it does so in a code path that is not reachable at runtime when dynamic code is not supported. We should not generate a warning.

static void M1()
{
    if (RuntimeFeature.IsDynamicCodeSupported)
    {
        M2();
    }
}

[RequiresDynamicCode("...")]
static void M2()
{
}

Cc @tlakollo

@sbomer
Copy link
Member

sbomer commented Mar 30, 2022

Related is a similar issue for feature switches in the linker - there it's a problem both for the trim analyzer, and when trimming libraries in "library mode" where the feature switches aren't baked in. This is the issue I mentioned in #2004 (comment).

On the linker side I think this is +1 to the refcount for moving feature switch definitions out of XML. Otherwise we would need to add support to the analyzer for reading XML, which we have been trying so hard to avoid.

@MichalStrehovsky
Copy link
Member Author

Yep, this could be a special case of feature switch support.

I think a problem that we'll run into as we're trying to generalize this is the constant propagation across methods that illink is doing - the constant propagation affects the analysis that happens within illink, but it's not something we can fully emulate in the analyzer - we can do at most one level.

I've had mixed feelings about the multi-layer constant propagation - one one hand I understand the problem constant propagation is trying to solve, but on the other hand I wish we instead worked on refactorings within framework so that we don't need to constant propagate across more than one level (e.g. if someone wraps RuntimeFeature.IsDynamicCodeSupported in a wrapper method and then calls that wrapper and expects the bool to get propagated across 2 methods).

@marek-safar
Copy link
Contributor

I wish we instead worked on refactorings within framework so that we don't need to constant propagate across more than one level

I don't think that would solve the problem either. I'm not sure if with the existing analyzers design we can solve this effectively because the analyzer works with assembly references hence have a minimal understanding of what external dependencies do.

@MichalStrehovsky
Copy link
Member Author

I wish we instead worked on refactorings within framework so that we don't need to constant propagate across more than one level

I don't think that would solve the problem either. I'm not sure if with the existing analyzers design we can solve this effectively because the analyzer works with assembly references hence have a minimal understanding of what external dependencies do.

It would require a combination of what Sven wrote (feature switches encoded as custom attributes) + only doing one level of constant propagation. We would then see that RuntimeFeature.IsDynamicCodeSupported is a constant even from a ref assembly and be able to handle it as such in the analyzer.

@agocke agocke modified the milestones: .NET 7.0, Future Jul 27, 2022
@eerhardt
Copy link
Member

Should the same rule apply to RuntimeFeature.IsDynamicCodeCompiled?

image

I assume I shouldn't get an error there because using the "AOT-unfriendly" API is behind a RuntimeFeature.IsDynamicCodeCompiled switch.

@MichalStrehovsky
Copy link
Member Author

Yeah, IsDynamicCodeCompiled would never be true if IsDynamicCodeSupported is false, so this should also be treated as unreachable code in full-AOT-with-no-fallbacks modes.

@agocke
Copy link
Member

agocke commented Dec 14, 2022

Sounds like this is something worth solving in .NET 8. I'd like to do the "extended" feature where we basically allow this to be part of a Requires attribute definition and it's all well understood and supported in the code, but I think special-casing would also be fine for now.

@agocke agocke assigned sbomer and tlakollo and unassigned sbomer Dec 14, 2022
@vitek-karas
Copy link
Member

Depending on how much we want this to work this will require:

  • Constant expression eval (we already have some bits of this, but we'll need more) - note that we can't rely on Roslyn to do this for us since we will turn a property getter into a constant expression (so it's not a constant from Roslyn's point of view, and AFAIK we can't create a second copy of the expression tree from Roslyn and make it work with existing constant prop)
  • Ability to avoid analyzing branches of code

Eventually it would be really nice to support

  • Feature switches - since they're basically the same thing, just more generalized
  • Constant propagation - since feature switches rely on the ability to propagate the const over multiple returns (for example EventSource feature switch is like that)
  • Probably some more constant expression eval/data flow

This also probably means that we should move the "Requires" analysis into the data flow analyzer - since we'll need CFGs for this and we already have them there. We also want to avoid analyzing the dead branches with data flow anyway, so we need to unite the two.

@MichalStrehovsky
Copy link
Member Author

Can we just do something dumb as we walk the AST and check if the condition on the if and ? : is RuntimeFeature.IsDynamicCodeXXX and stop walking the part of the AST that is unreachable? It can be documented as the only supported thing and we could move on with life, looking at feedback.

The constant propagation rules are different between what IL Linker constprop is doing and the engine we have in dataflow. Unifying those would be another necessary undertaking (not to mention NativeAOT also has a constprop, and that does even fewer things because I went with 20% of effort for 80% of result).

I think a minimal implementation would still go a long way and cover majority of use cases. Yep, one would have to add an else branch for code like this: https://github.com/dotnet/runtime/blob/b34029154a1da85e1ce37471bd93eb25b015b0db/src/libraries/System.Data.Common/src/System/Data/DataRowExtensions.cs#L151-L154. But it's not the end of the world.

@vitek-karas
Copy link
Member

I'd be perfectly happy to start small on this. There are some things though which we can't really avoid:

  • Moving some of the Requires analysis into the data flow engine - mainly because currently some of this is based on syntax analysis, which is not the right level to decide about conditions. And data flow already deals with CFG and intrinsics, so it would be much easier to add there. We already do this mostly for RUC, so it's not super complex.
  • Improve the CFG analysis to be able to eliminate branches - this is probably not too difficult either, we just don't process the basic blocks in the eliminated branch.
  • Very basic constant propagation (not sure how much we need, but for the typical case it's probably like 10 lines of code)

@agocke
Copy link
Member

agocke commented Dec 15, 2022

I like the idea of only supporting feature switches in an if check, if we won't have big back-compat problems.

I think the predictability would be nice. However, I think it would be better if we could use the analyzer to warn if the check is not in an if switch.

@marek-safar
Copy link
Contributor

Btw, the same if condition with && and || conditions is supported by Platform compatibility analyzer and I expect the analyzer logic to be very much same.

@MichalStrehovsky
Copy link
Member Author

Very basic constant propagation (not sure how much we need, but for the typical case it's probably like 10 lines of code)

This is the place that I want to be very careful about. There's a difference between what dataflow analysis can model and what consprop (in IL linker or in NativeAOT) can do.

For example for this:

if (IntPtr.Size == 3)
    Console.WriteLine("Hi");

int size = IntPtr.Size;
Type t = typeof(List<>);

try
{
  new object();
}
catch (Exception)
{
}

if (size == 3)
    Console.WriteLine("There");

t.GetMethod("Add");

Illink's consprop eliminates the first if as unreachable, but not the second. Dataflow analysis on the other hand doesn't complain about the GetMethod call because it can easily track through the local.

Add to that that NativeAOT does even less consprop work, because the north star there is for this to be handled by RyuJIT and there's no point trying to write a better C# logic when RyuJIT can already do it, and a lot better.

Analyzer and the tool used by publish differing in their behaviors is a source of pain, especially for library developers (who may only run the analyzer). If the analyzer is too smart, we end up with tools producing warnings.

sbomer added a commit to dotnet/runtime that referenced this issue Nov 7, 2023
This adds an understanding of `RuntimeFeature.IsDynamicCodeSupported`
to the ILLink Roslyn analyzer. This is done by maintaining an
additional state, `FeatureContext`, alongside the tracking done for
local variables.

The `FeatureContext` maintains a set of features known to be enabled
at a given point. The lattice meet operator is set intersection, so
that we get features enabled along all paths, and the top value is the
set of all features. At the beginning of the analysis, each basic
block starts out in the "top" state, but the entry point gets
initialized to the empty set of features. Feature checks (calls to
`RuntimeFeature.IsDynamicCodeSupported`) modify the state in the
outgoing branch to track the fact that a given feature is enabled.

When a branch operation is seen at the end of a basic block, the
analysis examines it to see if it is a supported feature check. If so,
it returns an abstract representation of the checked condition (which
can be a boolean expression involving various features):
`FeatureCheckValue`. Because this may be assumed to be true or false
(depending on the outgoing branch), it has to track included and
excluded features, unlike `FeatureContext`.

The analysis then separates the output state for the basic block into
two, one for the conditional successor, and one for the fall-through
successor. It applies the `FeatureCheckValue` to each state
separately, once assuming that the check is true, and once assuming
that it's false, depending on the branch semantics, possibly modifying
the `FeatureContext` in each branch.

To support this, the dataflow state tracking is now done per branch,
instead of per basic block. Previously we tracked the state at the end
of each block; now we track the state at the input to each edge in the
control-flow graph.

The supported feature checks are hard-coded in the analyzer (this
change isn't introducing any kind of attribute-based model to replace
the feature xml files). For now the only supported check is
`RuntimeFeature.IsDynamicCodeSupported`, but it should be easy to add
new feature checks. This change includes testing of feature checks for
`RequiresUnreferencedCodeAttribute` and
`RequiresAssemblyFilesAttribute`, by including code in the analyzer
that looks for specific feature guards in the test namespace. Happy to
change to another approach if we don't like this.

There are still some pieces of the Requires analyzer logic (generic
instantiations and dynamic objects, for example) that need to be moved
over to the dataflow analysis, so that the feature checks can act as
guards for all of the related patterns. Until that is done, feature
checks won't silence those particular warnings. I'll continue working
on moving the remaining logic over, but I don't think it needs to
block this change.

Fixes dotnet/linker#2715

This tests the ILLink/ILCompiler behavior by adding XML substitutions
that cause TestFeatures.IsUnreferencedCodeSupported and
TestFeatures.IsAssemblyFilesSupported to be treated as constants.

Most of the tests check IsUnreferencedCodeSupported and
RequiresUnreferencedCode, since that logic is shared by all three
tools.

There are some differences in the current ILLink/ILC test infra:

- ILC only allows embedded substitutions (not separate global
  substitutions)

- ILLink doesn't allow modifying CoreLib from test embedded
  substitutions

Because of this, we don't substitute IsDynamicCodeSupported for ILLink
(it is already substituted by default for NativeAot). This matches the
product behavior for ILLink, and leads to a small difference (extra
warning) in the tests.

We also substitute IsAssemblyFilesSupported so that NativeAot treats
it as a constant, to keep it close to the product behavior and the
analyzer behavior. For simplicity, this is done for both NativeAot and
ILLink, even though it should really be treated the same as
IsDynamicCodeCompiled for ILLink. This makes no difference in the test
behavior because we don't test how ILLink specifically behaves with
IsAssemblyFilesSupported.
sbomer added a commit to dotnet/runtime that referenced this issue Nov 14, 2023
The ILLink roslyn analyzer, ILLink, and ILCompiler all support
different forms of branch removal and constant propagation. This
document is meant to describe the supported/recommended patterns
that should work with all of these tools. Related to the
discussion in dotnet/linker#2715.
eerhardt added a commit to eerhardt/opentelemetry-dotnet-contrib that referenced this issue Mar 25, 2024
The AOT analyzer warns about AOT incompatible code inside of a runtime check for RuntimeFeature.IsDynamicCodeSupported. See dotnet/linker#2715.

Suppress this warning until the AOT analyzer is fixed.

Although this repo doesn't have the AOT analyzer enabled yet, this code can be copied into other repos that do enable the analyzer - ex. dotnet/aspire. Porting this change into the otel source to make taking future updates easier. It will also be useful once this repo targets .NET 8 and enables the AOT analyzer.
amcasey added a commit to dotnet/aspnetcore that referenced this issue May 20, 2024
...now that dotnet/linker#2715 has been fixed.
amcasey added a commit to dotnet/aspnetcore that referenced this issue May 28, 2024
* Remove unnecessary pragmas

...now that dotnet/linker#2715 has been fixed.

* Remove some additional suppressions

* Fix build

* Add RequiresDynamicCode attribute because we are using MakeGenericMethod on a Type that may be a value type.

---------

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

7 participants