-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use [NotNullWhenFalse] and [EnsuresNotNull] to drive nullability analysis #26656
Use [NotNullWhenFalse] and [EnsuresNotNull] to drive nullability analysis #26656
Conversation
b6bb6a0
to
ad53fbf
Compare
I feel like it would be better if these methods could somehow be annotated. Or if this info could be plugged intot he compiler through some extensibility mechanism. Otherwise, this is just going to be wackamole. #Resolved |
@CyrusNajmabadi We're planning to do a little of both. #Resolved |
return method.Equals(compilation.GetWellKnownTypeMember(WellKnownMember.System_Diagnostics_Debug_Assert1)) || | ||
method.Equals(compilation.GetWellKnownTypeMember(WellKnownMember.System_Diagnostics_Debug_Assert2)) || | ||
method.Equals(compilation.GetWellKnownTypeMember(WellKnownMember.System_Diagnostics_Debug_Assert3)) || | ||
method.Equals(compilation.GetWellKnownTypeMember(WellKnownMember.System_Diagnostics_Debug_Assert4)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why use WellKnownMember
rather than the extra annotations table? (Presumably, we won't need these well-known members once we support annotations.) #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What we discussed in LDM was that some members would be well-known. That said, maybe that was only for built-in operators?
I like the extra annotations approach. I'll try that. #Closed
{ | ||
VisitTrueWhenNotNull(arguments[0], method.ReturnType.TypeSymbol); | ||
_result = method.ReturnType; | ||
return null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are the last two statements necessary? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to return, since the assertion below would be violated.
We need to set the result, otherwise another assertion breaks (we check that the result is not left as invalid type). #Closed
new BoundBinaryOperator(operand.Syntax, BinaryOperatorKind.Equal, operand, | ||
new BoundLiteral(operand.Syntax, ConstantValue.Null, operand.Type), | ||
constantValueOpt: null, methodOpt: null, LookupResultKind.Viable, boolType), | ||
new BoundLiteral(operand.Syntax, constantValueOpt: null, boolType)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're tracking "maybe null" rather than "definitely null". In short, is the || someUnknownBoolean
necessary? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be implemented without synthesizing nodes? #Closed
|
||
// if (!condition) throw; | ||
VisitIfStatement(new BoundIfStatement(condition.Syntax, SyntheticBoundNodeFactory.NotCore(condition), | ||
new BoundThrowStatement(condition.Syntax, expressionOpt: null), alternativeOpt: null)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we implement this visit method without synthesizing nodes, by mimicking the behavior of VisitIfStatement
and VisitThrowExpression
? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't want to duplicate that logic, but we can discuss some alternatives. #Closed
return false; | ||
} | ||
|
||
return method.Equals(compilation.GetWellKnownTypeMember(WellKnownMember.System_String_IsNullOrEmpty)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a // PROTOTYPE
comment to replace this with checking annotations. Same for MethodEnsuresTrueWhenExits
. #Resolved
return NotCore(expression); | ||
} | ||
|
||
internal static BoundUnaryOperator NotCore(BoundExpression expression) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps change the return type of the original method. #Resolved
@@ -1568,6 +1574,18 @@ public override BoundNode VisitCall(BoundCall node) | |||
ReplayReadsAndWrites(localFunc, node.Syntax, writes: true); | |||
} | |||
|
|||
if (MethodEnsuresTrueWhenExits(method) && arguments.Length > 0) | |||
{ | |||
VisitTrueWhenExits(condition: arguments[0]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The beginning of VisitCall
has a check for method.CallsAreOmitted
with a comment to skip the visitor in the future when the [Conditional]
attribute is not satisfied. But for methods such as Debug.Assert
(and similar methods), it seems we should visit the condition argument regardless of whether we visit other arguments. Consider having one test with both TestOptions.DebugDll
and TestOptions.ReleaseDll
to ensure that. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said, visiting the condition regardless of whether the call is skipped may set the state incorrectly. For instance, the following should probably generate a warning in RELEASE. Admittedly this is an edge case.
object? o = null;
Debug.Assert((o = new object()) != null);
o.ToString(); // may be null
#Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added test EnsuresNotNull_ConditionalMethodInReleaseMode
In reply to: 186272483 [](ancestors = 186272483)
@AArnott has been dealing with this in the vs-threading analyzers. The additional file content that would be used in this case would be the following:
#Resolved |
This one is interesting. Do you have examples of where it impacts code? #Resolved |
The syntax I use was inspired by powershell, incidentally. #Resolved |
d8a7a5d
to
4e6e56a
Compare
@@ -11,6 +11,7 @@ | |||
using System.Diagnostics; | |||
using System.Linq; | |||
using Microsoft.CodeAnalysis.CSharp.Symbols; | |||
using Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps move ExternalAnnotations
to Microsoft.CodeAnalysis.CSharp.Symbols
. #Resolved
@@ -1577,6 +1606,97 @@ public override BoundNode VisitCall(BoundCall node) | |||
return null; | |||
} | |||
|
|||
private BoundExpression TryGetFirstArgument(MethodSymbol method, ImmutableArray<BoundExpression> arguments) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
static
#Closed
internal enum AttributeAnnotations | ||
{ | ||
None = 0, | ||
NotNullWhenFalse, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NotNullWhenFalse [](start = 8, length = 16)
Please use explicit values for ````[Flags]``` enums #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done already in follow-up PR https://github.com/dotnet/roslyn/pull/26828/files#diff-d02e2d6d70e84697ca87520824236d04R15 #Resolved
/// index 0 is used for return type | ||
/// other parameters follow | ||
/// </summary> | ||
internal static (bool hasAny, AttributeAnnotations annotations) GetExtraAttributes(string key, int parameterIndex) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bool hasAny [](start = 25, length = 11)
I am not sure what extra information do we get from hasAny
flag. Would it be sufficient to simply check ````AttributeAnnotations``` for 0? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in follow-up PR https://github.com/dotnet/roslyn/pull/26828/files#diff-d02e2d6d70e84697ca87520824236d04R169 #Resolved
@@ -385,6 +385,22 @@ internal sealed override ObsoleteAttributeData ObsoleteAttributeData | |||
|
|||
internal abstract bool IsCallerMemberName { get; } | |||
|
|||
internal abstract AttributeAnnotations FlowAnalysisAnnotations { get; } | |||
|
|||
protected (bool memberHasExtra, AttributeAnnotations annotations) TryGetExtraAttributeAnnotations() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
memberHasExtra [](start = 24, length = 14)
The value of AttributeAnnotations
member is not obvious to me. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -151,6 +151,11 @@ internal override bool IsCallerMemberName | |||
get { return _originalParam.IsCallerMemberName; } | |||
} | |||
|
|||
internal override AttributeAnnotations FlowAnalysisAnnotations | |||
{ | |||
get { return _originalParam.FlowAnalysisAnnotations; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_originalParam.FlowAnalysisAnnotations [](start = 25, length = 38)
Do we have a test covering this code path? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added EnsuresNotNull_BeginInvoke
and adjusted behavior (the attributes don't help nullability analysis on BeginInvoke/EndInvoke).
In reply to: 187942517 [](ancestors = 187942517)
(bool memberHasAny, AttributeAnnotations annotations) = TryGetExtraAttributeAnnotations(); | ||
if (memberHasAny) | ||
{ | ||
return annotations; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return annotations [](start = 20, length = 18)
Do we have a test covering this code path? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add in following PR. It's much easier to test after I add annotations for Debug.Assert method, which has two parameters. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Debug.Assert is actually not in the follow-up PR, but the one after that. Added a tracking comment in the meantime: https://github.com/dotnet/roslyn/pull/26828/files#diff-829bf456227af3ee271c7ae37b62ffc0R7056 #Resolved
|
||
CommonParameterWellKnownAttributeData attributeData = GetDecodedWellKnownAttributeData(); | ||
|
||
return AttributeAnnotations.None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return AttributeAnnotations.None [](start = 16, length = 32)
Do we have a test covering this code path? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EnsuresNotNull_Indexer
among others. I think all nullability analysis tests that have complex parameters (ie. with attributes, or default value) come here. #Resolved
get | ||
{ | ||
(_, AttributeAnnotations annotations) = TryGetExtraAttributeAnnotations(); | ||
return annotations; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return annotations; [](start = 16, length = 19)
Do we have tests covering this code path? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NotNullWhenFalse_String_IsNullOrEmpty_WithoutCorlib
and NotNullWhenFalse_String_IsNullOrWhiteSpace_WithoutCorlib
for case with annotations.
Most existing nullability tests for case with no annotations returned. #Resolved
@@ -153,7 +153,12 @@ internal override bool IsCallerMemberName | |||
get { return _underlyingParameter.IsCallerMemberName; } | |||
} | |||
|
|||
public override string GetDocumentationCommentXml(CultureInfo preferredCulture = null, bool expandIncludes = false, CancellationToken cancellationToken = default(CancellationToken)) | |||
internal override AttributeAnnotations FlowAnalysisAnnotations |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
internal override AttributeAnnotations FlowAnalysisAnnotations [](start = 8, length = 62)
In general, in symbols only leaf classes should provide implementation and each leaf class should be covered by targeted tests. This implementation is fine for the prototype, but, if we decide to go with this approach, we should follow up. For example, I will not be surprised if retargeting symbols would need special handling, etc. Please add a prototype comment. #Closed
@@ -306,19 +306,21 @@ public static SyntaxTree ParseWithRoundTripCheck(string text, CSharpParseOptions | |||
string ilSource, | |||
IEnumerable<MetadataReference> references = null, | |||
CSharpCompilationOptions options = null, | |||
bool appendDefaultHeader = true) => CreateCompilationWithILAndMscorlib40(source, ilSource, TargetFramework.Standard, references, options, appendDefaultHeader); | |||
CSharpParseOptions parseOptions = null, | |||
bool appendDefaultHeader = true) => CreateCompilationWithILAndMscorlib40(source, ilSource, TargetFramework.Standard, references, options, parseOptions, appendDefaultHeader); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bool appendDefaultHeader = true) => CreateCompilationWithILAndMscorlib40(source, ilSource, TargetFramework.Standard, references, options, parseOptions, appendDefaultHeader); [](start = 12, length = 173)
Given a number of parameters for the method and the presence of optional ones, the expression body form is "unreadable". Consider converting this method to the regular body form. #Closed
|
||
// We'll ignore NotNullWhenFalse that is misused in metadata | ||
if ((annotations & AttributeAnnotations.NotNullWhenFalse) != 0 && | ||
parameter.ContainingSymbol.GetTypeOrReturnType().SpecialType != SpecialType.System_Boolean) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
parameter [](start = 20, length = 9)
Should we use original definition for this check? Is generic case covered by tests? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added NotNullWhenFalse_RequiresBoolReturn_OnGenericMethod
. The behavior of checking the specialized definition, rather than the original definition, seem ok.
In reply to: 187948468 [](ancestors = 187948468)
|
||
// We'll ignore EnsuresNotNull that is misused in metadata | ||
if ((annotations & AttributeAnnotations.EnsuresNotNull) != 0 && | ||
(parameter.Type?.IsValueType != false || parameter.IsParams)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
parameter [](start = 21, length = 9)
Should we use original definition for this check? Is generic case covered by tests? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add tests.
To do that more easily, I'll pull the change to remove the warning for misused attributes in this PR. That'll also address some other feedback. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added tests EnsuresNotNull_Generic_WithRefType
, EnsuresNotNull_Generic_WithValueType
and EnsuresNotNull_Generic_WithUnconstrainedGenericType
.
In reply to: 187948601 [](ancestors = 187948601)
} | ||
else if (attribute.IsTargetAttribute(this, AttributeDescription.EnsuresNotNullAttribute)) | ||
{ | ||
if (this.Type.IsValueType) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this.Type.IsValueType [](start = 20, length = 21)
What if type is an unconstraint generic type parameter? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unconstrained generic type gives IsValueType
as false
and no warning is produced.
That said, I'm removing those warnings in follow-up PR, from discussion with Neal and Chuck (those warning create compat issue once the attributes are released, if someone uses the attribute with C# 7 compiler then upgrades to C# 8 compiler).
Update: I've pulled the change (removing warnings for misused diagnostics) into this PR. #Resolved
@@ -605,6 +623,28 @@ internal override void DecodeWellKnownAttribute(ref DecodeWellKnownAttributeArgu | |||
// NullableAttribute should not be set explicitly. | |||
arguments.Diagnostics.Add(ErrorCode.ERR_ExplicitNullableAttribute, arguments.AttributeSyntaxOpt.Location); | |||
} | |||
else if (attribute.IsTargetAttribute(this, AttributeDescription.NotNullWhenFalseAttribute)) | |||
{ | |||
if (this.ContainingSymbol.GetTypeOrReturnType().SpecialType == SpecialType.System_Boolean) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this.ContainingSymbol.GetTypeOrReturnType() [](start = 20, length = 43)
What if type is an unconstraint generic type parameter? #Closed
else if (notNullWhenFalse) | ||
{ | ||
// We'll use the WhenTrue/False states to represent whether the invocation returns true/false | ||
Split(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Split(); [](start = 20, length = 8)
I think think this might be a wrong place to do the Split, it should be done when we are visiting invocation, not the arguments. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see a functional advantage to splitting early. Do you mean that would be better for clarity? #Resolved
@@ -3573,6 +3748,19 @@ public bool Reachable | |||
return _knownNullState.Capacity > 0; | |||
} | |||
} | |||
|
|||
public override string ToString() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ToString [](start = 35, length = 8)
Is this for debugging purposes? Consider adding a specialized helper for that rather than to changing ToString. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've changed that to use debugger display method instead, in follow-up PR https://github.com/dotnet/roslyn/pull/26828/files#diff-e3717bc6a837e218d79277c0e45f213aR3791 #Resolved
Done with review pass (iteration 13) #Closed |
} | ||
else | ||
{ | ||
this.State[slot] = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this.State[slot] = true [](start = 24, length = 23)
It feels like in general case prior arguments should be able to take advantage of this information. #Closed
} | ||
else | ||
{ | ||
this.State[slot] = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this.State[slot] = true; [](start = 24, length = 24)
It feels like in general case this information should flow into type argument inference for the method, but it doesn't look like it does. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added tests EnsuresNotNull_BackEffect
, EnsuresNotNull_ForwardEffect
(s), EnsuresNotNull_TypeInference
In reply to: 188022000 [](ancestors = 188022000)
{ | ||
public void Main(string? s) | ||
{ | ||
if (MyIsNullOrEmpty(s)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MyIsNullOrEmpty(s) [](start = 12, length = 18)
Please add a test for conditions of the form MyIsNullOrEmpty(s) == true
and MyIsNullOrEmpty(s) == false
. And also with comparison operands in different order. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is indeed broken. Thanks!
Add tests NotNullWhenFalse_ComparedToTrue
and NotNullWhenFalse_ComparedToFalse
In reply to: 188023547 [](ancestors = 188023547)
{ | ||
if (false == MyIsNullOrEmpty(s)) | ||
{ | ||
s.ToString(); // warn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// ok
?
@AlekseyTs This PR is ready for another look. Thanks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (iteration 18)
test windows_release_unit64_prtest please |
This PR implements
NotNullWhenFalse
(forstring.IsNullOrEmpty
) andEnsuresNotNull
(forThrowIfNull
) attributes, and their effects on nullable analysis.The
EnsuresTrue
(forAssert.Debug
) andEnsuresFalse
attributes will come in a separate PR (jcouv@1a78718).Assert.Debug(condition)
is analyzed asif (!condition) throw;
.string.IsNullOrEmpty(s)
is analyzed ass == null || someBoolean
.Relates to #23259
Fixes #9977 (string.IsNullOrEmpty / IsNullOrWhitespace not recognized)
More design notes