-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use synthesized attribute infrastructure for System.NullableAttribute #22820
Conversation
@dotnet/roslyn-compiler, @OmarTawfik, @AlekseyTs please review. #Resolved |
} | ||
return new SynthesizedAttributeData( | ||
_lazyEmbeddedAttribute.Constructors[0], | ||
ImmutableArray<TypedConstant>.Empty, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking out loud: Is there another way of selecting the correct constructor rather than index? what happens if later on another constructor was added?
Can we filter based on signature?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Index seems like a bad idea. In general the order of metadata is "unordered" hence depending on a specific index seems wrong. Maybe something i'm missing here that makes this safer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_lazy*Attribute
are SynthesizedEmbeddedAttribute
instances. The constructors are an immutable collection created with a fixed order.
ImmutableArray<KeyValuePair<string, TypedConstant>>.Empty); | ||
} | ||
return new SynthesizedAttributeData( | ||
_lazyEmbeddedAttribute.Constructors[0], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_lazyEmbeddedAttribute [](start = 16, length = 22)
Is it still possible for _lazyEmbeddedAttribute
to be null
? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be a bug to call SynthesizeEmbeddedAttribute
without setting _lazyEmbeddedAttribute
first. Previously, we were relying on base.SynthesizeEmbeddedAttribute()
which threw ExceptionUtilities.Unreachable
. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From discussion with Chuck, we need to decide if NullableAttribute
should only be embedded, or whether we could add it to frameworks at some point.
Btw, my assumption was it would become part of frameworks.
|
||
private bool _needsGeneratedIsReadOnlyAttribute_IsFrozen; | ||
private bool _needsGeneratedIsByRefLikeAttribute_IsFrozen; | ||
private bool _needsGeneratedNullableAttribute_IsFrozen; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that we use the IsFrozen
flags to enforce a phase ordering constraint. Do we really need one flag per kind of attribute, or could we do with a single one? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like we only need a single flag. Updated. #Resolved
using Roslyn.Test.Utilities; | ||
using Xunit; | ||
|
||
namespace Microsoft.CodeAnalysis.CSharp.UnitTests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider testing:
(already there)class C : List<object?>
(already there)class C : I<object?>
class C<T> where T : object?
class C<T?>
if that's allowed
What happens if NullableAttribute exists in source, or in a reference?
Consider verifying the emitted attribute (with serialized booleans) in some cases, as we discussed. #Resolved
@@ -499,11 +499,15 @@ internal override void AddSynthesizedAttributes(PEModuleBuilder moduleBuilder, r | |||
|
|||
if (type.ContainsNullableReferenceTypes()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not related to this PR: it seems ContainsNullableReferenceTypes
is implemented by manually visiting type symbols. Is it possible to use VisitType
like ContainsDynamic
and ContainsTupleNames
?
Maybe this needs to wait until we try to merge type annotations into type symbols (assuming that nullability will still be part of binding, not just handled by flow analysis)? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a // PROTOTYPE
comment to TypeSymbol.ContainsNullableReferenceTypes
. #Resolved
@@ -482,6 +486,18 @@ internal void EnsureIsByRefLikeAttributeExists(DiagnosticBag diagnostics, Locati | |||
} | |||
} | |||
|
|||
internal void EnsureNullableAttributeExists(DiagnosticBag diagnostics, Location location, bool modifyCompilationForNullable) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I couldn't find where we pass modifyCompilationForNullable
as false
. Am I missing something? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The argument was not used previously but should have been used for bound lambdas. Updated. #Closed
{ | ||
// PROTOTYPE(NullableReferenceTypes): Create attribute | ||
// with PEModuleBuilder.SynthesizeNullableAttribute(). | ||
// See AttributeTests_Nullable.EmitAttribute_Interface. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a note to test a mix of tuple names and nullability? The code skeleton looks like it will not handle that. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. Updated extension method and unit test. #Resolved
{ | ||
var type = module.ContainingAssembly.GetTypeByMetadataName("C"); | ||
var property = (PropertySymbol)type.GetMembers("this[]").Single(); | ||
AssertNullableAttribute(property.Parameters[1].GetAttributes()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider asserting no nullable attribute on Parameters[0] #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added. #Resolved
} | ||
|
||
internal override bool UtilizesNullableReferenceTypes | ||
{ | ||
get | ||
{ | ||
// PROTOTYPE(NullableReferenceTypes): C#8 projects require System.Attribute. | ||
return _assemblySymbol.DeclaringCompilation.IsFeatureEnabled(MessageID.IDS_FeatureStaticNullChecking); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the plan to remove UtilizesNullableReferenceTypes
altogether? If not, can you clarify what it should mean? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a // PROTOTYPE
comment on the base method in ModuleSymbol
to remove this property. #Resolved
@@ -1034,6 +1034,9 @@ internal override void AfterAddingTypeMembersChecks(ConversionsBase conversions, | |||
} | |||
|
|||
ParameterHelpers.EnsureIsReadOnlyAttributeExists(Parameters, diagnostics, modifyCompilationForRefReadOnly: true); | |||
|
|||
this.EnsureNullableAttributeExistsIfNecessary(ReturnType, diagnostics); | |||
ParameterHelpers.EnsureNullableAttributeExistsIfNecessary(Parameters, diagnostics); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR doesn't change local function or lambda symbols. Should it? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. Added. #Closed
{ | ||
if (type.ContainsNullableReferenceTypes()) | ||
{ | ||
symbol.DeclaringCompilation.EnsureNullableAttributeExists(diagnostics, symbol.Locations[0], modifyCompilationForNullable: true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we sure symbol.Location.Length
is always > 0? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, Locations[0]
was not correct. Fixed. #Resolved
@@ -70,6 +70,14 @@ internal bool NeedsGeneratedIsByRefLikeAttribute | |||
} | |||
} | |||
|
|||
internal bool NeedsGeneratedNullableAttribute |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this property used? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is one caller, PEAssemblyBuilder.CreateEmbeddedAttributesIfNeeded
, similar to NeedsGeneratedIsReadOnlyAttribute
and NeedsGeneratedIsByRefLikeAttribute
. I've changed the properties to protected
so it's more obvious where the properties are used. #Closed
// Also freeze generated attribute flags. | ||
// Symbols bound after getting the declaration | ||
// diagnostics shouldn't need to modify the flags. | ||
_needsGeneratedAttributes_IsFrozen = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not blocking this PR: this raises the follow-up question: are declaration diagnostics and generated attributes getting frozen at the same phase?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What did you think about this? Should we only have one "frozen" flag here too?
In reply to: 146961651 [](ancestors = 146961651)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logged #22887.
parseOptions: TestOptions.Regular8); | ||
comp.VerifyEmitDiagnostics( | ||
// error CS0518: Predefined type 'System.Attribute' is not defined or imported | ||
Diagnostic(ErrorCode.ERR_PredefinedTypeNotFound).WithArguments("System.Attribute").WithLocation(1, 1), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is probably not a new issue, or related to nullable feature, but I wonder if we could tie this diagnostic to the first syntax that triggered it.
Should I file a follow-up issue? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The two errors in the test are use-site errors when failing to create two distinct synthesized attribute types: NullableAttribute
and EmbeddedAttribute
. In this case, the use-site errors happen to be the same error. We will only generate one error per synthesized attribute type declaration for a single compilation though. #Resolved
@@ -3,6 +3,7 @@ | |||
using System.Collections.Generic; | |||
using System.Collections.Immutable; | |||
using System.Diagnostics; | |||
using System.Linq; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this using
? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Used for Parameters.Any()
. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad. I thought ImmutableArrayExtensions
was in one of our namespaces. Thanks #Closed
{ | ||
if (parameter.Type.ContainsNullableReferenceTypes()) | ||
{ | ||
parameter.DeclaringCompilation.EnsureNullableAttributeExists(diagnostics, parameter.GetNonNullSyntaxNode().Location, modifyCompilation); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Method above handles null DeclarationCompilation
. That's not needed here? #Closed
Location GetReturnTypeLocation() | ||
{ | ||
var syntax = (DelegateDeclarationSyntax)SyntaxRef.GetSyntax(); | ||
return syntax.ReturnType.GetLocation(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: empty line below #Closed
@@ -301,15 +301,28 @@ internal override LexicalSortKey GetLexicalSortKey() | |||
|
|||
internal override void AfterAddingTypeMembersChecks(ConversionsBase conversions, DiagnosticBag diagnostics) | |||
{ | |||
Location GetReturnTypeLocation() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: I think we've used camelCase for local functions in compiler code so far. #Closed
@@ -1004,6 +1004,8 @@ internal override void ForceComplete(SourceLocation locationOpt, CancellationTok | |||
|
|||
internal override void AfterAddingTypeMembersChecks(ConversionsBase conversions, DiagnosticBag diagnostics) | |||
{ | |||
Location GetReturnTypeLocation() => GetSyntax().ReturnType.Location; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: same comment on camelCasing #Closed
diagnosticsOpt, | ||
locationOpt, | ||
WellKnownType.System_Runtime_CompilerServices_NullableAttribute, | ||
WellKnownMember.System_Runtime_CompilerServices_NullableAttribute__ctor); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
System_Runtime_CompilerServices_NullableAttribute__ctor [](start = 32, length = 55)
It looks like this attribute has two constructors. Should we check both of them? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. See AttributeTests_Nullable.ExplicitAttribute_MissingConstructor
.
{ | ||
var constructorIndex = (member == WellKnownMember.System_Runtime_CompilerServices_NullableAttribute__ctorTransformFlags) ? 1 : 0; | ||
return new SynthesizedAttributeData( | ||
_lazyNullableAttribute.Constructors[constructorIndex], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_lazyNullableAttribute [](start = 16, length = 22)
It looks like this code is going to cause a NullReferenceException in case when the attribute doesn't have to be embedded. In this case we should fall back to the regular way of synthesizing attributes by using already available type. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. I will handle existing attribute types in a separate PR. See ExplicitAttributeFromSource
test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will handle existing attribute types in a separate PR.
Why? It feels like we should simply follow the pattern of the other similar methods, i.e. call the base when _lazyNullableAttribute
is null. Is there something special about Nullable attribute that I am missing. Is there a reason to deviate? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More tests will be required when supporting existing NullableAttribute
definitions, testing for missing constructors, etc.
NamedTypeSymbol containingType, | ||
DiagnosticBag diagnostics) | ||
{ | ||
var boolType = compilation.GetSpecialType(SpecialType.System_Boolean); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GetSpecialType [](start = 39, length = 14)
Use-site diagnostics? #Closed
@@ -1476,8 +1526,16 @@ protected virtual SynthesizedAttributeData TrySynthesizeIsByRefLikeAttribute() | |||
return Compilation.TrySynthesizeAttribute(WellKnownMember.System_Runtime_CompilerServices_IsByRefLikeAttribute__ctor); | |||
} | |||
|
|||
protected virtual MethodSymbol GetSynthesizedAttributeConstructor(WellKnownMember constructor) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GetSynthesizedAttributeConstructor [](start = 39, length = 34)
It doesn't look like this method is used. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. Removed.
@@ -97,7 +97,7 @@ internal override void AddSynthesizedAttributes(PEModuleBuilder moduleBuilder, r | |||
|
|||
if (returnType.ContainsNullableReferenceTypes()) | |||
{ | |||
AddSynthesizedAttribute(ref attributes, compilation.SynthesizeNullableAttribute(returnType)); | |||
AddSynthesizedAttribute(ref attributes, moduleBuilder.SynthesizeNullableAttribute(this, returnType)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AddSynthesizedAttribute(ref attributes, moduleBuilder.SynthesizeNullableAttribute(this, returnType)); [](start = 16, length = 101)
Where does this attribute go? Shouldn't it be applied to a return type instead? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added EmitAttribute_ExplicitImplementationForwardingMethod
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added EmitAttribute_ExplicitImplementationForwardingMethod
It doesn't look like that test verifies the result of this method. I believe this method adds attributes on the method itself, rather than on the return type of the method, which is incorrect. I think we should simply remove this override of AddSynthesizedAttributes and open an issue to do the same in the "parent" branch. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logged #22952.
}"; | ||
var comp = CreateStandardCompilation(source, parseOptions: TestOptions.Regular8); | ||
comp.VerifyEmitDiagnostics( | ||
// error CS0518: Predefined type 'System.Boolean' is not defined or imported |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// error CS0518: Predefined type 'System.Boolean' is not defined or imported [](start = 16, length = 76)
Doesn't CreateStandardCompilation add mscorlib reference? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated test.
}"; | ||
var comp = CreateStandardCompilation(source, references: new[] { ref0 }, parseOptions: TestOptions.Regular8); | ||
comp.VerifyEmitDiagnostics( | ||
// error CS0518: Predefined type 'System.Boolean' is not defined or imported |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// error CS0518: Predefined type 'System.Boolean' is not defined or imported [](start = 16, length = 76)
Doesn't CreateStandardCompilation add mscorlib reference? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated test.
Done with review pass (iteration 10). #Closed |
@dotnet/roslyn-compiler, @AlekseyTs, please provide a second review, thanks. |
// Generated iterator methods have no [Nullable] attributes. | ||
// Since the methods are only called through IEnumerable<T> | ||
// or IEnumerator<T>, that is not an issue. | ||
AssertNoNullableAttribute(property.GetAttributes()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AssertNoNullableAttribute(property.GetAttributes()); [](start = 20, length = 52)
Please test attributes on the accessor's return type as well. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
symbolValidator: module => | ||
{ | ||
var method = module.ContainingAssembly.GetTypeByMetadataName("B").GetMethod("I.F"); | ||
AssertNullableAttribute(method.GetReturnTypeAttributes()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AssertNullableAttribute(method.GetReturnTypeAttributes()); [](start = 20, length = 58)
Please assert that we don't have the attribute on the method itself, same for other test scenarios. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
Done with review pass (iteration 13). #Closed |
It looks like Microsoft.CodeAnalysis.CSharp.UnitTests.CodeGen.CodeGen_DynamicTests.Iterator needs a base-line adjustment #Closed |
@AlekseyTs, all feedback should be addressed. |
AddSynthesizedAttribute(ref attributes, compilation.SynthesizeNullableAttribute(this.ReturnType)); | ||
} | ||
diagnostics.Free(); | ||
AddSynthesizedAttribute(ref attributes, moduleBuilder.SynthesizeNullableAttribute(this, this.ReturnType)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AddSynthesizedAttribute(ref attributes, moduleBuilder.SynthesizeNullableAttribute(this, this.ReturnType)); [](start = 16, length = 106)
It looks like this line is going to add a duplicate attribute, the base class adds one too. Please add a unit-test to ensure this is not happening. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added ExpressionCompilerTests.EmitNullableAttribute_ExpressionType
test. That test is failing for a distinct reason however.
// or IEnumerator<T>, that is not an issue. | ||
AssertNoNullableAttribute(property.GetAttributes()); | ||
var method = property.GetMethod; | ||
AssertNoNullableAttribute(method.GetReturnTypeAttributes()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AssertNoNullableAttribute(method.GetReturnTypeAttributes()); [](start = 20, length = 60)
This feels unexpected. Does it mean that we are dropping nullability of return type on the symbol itself? What happens when nullable modifier is nested, i.e. the type is an array of nullable objects? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like we are dropping nullability on the top-level type. Added a test for IEnumerable<object?[]>
and a // PROTOTYPE
comment for now.
Done with review pass (iteration 14). #Closed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (iteration 15).
…dotnet#22820) Use synthesized attribute infrastructure for System.NullableAttribute Misc. Misc. Misc. Misc. Misc. Misc. PR feedback Override in PEMethodSymbol
No description provided.