Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make config binding gen incremental #89587

Merged
merged 11 commits into from
Sep 27, 2023
8 changes: 6 additions & 2 deletions src/libraries/Common/src/Roslyn/DiagnosticDescriptorHelper.cs
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using Microsoft.CodeAnalysis;

namespace Microsoft.CodeAnalysis.DotnetRuntime.Extensions
{
internal static partial class DiagnosticDescriptorHelper
Expand All @@ -21,5 +19,11 @@ public static DiagnosticDescriptor Create(

return new DiagnosticDescriptor(id, title, messageFormat, category, defaultSeverity, isEnabledByDefault, description, helpLink, customTags);
}

/// <summary>
/// Creates a copy of the Location instance that does not capture a reference to Compilation.
/// </summary>
public static Location GetTrimmedLocation(this Location location)
eiriktsarpalis marked this conversation as resolved.
Show resolved Hide resolved
=> Location.Create(location.SourceTree?.FilePath ?? "", location.SourceSpan, location.GetLineSpan().Span);
}
}
44 changes: 44 additions & 0 deletions src/libraries/Common/src/SourceGenerators/DiagnosticInfo.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Linq;
using System.Numerics.Hashing;
using Microsoft.CodeAnalysis;

namespace SourceGenerators;

/// <summary>
/// Descriptor for diagnostic instances using structural equality comparison.
/// Provides a work-around for https://github.com/dotnet/roslyn/issues/68291.
/// </summary>
internal readonly struct DiagnosticInfo : IEquatable<DiagnosticInfo>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise, I believe the regex generator defines such a struct as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At a glance this seems like a type that would be defined as a record if arrays were naturally equitable. Why wasn't this just defined as a record with ImmutableEquatableArray<object?> wrapping MessageArgs?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would force an intermediate allocation from object[] to ImmutableEquatableArray then back to an object[] when the Diagnostic gets materialized. I think it's small enough to justify a bespoke type. Longer term though we should be looking at addressing dotnet/roslyn#68291 and use Diagnostic in the equatable models directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diagnostic will never be ok to use in a model. Location has a reference to the SyntaxTree that it was created from, and that will change every time the file changes. This wrapper has similar issues: even if message args compared correctly, it would still be broken for equality.

Copy link
Contributor Author

@layomia layomia Sep 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diagnostic will never be ok to use in a model.

Per #89587 (comment) this bit is mitigated for this PR, follow up in #92509.

{
public required DiagnosticDescriptor Descriptor { get; init; }
public required object?[] MessageArgs { get; init; }
public required Location? Location { get; init; }
layomia marked this conversation as resolved.
Show resolved Hide resolved

public Diagnostic CreateDiagnostic()
=> Diagnostic.Create(Descriptor, Location, MessageArgs);

public override readonly bool Equals(object? obj) => obj is DiagnosticInfo info && Equals(info);

public readonly bool Equals(DiagnosticInfo other)
{
return Descriptor.Equals(other.Descriptor) &&
MessageArgs.SequenceEqual(other.MessageArgs) &&
Location == other.Location;
}

public override readonly int GetHashCode()
{
int hashCode = Descriptor.GetHashCode();
foreach (object? messageArg in MessageArgs)
{
hashCode = HashHelpers.Combine(hashCode, messageArg?.GetHashCode() ?? 0);
}

hashCode = HashHelpers.Combine(hashCode, Location?.GetHashCode() ?? 0);
return hashCode;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Numerics.Hashing;

namespace SourceGenerators
{
/// <summary>
/// Provides an immutable list implementation which implements sequence equality.
/// </summary>
public sealed class ImmutableEquatableArray<T> : IEquatable<ImmutableEquatableArray<T>>, IReadOnlyList<T>
layomia marked this conversation as resolved.
Show resolved Hide resolved
where T : IEquatable<T>
{
public static ImmutableEquatableArray<T> Empty { get; } = new ImmutableEquatableArray<T>(Array.Empty<T>());

private readonly T[] _values;
public T this[int index] => _values[index];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public T this[int index] => _values[index];
public ref T this[int index] => ref _values[index];

This provides a more array like semantic

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it also makes it mutable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the goal is immutable then make the return readonly ref.

If the goal is immutalbe though then why are we inroducing record with set members?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should aim for the model being immutable everywhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW it seems readonly ref isn't supported on indexers yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, should be ref readonly not readonly ref

public int Count => _values.Length;

public ImmutableEquatableArray(IEnumerable<T> values)
=> _values = values.ToArray();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you consider just making the parameter type here T[] so that consumer can potentially avoid an allocation?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes a defensive copy to avoid encapsulating a potentially mutable source enumerable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct but if you make the parameter T[] there is no mutable source enumerable. The caller is forced to allocate the array and in the case the collection is already an array we don't double alloc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but the array passed in by the caller could theoretically be mutated (e.g. it could be passing a rented buffer). Not super likely to happen here though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is all internal, i would make this just a T[]. You know your creators and can ensure they're doing the right thing.


public bool Equals(ImmutableEquatableArray<T>? other)
=> other != null && ((ReadOnlySpan<T>)_values).SequenceEqual(other._values);

public override bool Equals(object? obj)
=> obj is ImmutableEquatableArray<T> other && Equals(other);

public override int GetHashCode()
{
int hash = 0;
foreach (T value in _values)
{
hash = HashHelpers.Combine(hash, value is null ? 0 : value.GetHashCode());
}

return hash;
}

public Enumerator GetEnumerator() => new Enumerator(_values);
IEnumerator<T> IEnumerable<T>.GetEnumerator() => ((IEnumerable<T>)_values).GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => _values.GetEnumerator();

public struct Enumerator
{
private readonly T[] _values;
private int _index;

internal Enumerator(T[] values)
{
_values = values;
_index = -1;
}

public bool MoveNext()
{
int newIndex = _index + 1;

if ((uint)newIndex < (uint)_values.Length)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why these uint casts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider:

=> ++_index < _values.Length;

Consider extracting out _values.Length to a fiel.

{
_index = newIndex;
return true;
}

return false;
}

public readonly T Current => _values[_index];
}
}

internal static class ImmutableEquatableArray
layomia marked this conversation as resolved.
Show resolved Hide resolved
{
public static ImmutableEquatableArray<T> Empty<T>() where T : IEquatable<T>
=> ImmutableEquatableArray<T>.Empty;

public static ImmutableEquatableArray<T> ToImmutableEquatableArray<T>(this IEnumerable<T> values) where T : IEquatable<T>
=> new(values);
}
}
59 changes: 59 additions & 0 deletions src/libraries/Common/src/SourceGenerators/TypeModelHelper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

using Microsoft.CodeAnalysis;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;

namespace SourceGenerators
{
Expand Down Expand Up @@ -32,5 +34,62 @@ void TraverseContainingTypes(INamedTypeSymbol current)
}
}
}

public static string GetFullyQualifiedName(this ITypeSymbol type) => type.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat);

/// <summary>
/// Removes any type metadata that is erased at compile time, such as NRT annotations and tuple labels.
layomia marked this conversation as resolved.
Show resolved Hide resolved
/// </summary>
public static ITypeSymbol EraseCompileTimeMetadata(this Compilation compilation, ITypeSymbol type)
layomia marked this conversation as resolved.
Show resolved Hide resolved
{
if (type.NullableAnnotation is NullableAnnotation.Annotated)
{
type = type.WithNullableAnnotation(NullableAnnotation.None);
}

if (type is INamedTypeSymbol namedType)
{
if (namedType.IsTupleType)
{
if (namedType.TupleElements.Length < 2)
{
return type;
}

ImmutableArray<ITypeSymbol> erasedElements = namedType.TupleElements
.Select(e => compilation.EraseCompileTimeMetadata(e.Type))
.ToImmutableArray();

type = compilation.CreateTupleTypeSymbol(erasedElements);
}
else if (namedType.IsGenericType)
{
if (namedType.IsUnboundGenericType)
{
return namedType;
}

ImmutableArray<ITypeSymbol> typeArguments = namedType.TypeArguments;
INamedTypeSymbol? containingType = namedType.ContainingType;

if (containingType?.IsGenericType == true)
{
containingType = (INamedTypeSymbol)compilation.EraseCompileTimeMetadata(containingType);
type = namedType = containingType.GetTypeMembers().First(t => t.Name == namedType.Name && t.Arity == namedType.Arity);
}

if (typeArguments.Length > 0)
{
ITypeSymbol[] erasedTypeArgs = typeArguments
.Select(compilation.EraseCompileTimeMetadata)
.ToArray();

type = namedType.ConstructedFrom.Construct(erasedTypeArgs);
}
}
}

return type;
}
}
}
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Diagnostics;
using Microsoft.CodeAnalysis;

namespace System.Text.Json.SourceGeneration
namespace SourceGenerators
{
/// <summary>
/// An equatable value representing type identity.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using Xunit;

namespace SourceGenerators.Tests
{
internal static class GeneratorTestHelpers
{
/// <summary>
/// Asserts for structural equality, returning a path to the mismatching data when not equal.
/// </summary>
public static void AssertStructurallyEqual<T>(T expected, T actual)
{
CheckAreEqualCore(expected, actual, new());
static void CheckAreEqualCore(object expected, object actual, Stack<string> path)
{
if (expected is null || actual is null)
{
if (expected is not null || actual is not null)
{
FailNotEqual();
}

return;
}

Type type = expected.GetType();
if (type != actual.GetType())
{
FailNotEqual();
return;
}

if (expected is IEnumerable leftCollection)
{
if (actual is not IEnumerable rightCollection)
{
FailNotEqual();
return;
}

object?[] expectedValues = leftCollection.Cast<object?>().ToArray();
object?[] actualValues = rightCollection.Cast<object?>().ToArray();

for (int i = 0; i < Math.Max(expectedValues.Length, actualValues.Length); i++)
{
object? expectedElement = i < expectedValues.Length ? expectedValues[i] : "<end of collection>";
object? actualElement = i < actualValues.Length ? actualValues[i] : "<end of collection>";

path.Push($"[{i}]");
CheckAreEqualCore(expectedElement, actualElement, path);
path.Pop();
}
}

if (type.GetProperty("EqualityContract", BindingFlags.Instance | BindingFlags.NonPublic, null, returnType: typeof(Type), types: Array.Empty<Type>(), null) != null)
{
// Type is a C# record, run pointwise equality comparison.
foreach (PropertyInfo property in type.GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
path.Push("." + property.Name);
CheckAreEqualCore(property.GetValue(expected), property.GetValue(actual), path);
path.Pop();
}

return;
}

if (!expected.Equals(actual))
{
FailNotEqual();
}

void FailNotEqual() => Assert.Fail($"Value not equal in ${string.Join("", path.Reverse())}: expected {expected}, but was {actual}.");
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System.Collections.Immutable;
using Microsoft.CodeAnalysis;
using SourceGenerators;

Expand All @@ -11,19 +10,22 @@ public sealed partial class ConfigurationBindingGenerator : IIncrementalGenerato
{
private sealed partial class Emitter
{
private readonly SourceProductionContext _context;
private readonly SourceGenerationSpec _sourceGenSpec;
private readonly InterceptorInfo _interceptorInfo;
private readonly BindingHelperInfo _bindingHelperInfo;
private readonly TypeIndex _typeIndex;

private readonly SourceWriter _writer = new();

public Emitter(SourceProductionContext context, SourceGenerationSpec sourceGenSpec)
public Emitter(SourceGenerationSpec sourceGenSpec)
{
_context = context;
_sourceGenSpec = sourceGenSpec;
_interceptorInfo = sourceGenSpec.InterceptorInfo;
_bindingHelperInfo = sourceGenSpec.BindingHelperInfo;
_typeIndex = new TypeIndex(sourceGenSpec.ConfigTypes);
}

public void Emit()
public void Emit(SourceProductionContext context)
{
if (!ShouldEmitBindingExtensions())
if (!ShouldEmitMethods(MethodsToGen.Any))
{
return;
}
Expand Down Expand Up @@ -52,7 +54,7 @@ file static class {{Identifier.BindingExtensions}}

EmitEndBlock(); // Binding namespace.

_context.AddSource($"{Identifier.BindingExtensions}.g.cs", _writer.ToSourceText());
context.AddSource($"{Identifier.BindingExtensions}.g.cs", _writer.ToSourceText());
}

private void EmitInterceptsLocationAttrDecl()
Expand All @@ -79,7 +81,7 @@ public InterceptsLocationAttribute(string filePath, int line, int column)

private void EmitUsingStatements()
{
foreach (string @namespace in _sourceGenSpec.Namespaces.ToImmutableSortedSet())
foreach (string @namespace in _bindingHelperInfo.Namespaces)
{
_writer.WriteLine($"using {@namespace};");
}
Expand Down
Loading
Loading