Discussion: Code Generator Catalog

[alrz]

Here is an incomplete list of potential use cases for code generators to explore ideas and discussion. I believe this will help to shape generator APIs and language features around it.

1. NPC implementation (huh)

partial class Person
{
  public string FirstName { get; set; }
  public string LastName { get; set; }
}

// generated
partial class Person : INotifyPropertyChanged
{
  private string _firstName;
  public replace string FirstName
  {
    get => _firstName;
    set 
    {
      if (_firstName != value)
      {
         _firstName = value;
         OnPropertyChanged(nameof(FirstName));
      }
    }    
  }
}

Also, if a property depends on others, we could also raise additional events by inspecting its accessor.

[NPC]
public string FullName => this.FirstName + ", " + this.LastName;

// generated
public replace string FirstName
{
  get => _firstName;
  set 
  {
    if (_firstName != value)
    {
       _firstName = value;
       OnPropertyChanged(nameof(FirstName));
       OnPropertyChanged(nameof(FullName));
    }
  }
}

Note, #850 can lead to simpler code generation i.e does not need to figure out a name for backing field:

public replace string FirstName
{
  string field;
  get => field;
  set 
  {
    if (field != value)
    {
      field = value;
      OnPropertyChanged(nameof(FirstName));
      OnPropertyChanged(nameof(FullName));
    }
  }
}

Or the type (#8364):

public replace string FirstName
{
  get;
  set 
  {
    if (field != value)
    {
      field = value;
      OnPropertyChanged(nameof(FirstName));
      OnPropertyChanged(nameof(FullName));
    }
  }
}

Note: If none of replaced members call original the original declaration should be removed as the backing fields are no longer being used.

2. Dependency Injection

Dependency containers commonly depend on reflection to create objects. This could be moved to compile-time via code generators. In this scenario, there should be a way to parametrize the code generator to switch between implementations. e.g. mocks. (attributes does not belong to MEF).

[Export]
class Service1 : IService1
{
  [Import]
  private IService2 Service2 { get; }
}

Note that this will be only useful to manage object lifecycle. If implementations come from outside of assembly boundaries we should probably fallback to reflection under the hood.

3. Caching / Lazy Initialization

[Cache]
public string Property => ComputeProperty();

// generated
private string _property;
public replace string Property => _property ?? (_property = original);

4. Memoization

[Memoize]
public string Function(string arg) { ... }

// mind you, a simple demonstration without thread-safety
private readonly Dictionary<string, string> _fCache = new();
public replace string F(string arg)
  => _fCache.TryGetValue(arg, out var result) ? result : _fCache[arg] = original(arg);

5. Dependency Properties

partial class Foo : DependencyObject
{
  [PropertyMetadata(defaultValue: string.Empty)]
  public string Name { get; set; }
  public int Size { get; }
}

// generated
partial class Foo 
{
  public static readonly DependencyProperty NameProperty =
    DependencyProperty.Register(nameof(Name), typeof(string), typeof(Foo), new(string.Empty));
    
  public replace string Name
  {
    get => (string)GetValue(NameProperty);
    set => SetValue(NameProperty, value);
  }
  
  internal static readonly DependencyPropertyKey SizePropertyKey =
        DependencyProperty.RegisterReadOnly(nameof(ReadonlyNam), typeof(string), typeof(Foo));

  public static readonly DependencyProperty SizeProperty = SizePropertyKey.DependencyProperty;
    
  public replace int Size
  {
    get => (int)GetValue(SizeProperty);
    internal set => SetValue(SizePropertyKey, value);
  }
}

Note, it would be nice to be able to inspect property initializer and use it in the generated code, e.g.

public string Name { get; set; } = string.Empty;

6. ORMs

Currently ORMs) use runtime code generation (NH) or proxies (EF) to enable change tracking and lazy loading in POCOs. They could ship with a code generator to move this procedure to the compile-time.

class BlogPost
{
   public string Title { get; set; }
   public string Body { get; set; }
   public List<Comment> Comments { get; }
}

7. Mixins

It's possible to implement member delegation as an analyzer but if we want to delegate all members, perhaps a code generator is more likely preferable.

partial class Class
{
  [Mixin] private readonly ISomething _something = new Something();
}

// generated
partial class Class : ISomething
{
  public void DoSomething() => _something.DoSomething();
}

8. Double Dispatch

This could be used to implement visitor pattern or a simple double dispatch:

public object F(T x) { .. }
public object F(U x) { .. }
[DoubleDispatch]
public extern object F(Base p);

// generated
public object F(Base p)
{
  switch(p)
  {
    case T x: return F(x);
    case U x: return F(x);
    default: throw ExceptionHelper.UnexpectedValue(p);
  }
}

You could use a similar technique to generate semi-virtual extension methods. Since this is generated by a generator, you are free to handle the failure case differently.

Generators should be able to produce diagnostics if target members are malformed, e.g. a method instead of a property.

9. Duck Typing

interface IDuck
{
  void Quack();
}

[Duck(typeof(IDuck))]
class A { public void Quack() {} }

void F(IDuck d) { }

F(new A().Wrap());

// generated
static class Extensions
{
  class WrapperA : IDuck
  {
    private readonly A _obj;
    public WrapperA(A obj) => _obj = obj;
    public Quack() => obj.Quack();
  }
  public static IDuck Wrap(this A obj) => new WrapperA(_obj);
}

Though, #11159 + #258 = #8127 can greatly improve this scenario in terms of perf and easier code gen.

Note: An assembly attribute could be used to annotate exterior types,

[assembly: Duck(typeof(IDuck), typeof(AnotherAssembly.B))]

10. Type providers for xml, json, csv, sql, etc

F# type providers can take a string as parameter to bootstrap code generation. This requires code generators to accept a parameter which is not possible in #5561.

11. Variadic generics

There are types with variable arities including Func, Action and ValueTuple. It is a common scenario where we want to have a bunch of similar methods that are only different in number of generic types.

12. Basic implementation

Some of interfaces like IEquitable, IComparable etc, given "key properties" could be implemented via generators . This can also be implemented as an analyzer, but with generators it would be totally transparent. ToString overrides also belong to this category.

13. Serialization

Serialization to various formats like json, can be provided at compile-time without using reflection.

[HaloFour]

I get where you're going with extern but that seems like it would add a big layer of complexity given the member then must be replaced but the replacement must not call original. With that as a possibility the conerns of ordering becomes much more important.

[alrz]

@HaloFour Only if you don't mind the additional backing field per property (or having to return default for methods.) The latter wouldn't be that much trouble (if you don't have out parameters) but the former would definitely affect memory usage. Note that extern is just a non-ambigious alternative to #9178 which was originaly mentioned in #5292 (comment).

[HaloFour]

@alrz

The partial keyword might also work there given that's sort of how it is used today, although with numerous behavioral differences and the compiler eliding calls if never implemented.

Either way, I don't disagree with the concept, just mentioning the additional complexity it creates. Order of member replacement is already a known concern.

[alrz]

The compiler should remove the original declaration if none of replaced members called original. This could address the problem with additional backing field but I think extern is still useful on methods.

/cc @HaloFour I've updated OP to reflect this.

[bbarry]

One could implement IDisposable via a generator:

[GenerateDisposable]
public partial class Resource : IDisposable   
{
    [Dispose]
    private IntPtr nativeResource; 
    [Dispose]
    private AnotherResource managedResource;

    ...
}

---

Generating async (code lifted from https://github.com/npgsql/npgsql, idea courtesy @roji)

[RewriteAsync]
void SendClosureAlert()
{
    _buf[0] = (byte)ContentType.Alert;
    Utils.WriteUInt16(_buf, 1, (ushort)_connState.TlsVersion);
    _buf[5 + _connState.IvLen] = (byte)AlertLevel.Warning;
    _buf[5 + _connState.IvLen + 1] = (byte)AlertDescription.CloseNotify;
    int endPos = Encrypt(0, 2);
    _baseStream.Write(_buf, 0, endPos);
    _baseStream.Flush();
}

//generated:
async Task SendClosureAlertAsync(CancellationToken cancellationToken)
{
    _buf[0] = (byte)ContentType.Alert;
    Utils.WriteUInt16(_buf, 1, (ushort)_connState.TlsVersion);
    _buf[5 + _connState.IvLen] = (byte)AlertLevel.Warning;
    _buf[5 + _connState.IvLen + 1] = (byte)AlertDescription.CloseNotify;
    int endPos = Encrypt(0, 2);
    await _baseStream.WriteAsync(_buf, 0, endPos, cancellationToken);
    await _baseStream.FlushAsync(cancellationToken);
}

---

Allocation free yield (ignore the overflow issue)

public partial class MyFib
{
    public partial struct FibIter {}
    [Iterator(typeof(FibIter), "Fib", BindingFlags.Public)]
    private IEnumerable<int> GenFib()
    {
        int j = 0, i = 1;
        while(true)
        {
            yield return i;
            j += i;
            yield return j;
            i += j;
        }
    }
}

// generated
public partial class MyFib
{
    public FibIter Fib()
    {
        return default(FibIter);
    }
    public partial struct FibIter: IEnumerable<int>, IEnumerator<int>
    {
        private int _1, _2, j, i;
        public FibIter GetEnumerator() => this;
        public int Current => _2;
        public bool MoveNext()
        {
            switch (_1)
            {
            case 0:
                this._1 = -1;
                this.j = 0;
                this.i = 1;
                break;
            case 1:
                this._1 = -1;
                this.j += this.i;
                this._2 = this.j;
                this._1 = 2;
                return true;
            case 2:
                this._1 = -1;
                this.i += this.j;
                break;
            default:
                return false;
            }
            this._2 = this.i;
            this._1 = 1;
            return true;
        }
        object IEnumerator.Current => _2;
        void IDisposable.Dispose() {}
        void IEnumerator.Reset()=> throw new NotSupportedException();
        IEnumerator<int> IEnumerable<int>.GetEnumerator() => this;
        IEnumerator IEnumerable.GetEnumerator() => this;
    }
}

A very similar thing could be done to implement async differently and save allocations, for example #15491 (comment)

[alrz]

@bbarry Worth to mention: #10449.

[iam3yal]

@alrz you can add #15282 to the list. :)

[ufcpp]

Records

This is a temporary solution until Records are released, but,
I have an generator (by using Analyzer and Code Fix) for Record-like purpose:

https://github.com/ufcpp/RecordConstructorGenerator

Type aliases/Strong typedef

[Alias]
struct Name { string value; }
[Alias]
struct Distance { double value; }

// generated

partial struct Name
{
    public string Value => value;
    public Name(string value) { this.value = value; }
    // equality, cast operator, etc.
}

partial struct Distance
{
    public double Value => value;
    public Distance(double value) { this.value = value; }
    // equality, cast operator, etc.
}

Mixins

For mixin use cases, I want an ability to pass this pointer to the mixin.

public class Sample
{
    NotifyPropertyChangedMixin _npc;
}

// generated
public class Sample : INotifyPropertyChanged
{
    NotifyPropertyChangedMixin _npc;

    public event PropertyChangedEventHandler PropertyChanged { add { _npc.PropertyChanged += value; } remove { _npc.PropertyChanged -= value; } }

    protected void OnPropertyChanged(PropertyChangedEventArgs args) => _npc.OnPropertyChanged(this, args);

    protected void OnPropertyChanged([CallerMemberName] string propertyName = null) => _npc.OnPropertyChanged(this, propertyName);

    protected void SetProperty<T>(ref T storage, T value, PropertyChangedEventArgs args) => _npc.SetProperty(this, ref storage, value, args);

    protected void SetProperty<T>(ref T storage, T value, [CallerMemberName] string propertyName = null) => _npc.SetProperty(this, ref storage, value, propertyName);
}

[asdfgasdfsafgsdfa]

Personally I would love having a generator write serialization functions for my objects.
That should easily be possible as well, right?

How would one go about debugging such generated code tho?
I think it would be very important to be able to preview, inspect, debug(including edit and continue) all generated code. Would the additional code be generated into "nested files" (like T4 generated results or code generated by the winforms designer) ?

Manipulating code-text directly might not be the best approach though, so we would end up manipulating the Ast?

So this feature would most likely be some kind of more thightly integrated T4 system (or a variant using the AST)?
What alternatives are there that don't prevent inspection and live debugging of the code?

This could also be utilized in some obfuscators maybe...
To that end users might want to debug the obfuscated versions in two ways:
Viewing and stepping through the code as if it were unmodified (even though the code might be heavily obfuscated by some post processing code generator) and on the other hand someone might want to view the code as it was really compiled, maybe to find some bug in the obfuscation...

Thoughts?

[alrz]

@asdfgasdfsafgsdfa You can refer to the original proposal here: #5561. It would not manipulate AST or anything, it is a "code generator" so it can only add code to the compilation unit. Debugging works directly on the generated code.

[asdfgasdfsafgsdfa]

@alrz
I'm not sure I completely understand.
In the original discussion people mentioned that dealing with strings alone would be somewhat of a hassle.
Same with replacing vs. adding code.

I would rather see replacement instead of (or in addition to) addition of code and AST modification instead of manipulating code-strings. Should I focus on #5561 then?
This issue ( #16160 ) doesn't close or supersede the discussion over at #5561 right? It is just an alternative approach to solve a similar problem, right?

Oh and another quick question: Has there been any official opinion on those questions? I saw it on a "strong interest" list somewhere, but as I understand the actual implementation and usage is still pretty unclear and heavily discussed. Have any of the people that will decide in the end mentioned their own opinion on those more detailed questions?

[fanoI]

One other possible use of Generators could be for bitfield structs this implementation used reflection and so was 10'000 times slower than using bitmasks (!) but using Generators the conversion functions to ToUInt64() and ToBinaryString() could be generated at runtime:

https://www.codeproject.com/Articles/1095576/Bit-Field-in-Csharp-using-struct

[iam3yal]

Another thing that I thought about is using it in the following manner:

[Calculator(Method.Fib, 40)]
const int fib = 0;

[Calculator(Method.CubeVolume, 2)]
const int cube = 0;

[gulshan]

F# community is discussing a type provider to generate types from types. Felt kind of similar. This comment has some probable usage- fsharp/fslang-design#125 (comment)

[axel-habermaier]

Generate PInvoke Code
Code generators could also be used to generate PInvoke code from C/C++ header files.

[asdfgasdfsafgsdfa]

• Having a code generator that does two-way databinding from my settings class to my custom UI controls (when you have to replace the existing databinding solutions).

• A generator that provides your own implementation of [SyncVar] (from Unity3D) either in Unity, or in programs that have nothing to do with Unity.

[gulshan]

I want to add lightweight(local-only) Actor models like nAct and this one to the list- https://github.com/David-Desmaisons/EasyActor
They uses async method calls instead of message passing. A normal worker class attributed as [Actor] can be code generated to have basic thread management(attached to a specific thread) where the methods marked as [Actor.Message] would be put inside a async Task automatically.

[gafter]

Issue moved to dotnet/csharplang #341 via ZenHub