Extra memory used when attempting to JIT generics that have one or more structs as a generic type parameter or when accessed through an interface.

[jmaine]

Extra memory used when attempting to JIT generics that have one or more structs as a generic type parameter or when accessed through an interface. The idea is to reduce the number of times a new JITed method is created in both AOT and non-AOT environments. This may also be known as Universal Shared Code.

Description

Extra memory used when attempting to JIT generics that have one or more structs as a generic type parameter or when accessed through an interface. Secondly, it bloats binaries that generated as an AOT binary. Moreover, it bloats the memory used in non-AOT environments.

The proposal is to logically pass the struct into generic methods, methods in generic classes, and methods accessed by interfaces by reference/pointer behind the scenes without changing any source or IL code or boxing the struct on the heap. If done correctly, it can reduce and possibly eliminate quite a number of peeled methods when JITing and/or boxing.

In the case of a generic class with a struct as a type parameter, the JIT will add/use extra information provided in the class to make methods that can, in this case, be reused across multiple structs.

Out of Scope

Any change that requires a change in the IL or source code of any existing programs/libraries.

Configuration

public interface IMyInterface {
     int X;
     int Y;
}
public struct MyStruct : IMyInterface {
   public int X;
   public int Y;
}

public class MyClass {
    public void MyGenericMethod<T>(T paramater) {
         
    }

    public void MyInterfaceMethod(IMyInterface parameter) {

    }
} 

public class MyGenericClass<T> {
    T MyExampleProperty { get; set; }
}

Regression?

This is an enhancement to the .Net Core and Mono JIT.

Questions?

• Is this request asking to have Universal Shared Generics feature added into the .Net Core's JIT library? Yes.
• What would be the performance characteristics of the enhancements?
• It is very likely that primitive structs may not benefit from this enhancement, but larger structs could.
  • What would be the cutoff point where this enhancement starts becoming beneficial?
• How would we keep expected behavior when modifying structs?
  • Should we do a copy-on-write when a struct is written to when this enhancement is used?
    • Can this copy-on-write feature be implemented to be lazy in nature?
    • What would be the performance characteristics of this copy-on-write feature?
  • Should we only use this enhancement on read-only or effectively read-only structs?

Updates

Alternate solutions:

• Add trapline code that can jump to the specialized method efficiently.
  • For generics with a single type parameter, use a jump table simular to calling a virtual method to get to the specialized version of the method.
  • For generics with few generic type parameters, use a hash table to find the specialized implementation.
  • Otherwise, use a Universal Shared Generics implementation of the method.

[stephentoub]

There is a lot of code now, in the core libraries, in ASP.NET, and elsewhere, that relies on the generic specialization done for value types to get good throughput / reduced memory allocation. A blanket change to how this is done would result in massive regressions.

(For AOT and an explosion in the number of instantiations, solutions like Universal Shared Generics exist.)

[jmaine]

@stephentoub, the idea is not to do a blanket change but to provide a way to be able to use MethodInfo.MakeGenericMethod(Type[]) and Type.MakeGenericType(Type[]) in AOT environments and use Universal Shared Generics where the size of the struct would make it efficient.

[AndyAyersMS]

Mono already has a feature like this.

For .Net Core, sharing decisions are made by the runtime, so am going to change the area label.