Proposal: "Generic" fixed buffer with different sizes

[ygc369]

Example:

struct A<T, N> where N: const //N must be a compile-time const number
{
    fixed T buffer[N]; // generic fixed buffer, element type is T, element count is N
};

class B<N> where N: const
{
    A<long, N> field;
};

int main()
{
    A<uint, 10> a;
    A<object, 20> a1;
    var b = new B<15>();
}

//furtherly, class should also support fixed buffer directly.
class C<N> where N: const
{
    fixed byte buffer[N];
};

var c = new C<18>(); 

[orthoxerox]

#357

[ygc369]

@orthoxerox
seems not same.

[hez2010]

Seems like template meta-programming in C++:
meta-programming

Which will make C# be able to do a lot of fun but useless things haha.

Why not put buffer size in its constructor parameters?

struct A<T> where T: new()
{
    public A(int size) { buffer = new T[size]; }
    T[] buffer;
};

class B
{
    public B(int size) { field = new A<long>(size); }
    A<long> field;
};

int main()
{
    A<uint> a = new A<uint>(10);
    A<object> a1 = new A<object>(20);
    var b = new B(15);
}

[ygc369]

@hez2010
I want the buffer to be embedded in the struct/class, thus reducing object number in the heap, reducing the pressure of GC.

[HaloFour]

This would involve quite a few CLR changes for what seems to be limited benefit. Every version of the constructed generic with a different constant would be incompatible with the next, and that would lead to massive metadata bloat at runtime. With Span<T> and Memory<T> already added to the language what additional benefit would something like this proposal bring?

I'm also pretty sure that this has been requested before, I'll see if I can find it.

[Joe4evr]

@HaloFour Found it: #1185. No surprise seeing who opened that thread.

[HaloFour]

@Joe4evr

I thought that there was a proposal that specifically combined the idea of const int generic type parameters with fixed buffers, but perhaps not.

[tannergooding]

There is my own proposal for fixed-sized buffer types: #78, which formed the basis for the existing championed proposal: #1314

There has also been discussion about extending that further to allow "value arrays", but I don't believe there has been a championed issue yet.

One of the concerns here is the metadata bloat required to support arbitrary T, so I opened https://github.com/dotnet/coreclr/issues/23408 as a suggestion of how to deal with this via some basic runtime support in the VM layer. There was then some additional discussion in that thread about just expanding the generics to allow numeric constants.

However, there are some limitations with the generic approach, such as not working with pointers or ref structs; where-as the attribute based approach would (as well as working with marshalling; although that is addressable for the generic form as well).

[HaloFour]

@tannergooding

Sounds interesting, but it looks like the size of the buffer is still a constant defined at compile time? The only mention of a const int generic type seemed to be represented by a different struct for literally every value, e.g. _1, _2? Would that be a detail hidden by the CLR or would C# actually have to be able to reference once of these billions of potential structs from somewhere?

[tannergooding]

Sounds interesting, but it looks like the size of the buffer is still a constant defined at compile time?

Yes, I don't think you can reliably have dynamically sized "value arrays" that escape the scope of the current method. That is, the space needs to be allocated at the scope of highest use; and if it's dynamic, that size also needs to be determined at the same scope. So, you either need to have a constant size, or manually allocate the space and pass the Span<T> around.

The only mention of a const int generic type seemed to be represented by a different struct for literally every value, e.g. _1, _2? Would that be a detail hidden by the CLR or would C# actually have to be able to reference once of these billions of potential structs from somewhere?

That I'm unsure of; it didn't go into much further detail and given that it still requires a compile time constant, but also doesn't work in all scenarios; I think it is much more limiting than the attribute based approach.

[Korporal]

Seems like template meta-programming in C++:
meta-programming

Which will make C# be able to do a lot of fun but useless things haha.

Why not put buffer size in its constructor parameters?

struct A<T> where T: new()
{
    public A(int size) { buffer = new T[size]; }
    T[] buffer;
};

class B
{
    public B(int size) { field = new A<long>(size); }
    A<long> field;
};

int main()
{
    A<uint> a = new A<uint>(10);
    A<object> a1 = new A<object>(20);
    var b = new B(15);
}

@hez2010

Your example isn't using fixed buffers but a 1D array with element type T. Also fixed buffers currently require the bounds arg to be a compile time constant.

[aka-nse]

How about such as following?

using System;
using System.Runtime.InteropServices;

[StructLayout(LayoutKind.Sequential, Size = 1)] public struct __1 {}
[StructLayout(LayoutKind.Sequential, Size = 2)] public struct __2 {}
[StructLayout(LayoutKind.Sequential, Size = 3)] public struct __3 {}
[StructLayout(LayoutKind.Sequential, Size = 4)] public struct __4 {}
[StructLayout(LayoutKind.Sequential, Size = 5)] public struct __5 {}
[StructLayout(LayoutKind.Sequential, Size = 6)] public struct __6 {}
[StructLayout(LayoutKind.Sequential, Size = 7)] public struct __7 {}
[StructLayout(LayoutKind.Sequential, Size = 8)] public struct __8 {}
//  :
//  :

using System;
using System.Runtime.CompilerServices;


public struct Sample<N> where N : unmanaged
{
    public unsafe struct __Buffer
    {
        private N _Value;
        public ref byte this[int index]
            => ref ((byte*)Unsafe.AsPointer(ref _Value))[index];
    }
    
    private __Buffer FixedFieldLike;

    // You can access it as if fixed size buffer.
    public ref byte this[int index]
    {
        get
        {
            if((uint)Unsafe.SizeOf<N>() < (uint)index)
                throw new ArgumentOutOfRangeException();

            return ref FixedFieldLike[index];
        }
    }
}


public static class Program
{
    public static void Main()
    {
        var sample = new Sample<__7>();
        // :
        // :
    }
}

This can be run on current runtime.
Generating __7 and __Buffer as the result of syntax sugar processing does not seem difficult.
The unmanaged types which are structurally same are compatible (on Unsafe.As) so I think there is no problem in assembly portability.

[ygc369]

similiar to #749

[AartBluestoke]

Earlier there was an issue expressed about the potentially large number of classes this would create.
Given that this is an opt-in cost, why would that be an issue?

Also, it appears people are doing this anyway, by hand.

If that's a big concern, add a compiler warning "more than 100 specialisations of generic X detected, constant may not be constant"

[ygc369]

@AartBluestoke
Good point!