[Proposal]: Let ref structs implement interfaces and substitute into type parameters (VS 17.11, .NET 9)

[agocke]

Ref structs implementing interfaces

• Proposed
• Prototype: Not Started
• Implementation: Not Started
• Specification: https://github.com/dotnet/csharplang/blob/main/proposals/ref-struct-interfaces.md

Related Issues

• Runtime Feature Tracking
• Related Requests
  • generic constraint: where T : ref struct  #1148
  • Add string keyed dictionary ReadOnlySpan<char> lookup support runtime#27229

Design meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-02-26.md#ref-structs-in-generics
https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-06-10.md#ref-structs-implementing-interfaces-and-in-generics
https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-07-22.md#ref-structs-implementing-interfaces

[agocke]

@MadsTorgersen @333fred Could we get this on a triage list? It feels like we've spent more hours in LDM working around this problem than we would actually fixing it.

[huoyaoyuan]

Some comments not directly-related, but interesting to consider:

What about void as a generic parameter? It's more complicated in type system, but there's not safety problem. Variables or fields declared as void can be omitted. Comparing with ref struct, the omitted void can appear in non ref-struct types.

The unconstraint can solve the gap for concrete methods, but how about delegate types? The delegate type parameters can't be constrained, should we specialize for them?

[agocke]

Also, @jaredpar for lifetime issues. My thinking was that every parameter would implicitly carry a "heap-safe-to-escape" lifetime, but maybe there's something I'm not thinking of. Obviously people will request more lifetime stuff in the future to workaround that, but I figured that could be an additional set of language features that we do later.

[En3Tho]

Can you clarify an example with where T : ~box: how does it affect boxable types like classes? It feels to me like ~box allows ref structs only?

It somehow feels to me that this is not a constraint on a type but rather a contract of a callee: a promise of no-escape-to-the-heap. A lifetime feature rather than a property of a type itself.

[jaredpar]

Rather than where I think we should consider allow. There are more anti-constraints that have been proposed for the language including items like allowing pointers. Having a new keyword then a ~ for them feels unnatural where as the following feels much more natural to me:

void M<T>() 
  where T : IDisposable
  allow T : ref struct { 

}

We don't need much more than the above.

Disagree. As a ref struct can satisfy the where T : ~box proposal. That means that we have to consider all T which satisfy ~box to be potentially ref struct. The proposal doesn't seem to do this but rather focuses on the boxing aspect. That means without additional restrictions code like the following would be allowed:

class C<T> where T : ~box {
  T _field;
}

C<Span<int>> // not good

That is also why I prefer allow T : ref struct because it's very clear this is the capability being allowed here and would make it much clearer why the above code fails to compile (also makes it easier for us to generate better error messages).

you just want to prevent boxing.

No. You need to prevent all behaviors that are disallowed by a ref struct. Boxing is only one of them.

[agocke]

No. You need to prevent all behaviors that are disallowed by a ref struct. Boxing is only one of them.

Yeah I agree with that. And I think you're right that we might want to use different syntax. But I think the basic concept is still the same here.

[jaredpar]

There are a couple of other items.

Essentially any T constrained to allow T : ref struct needs to have all capabilities and restrictions of a ref struct:

1. Cannot be a field of a non ref struct
2. Declarations can be marked scoped
3. Is subject to lifetime tracking rules as a ref struct value would be
4. etc ...

Effectively for all intent and purposes the compiler would treat it like a ref struct value. Cause it potentially can be and that is the worst case from a restriction point of view. No harm is done to other types by being given such restrictions.

Another other item that's come up here is how such an anti-constraint should be applied to existing APIs in the core libraries. Consider for example that every variation of Func / Action should have allow T : ref struct for all their type parameters. In those cases it's a 100% upside change.

One approach to this is to simply go through and manually update every such delegate definition to have this anti-constraint. That's manual and benefits only the developers who put in the work. Another option is to simply embed it in the language rules. Essentially for a type parameter on a delegate definition where it only appears only as the type of a simple parameter (no in/out/ref modifiers) the language will automatically infer allow T : ref struct. Doing so would make the delegate more expressive than if it didn't. I've gone back and forth on this but generally lean towards automatic inferring it here. Could be talked off this point though.

Another item is that allowing ref struct to implement interfaces means we should consider allowing [UnscopedRef] attributes on interface members. Lacking that attribute there are patterns expressible in ref struct today that cannot be expressed when factored into an interface. Think that is relatively straight forward (has meaning when implemented on a struct but not on a class) but still needs to be spec'd out.

The implication of DIM on a ref struct also needs to be worked through. Consider as a concrete example:

interface I1
{
  public object M() => this;
}

// Error: implicit box in the implementation of M
ref struct S : I1 
{
}

Think the likely conclusion here is that ref struct cannot participate in DIM cause there isn't a reasonable way to prevent all ref safety issues that can come from it. That would require both compiler and runtime support.

Need to work out the rules for co / contra variance

[davidwrighton]

On top of all of those details that @jaredpar brings up, I'd really like to be able to use this anti-constraint with Span<T> so that we can do a better job around dealing with certain InlineArray scenarios, but that api is just messy.

[jaredpar]

On top of all of those details that @jaredpar brings up, I'd really like to be able to use this anti-constraint with Span so that we can do a better job around dealing with certain InlineArray scenarios, but that api is just messy.

That particular issue is a bit of a thorny problem because there are APIs in Span<T> that are illegal once the anti-constraint is added

ref struct Span<T> : allow T : ref struct {
 
  // Error 1: ref field to ref struct
  ref T data;
  int length;

  // Error 2: Can't use a ref struct as an array element 
  pulbic Span(T[] array) 
}

The error 2 type of problems are very thorny. Essentially there is existing public API surface area that directly violates what the anti-constraint would provide. There isn't a general solution to this that I can envision. Think the way to move forward here would be that the compiler simply ignores them. Essentially Span<T> is a primitive and we special case certain APIs as being illegal to call when we know T is potentially a ref struct. That would classify as hard on my brain but solvable.

The error 1 type of problems are more fundamental. There is a proposal out for allowing ref fields of ref struct but it's a decently complex feature.

[TahirAhmadov]

I think the notion of anti-constraints needs to be considered very thoroughly. It's almost a full on massive change all on its own. What do ~class, ~struct, ~new, etc. mean? ~notnull becomes weird, because it's a double negative. What about types (classes and interfaces)? I think I saw that brought up before, but I don't remember the discussion and why it was desirable.

The proposed allow keyword is also a little finicky. Does allow only allow (pun intended) ref struct? Are there any potential other scenarios? Why not just add ref struct to the list of where - surely adding a new keyword here is overkill?

[HaloFour]

I think the notion of anti-constraints needs to be considered very thoroughly

I don't think they're intended to be a general purpose feature. I think they're only to be used where they enable the language to support additional functionality, but they also don't make sense as a normal constraint. A ref struct constraint doesn't force the generic type argument to be a ref struct, it forces the code of the method to follow the rules as if it could be a ref struct, but the argument could be a normal struct (or even a class?)

[jaredpar]

Put up a full proposal for this feature

https://github.com/dotnet/csharplang/blob/main/proposals/ref-struct-interfaces.md

[timcassell]

Another alternative that was mentioned in the old issue is placing the ref keyword at the generic declaration site, much like in and out on co/contravariant interfaces.

public void M<ref T>(T o) where T : IDisposable
    => o.Dispose();

[CyrusNajmabadi]

My main issue there @timcassell is that reads to me like "this must be a ref struct" as opposed to "this is allowed to be a ref-struct". That said, we'll def consider syntactic possibilities here when designing this out.

[TahirAhmadov]

What about where T : allow ref struct, ISomeInterface? It's kind of strange to have 2 places with constraints and "expanders" for one generic argument.

[CyrusNajmabadi]

The syntax here is the least interesting part :) We'll likely consider a bunch and settle on one the group feels conveys the idea the best and feels the most c#-y :)

[jaredpar]

@timcassell

Another alternative that was mentioned in the old issue is placing the ref keyword at the generic declaration site, much like in and out on co/contravariant interfaces.

For me that is much too easily confused with supporting the ability to have ref as a type modifier. For example List<ref int>. That is a much different feature (and not one that I think anyone is considering given the fundamental change it would require to the runtime).

What about where T : allow ref struct, ISomeInterface? It's kind of strange to have 2 places with constraints and "expanders" for one generic argument.

I agree it's a bit strange to have both allow and where but unfortunately there need to be up and down modifiers here. That is because a generic parameter today has implicit constraints (can't be pointer, ref struct, etc ...). The syntax must provide a method to remove the implicit constraints (tallow) and add more (where).

As @CyrusNajmabadi mentioned though the syntax is the least interesting part. It is also likely the part we will end up debating the most in LDM. For now I would encourage people to focus on the behaviors and ensure they satisfy the scenarios you want to get out of this feature.

[TahirAhmadov]

The funny thing that having read the proposal and thought about it, it kind of makes sense to me so the only remaining question that I have is the syntax :) It's just not "what looks nice", but potential future enhancements. I think we should all imagine what allow can allow (pun intended) in the future, if anything; that IMO should be the main factor to decide the syntax.

[timcassell]

How will calling methods on System.Object like ToString() and GetHashCode() work? From what I understand, if a struct overrides those methods, they will not be boxed when called. But if they don't override them, they will be boxed. Will calling those methods simply be disallowed? Or that would be allowed in this case since the box is short-lived and doesn't escape?

[En3Tho]

@TahirAhmadov I thought the same. But I don't really like "allow" syntax. I'd rather think of this not as "allow" but as "restrict the lifetime of values of certain type T".

And I would like to see some work in this direction. E.g. if I didn't want some IDisposable like native resource handle to escape I would use similar mechanism.

This can lay groundwork for more general lifetime and ownership features. I think it's important to think of them when designing syntax/rules.

[IS4Code]

While the idea to have <ref T> instead of allow came from me, I think ultimately allow is the best and future-consistent approach. In the end we could have allow T : static, allow T : * (pointers), allow T : ref (ref to anything) etc. if needed, so this syntax could prepare the ground for it. Indeed <ref T> does sound like an even stronger anti-constraint (allow all byrefs and byref-likes).

[hez2010]

Will this also allow us to use allow pattern outside the ref struct scenarios?

For example,

T ParseValue<T>(string str) where T : IParseable<T> allow T : string
{
    if (typeof(T) == typeof(string))
    {
        return (T)str;
    }
    return T.Parse(str);
}

[HaloFour]

@hez2010

Will this also allow us to use allow pattern outside the ref struct scenarios?

Not without said features being explicitly designed and implemented.

[jhudsoncedaron]

So I just got news of this proposal. There's a major problem here: "adding instance default interface methods to existing interfaces becomes universally source+binary breaking". We depend on the ability to add default interface methods all over the place.

I'm reading this and going "really?" This breaks the primary motivating feature to have default interface methods.

I did come to an idea of how to fix it but this is probably bonkers hard: when you encounter a missing default interface method on a ref struct; jit the mehod and bail if the jit would emit a box operation.

[jaredpar]

This breaks the primary motivating feature to have default interface methods.

Default interface methods have always had edge cases where they do not work. For example the diamond problem where the runtime cannot pick the best DIM member results in runtime exceptions. This is just another case where DIM will have an edge case.

when you encounter a missing default interface method on a ref struct; jit the mehod and bail if the jit would emit a box operation.

I suspect that would not be very effective in practice. Most of the DIM members I've seen rely on calling other available members in the type. That implicitly uses this which forces a boxing operation on the value. This would really only work for DIM members that call into static members only and do not use this. Not sure that is a large amount.

[jhudsoncedaron]

"That implicitly uses this which forces a boxing operation on the value.": My testing is that doesn't box a struct. The jitter is smarter than that.

[jaredpar]

My testing is that doesn't box a struct. The jitter is smarter than that.

The assertion of the runtime team is that it does and that it breaks ref struct semantics.

[jhudsoncedaron]

How does that not break default implementations on normal mutable structs? (As in, the default implementation generates nonfunctional code because it mutates a copy)?

[timcassell]

Wouldn’t it be possible to do a stackalloc-kind of boxing? This would allow ref structs to be boxed

This won't solve the problem. With escape analysis, objects with limited lifetime can be moved onto stack. However, current generic parameter allows unlimited lifetime of defined variables. We need to ensure that ref struct isn't stored into any long-lived position.

It wouldn't work in general, but I think it would work for this feature to allow calling Object methods that I asked about earlier (ToString, GetHashCode).

[TahirAhmadov]

Question: would something like below become possible?

Span<char> span = ...;
string str = string.Create(10, span, (destSpan, sourceSpan)=> { ... });
// or
string str = string.Create(10, (x, y, span), (destSpan, state)=> { ... });

[jaredpar]

Assuming all of the delegates are changed to have the new anti constraint and there is no capture it should be possible.

[agocke]

In theory I'm not seeing why we couldn't update Func and Action and simplify this whole API surface.

The new API would be

string Create<TState>(int length, TState state, Func<Span<char>, TState> func);

and we would change Func to be

delegate T2 Func<T1, T2>(T1 arg1) 
  where T1 : allows ref struct
  where T2 : allows ref struct;

[timcassell]

Just curious, because I doubt anyone uses it anymore, but how would delegate BeginInvoke work with ref structs?

[jnm2]

@timcassell BeginInvoke throws PlatformNotSupportedException on newer .NET runtimes since .NET Core, and it sounds like the feature being discussed here would require runtime support and thus exclusively be for future .NET runtimes.

[timcassell]

I guess it was already possible by declaring a delegate with non-generic ref struct, so it's a non-issue for this feature.

[jaredpar]

In theory I'm not seeing why we couldn't update Func and Action and simplify this whole API surface.

My expectation is that these core delegates are all updated to have allows ref struct. The proposal actually speculated as to whether we should just automatically do this but ended up deciding against it. But expectation is that core types like Func, Action, IEnumerable would move to adopt this new anti-constraint.

[brianrourkeboll]

@jaredpar

My expectation is that these core delegates are all updated to have allows ref struct. The proposal actually speculated as to whether we should just automatically do this but ended up deciding against it. But expectation is that core types like Func, Action, IEnumerable would move to adopt this new anti-constraint.

(If this comment should be in a different place, let me know.)

I see that the proposal currently says that the allows ref struct anti-constraint is not propagated in C# and (by implication) must always be explicitly specified:

The anti-constraint is not "inherited" from a type parameter type constraint.
For example, S in the code below cannot be substituted with a ref struct:

class C<T, S>
    where T : allows ref struct
    where S : T
{}

Detailed notes:

• A where T : allows ref struct generic parameter cannot
  • Have where T : U where U is a known reference type
  • Have where T : class constraint
  • Cannot be used as a generic argument unless the corresponding parameter is also where T: allows ref struct
• The allows ref struct must be the last constraint in the where clause
• A type parameter T which has allows ref struct has all the same limitations as a ref struct type.

That doesn't present a huge problem in C#, since existing generic constraints already aren't propagated automatically.

Would F# need to suppress its default automatic generalization and generic constraint propagation for this specific (anti-)constraint? That would be a pretty major departure for F#.

For example, take these two existing F# functions:

let f (x : 'T when 'T : struct) = ignore x
let g x = f x

The generic constraint from f is automatically propagated to g, so that their type signatures look like:

val f : x:'T -> unit when 'T : struct
val g : x:'T -> unit when 'T : struct

Would F# need to suppress the propagation of this anti-constraint in particular? I.e., for some function g that calls a function f that has this anti-constraint, would we expect

val f : x:'T -> unit when 'T : allows ref struct
val g : x:'T -> unit // No constraint?

?

What if there were also another, regular constraint on 'T? Would the regular constraint be propagated but the allows ref struct anti-constraint be suppressed? That could get pretty confusing.

val f : x:'T -> unit when 'T :> ISomeInterface and 'T : allows ref struct
val g : x:'T -> unit when 'T :> ISomeInterface // This would be strange.

While those may be partly F#-specific design decisions, I think it is worth noting that the design of this feature in the runtime and in C# may have additional implications for F#, especially if fundamental BCL types like Func and IEnumerable are updated to use it.

[jaredpar]

Would F# need to suppress its default automatic generalization and generic constraint propagation for this specific (anti-)constraint? That would be a pretty major departure for F#.

Think that is a question for F# language designers. C# behavior here is essentially following how constraints are modeled in IL. There is nothing stopping F# from providing a different presentation here.

@vzarytovskii

[brianrourkeboll]

There is nothing stopping F# from providing a different presentation here.

Yes, but my comment was in part about how the choices that the current design makes available to F# here don't seem ideal:

• Treat this constraint differently from the way F# treats all others.
• Deal with the implications of this constraint being propagated through type signatures everywhere.
• A secret, third thing?

But it's entirely possible that I'm missing something or haven't thought things through enough, and that consuming this from F# will be simpler than I'm making it out to be.

[HaloFour]

Treat this constraint differently from the way F# treats all others.

Given anti-constraints are different from other constraints, it would feel very appropriate if F# would treat them differently than it treats constraints today.

[jaredpar]

Treat this constraint differently from the way F# treats all others.

This is how C# treats constraints today though: they do not propagate by default. This decision is following our existing patterns.

[agocke]

Not really. The anti-constraint is about cancelling out a constraint that already exists. The way it currently works is that .NET has an implicit heap constraint that's available to all unconstrainted type parameters. That constraint allows you to do things like box variables of that type, and store variables of that type as fields in classes and structs.

The "allows ref struct" anti-constraint cancels out the implicit heap constraint and removes it from the constraint list. It's not about adding constraints to an implicit union, it's about removing them.

[brianrourkeboll]

The "allows ref struct" anti-constraint cancels out the implicit heap constraint and removes it from the constraint list. It's not about adding constraints to an implicit union, it's about removing them.

It is indeed an intersection from that perspective, just like adding class? → mightbenull, I suppose, except mightnotbeboxable.

While it is removing a constraint from the consumer's perspective, it is also adding a constraint by which the declaring construct must abide...

[brianrourkeboll]

...Upon further thought, though, I think I'm back to where I was before. (Sorry for messing up the threading; I was on my phone.)

I think we were using the same metaphor to refer to different things, or rather referring to the same thing from different perspectives (API consumer versus API implementer).

Given my previous post:

In a way, adding an "anti-constraint" is really just adding a case to the implicit default constraint union T ≈ where T : default ≈ where T : (class | struct):

If —

void M<T>(T t) { … }

≈

void M<T>(T t) where T : default { … }

≈

void M<T>(T t) where T : (class | struct) { … }

— then

void M<T>(T t) where T : allows ref struct { … }

≈

void M<T>(T t) where T : (class | struct | ref struct) { … }

≈

void M<T>(T t) where T : (default | ref struct) { … }

And your response:

The "allows ref struct" anti-constraint cancels out the implicit heap constraint and removes it from the constraint list. It's not about adding constraints to an implicit union, it's about removing them.

This emerges from the fact that the set of operations that can be applied to a union is the intersection of the operations that can be applied to every case in the union.

From my other post (again, sorry for messing up the thread):

Yeah, that's what my imaginary "constraint union" syntax was supposed to mean — class constraint (and all that implies, including ability to be boxed on the heap) or struct constraint (and all that implies, including ability to be boxed on the heap) or ref struct constraint (and all that implies, including inability to be boxed on the heap). Any concrete type parameter can and must satisfy exactly one of those at a time, i.e., it is either a class, a regular struct, or a ref struct.

But by definition in a generic construct the type parameter is not yet concrete, so, since you can't box a ref struct, you can't box a T where T : (class | struct | ref struct), since T might end up being instantiated to a ref struct.

Seen that way, it actually makes sense that F# might not auto-propagate this constraint unless explicitly specified, since, unlike all existing constraints utterable in C# or F#, it is not being intersected with the implicit (class | struct) union of constraints.

Given two functions like this (imaginary syntax; F# does happen to already use and to represent constraint intersection, so it would probably make sense to use or to represent constraint union):

let f (x : 'T when 'T : struct or 'T : not struct or 'T : byref<struct>) = ignore x
let g x = f x

It would make sense that g would have the more general signature:

val f : x:'T -> unit

i.e., with the default, implicit constraint union

val f : x:'T -> unit when 'T : struct or 'T : not struct

since 'T : struct or 'T : not struct¹ is of course a subset of 'T : struct or 'T : not struct or 'T : byref<struct>.

Just because f has placed the restriction upon itself that T might be something unboxable doesn't mean that g must also have that restriction.

Footnotes

1. not struct is F#'s equivalent of the C# class constraint. ↩

[MitchRazga]

ref record struct was previously not possible since ref struct couldn't implement IEquatable<S> #5431
Would just the implementation of this proposal allow the compiler to synthesize the record members or would that be a separate proposal? @AlekseyTs

[jaredpar]

Would just the implementation of this proposal allow the compiler to synthesize the record members or would that be a separate proposal?

That would need to be a separate proposal. The inability to implement IEquatable<T> is just one of the issues that would need to be addressed before we could do ref record struct. Guessing when we sit down and look at all of the behaviors around record we'd find a few more we'd need to think about.

[timcassell]

One of the marquee original intentions for ref structs implementing interfaces was for Span<T> and ReadOnlySpan<T> to implement IEnumerable<T>. This would help
solve a number of betterness issues with adding new APIs, but because IEnumerable<T>.GetEnumerator() returns an IEnumerator<T>, we can't implement it in an allocation-free
manner. That would cause any IEnumerable API to become a performance trap for Span<T>, which is extremely undesirable.

Just add the IEnumerator<T> interface on the Span<T>.Enumerator type. I don't need the IEnumerable<T> interface on the span itself for now. Anyway, that could be saved for a future improved IEnumerable<T, TEnumerator> interface after C# improves generic inference for those.

My use case that I was hoping to start using this for:

public static Promise All(params ReadOnlySpan<Promise> promises)
    => All(promises.GetEnumerator());

public static Promise All<TEnumerator>(TEnumerator promises) where TEnumerator : IEnumerator<Promise>, allows ref struct
{
    using (promises)
    {
        ...
        while (promises.MoveNext())
        {
            var p = promises.Current;
            ...
        }
        return ...;
    }
}

Rules for ref structs in generics are approved. Rules for ref structs implementing interfaces generally look good, but need validation against real world scenarios before we
allow this part of the feature to ship in anything more than preview.

Allowing ref struct as generics but disallowing them to implement interfaces kind of neuters the feature.

[ds5678]

Allowing ref struct as generics but disallowing them to implement interfaces kind of neuters the feature.

While I agree that having this held back behind the preview language feature is undesirable, it's still available to the people who really want to use it. For example, I rewrote my HTML generation library to make use of the features.

https://github.com/AssetRipper/AssetRipper.Text.Html