Exploration: Roles, extension interfaces and static interface members

[MadsTorgersen]

Roles, extension interfaces and static interface members

This is an attempt to address the scenarios targeted by my previous "shapes" investigation (which was in turn inspired by the "Concept C#" work by Claudio Russo and Matt Windsor), but in a way that leverages interfaces rather than a new abstraction mechanism. It is not necessary to read the previous proposals in order to understand this one.

This proposal assumes some version of extension everything, and consists of a trio of features:

1. Roles: Lightweight "transparent wrapper types" that can be applied to individual objects of a given type, can add additional members to them, and allow them to implement given interfaces.
2. Extensions: A generalization of "extension everything" that can extend a given type not just with new members but also to implement additional interfaces. The relationship is in force within a certain static scope (just like extension methods today).
3. Static interface members: Allow static members to be specified in interfaces. A class or struct implementing the interface must implement a corresponding static member.

At the end there is a comparison to previous proposals, if you're eager to start there.

We'll look at roles first, and then at extension interfaces, which build on them. Later we'll get to the synergy that the independent feature of static interface members would have with those. Together they get close to the expressiveness that type classes have in Haskell.

Interfaces

Interfaces in C# and .NET are often talked about as "contracts". They abstract capabilities of objects (or values) as ultimately implemented by classes (or structs), but they aren't necessarily very tightly coupled to those objects and classes. The members of an interface are not inherited by an implementing class, and may be implemented "explicitly" so that they don't even show up in the surface area of the class. While a type can only have one base class, it can (and frequently will) implement several interfaces.

Interfaces can be used as types of objects (describing required properties of those individual objects), and as (part of) constraints on type parameters (describing required properties of the individual type arguments). Some interfaces, such as IEnumerable<T>, are frequently used as types, e.g. for parameters (as in LINQ), whereas others, such as IComparable<T>, are primarily used as constraints, and aren't very useful as types. We'll use both in examples further below.

Even though interfaces are somewhat loosely coupled to implementing classes, their implementation must currently be stated by the declaration of the implementing class itself, and they cannot apply e.g. to only some of a generic class's instantiations, or at all to certain kinds of type declarations such as delegates and enums. The purpose of roles and extension interfaces is to allow interfaces to be applied to any type after the fact, separate from that type's declaration. They can truly be a "perspective" on classes or objects, one that maybe only applies in a certain context that the original class declaration didn't know or care about.

Roles allow interfaces to be implemented on specific values of a given type. Extensions allow interfaces to be implemented on all values of a given type, within a specific region of code.

Roles

A role is somewhere in between a derived type and a wrapper type. It is a new kind of type that provides a specialized view or perspective on specific instances of another type, bestowing these instances with extra members and capabilities, while still providing "see-through" access to their inherent properties.

One of the capabilities a role could bestow on a type is making it implement a given interface.

An example: lightweight typing of dictionaries

A lot of data enters and leaves a running program through weakly typed representations such as JSON or XML, which are most faithfully represented as some form of dictionary objects. E.g. let's say we have a dictionary type DataObject with an indexer that takes a string and returns a DataObject.

public class DataObject
{
    public DataObject this[string index] { get; } // throw if not found
    public int ID { get; }
    public void Reload();
    public string AsString(); // throw if the DataObject does not represent a string
    public IEnumerable<DataObject> AsEnumerable(); // throw if not...
    ...
}

Now if we know or expect that the data comes in given shapes, we can create lightweight types for those in the form of roles:

public role Order of DataObject
{
    public Customer Customer => this["Customer"];
    public string Description => this["Description"].AsString();
    ...
}
public role Customer of DataObject
{
    public string Name => this["Name"].AsString();
    public string Address => this["Address"].AsString();
    public IEnumerable<Order> Orders => this["Orders"].AsEnumerable();
    ...
}
public static class CommerceFramework
{
    public IEnumerable<Customer> LoadCustomers();
    ...
}

Roles wouldn't be able to declare additional state on their own, so they can pop in and out of existence without much ado. On the other hand, they get to have implicit (identity) conversions to and from the type they extend; conversions which (normally - there are caveats) extend even to types constructed from them. You can see those conversions in the implementations of the properties above. For instance, even though this["Customer"] is of type DataObject, it can be converted to the role Customer when returned from the property Customer. And even though this["Orders"].AsEnumerable() returns an IEnumerable<DataObject>, the Orders property can return it as an IEnumerable<Order>. While there'd be implicit conversions up and down, though, they probably shouldn't exist sideways: a Customer should not implicitly convert to an Order.

The purpose is for program logic to bestow an additional lightweight static type layer upon given objects:

IEnumerable<Customer> customers = CommerceFramework.LoadCustomers();
foreach (Customer customer in customers)
{
    WriteLine($"{customer.Name}:");
    foreach (Order order in customer.Orders)
    {
        WriteLine($"    {order.Description}");
    }
}

This is a completely statically typed program, even though all the objects are actually only instances of DataObject at runtime. You get auto-completion, type checking, navigation, refactoring etc. according to the Order and Customer role types. Of course, the example assumes that the CommerceFramework "knows what it's doing"; i.e. that the DataObjects coming back from LoadCustomers do indeed have the right "shape" to behave like customers, which in turn means that they have "Name" and "Address" and "Orders" entries that also have the expected shapes, and so on. In other words, the roles let you statically express the types that you expect to be dynamically adhered to by the data.

This is not unlike the way TypeScript imposes static types on objects which at runtime are just dictionaries and may in principle not adhere to them at all: in practice, good programming practices make the type really useful, and rarely wrong.

Things get more interesting if we also allow roles to implement interfaces on behalf of the types they augment:

public interface IPerson
{
   public int ID { get; }
   public string Name { get; }
}

public role Customer of DataObject : IPerson
{
    public string Name => this["Name"].AsString();
    public string Address => this["Address"].AsString();
    public IEnumerable<Order> Orders => this["Orders"];
}

Now Customer implements IPerson, so a DataObject viewed as a Customer can be treated as an IPerson through conversion, and Customer can be passed as a type argument where the constraint is IPerson. Note that the IPerson members are implemented partly by the Customer role itself (the Name property), partly by the underlying DataObject type (the ID property).

What this means is that you can use roles to adapt existing objects to a contract that's expressed through an interface.

How does it work?

Most of how roles would work is pure type erasure: The compiler continues to use the underlying type, except where role-added members are accessed, in which case the compiler can rewire to those as e.g. static members or members of a wrapper struct. Lookup rules would treat the role much as a derived type, so that it can shadow any members from its "augmented" type.

The interface implementation, though, would need runtime help. We could almost get away with generating a wrapper struct, which could be a) "boxed" to the interface for conversion, and b) passed as a type argument in lieu of the wrapped type to satisfy the constraint.

For conversion to the interface, such wrapper struct boxing would actually sort of work. The boxed struct would not have the same object identity as the wrapped object, but maybe that's ok.

When it comes to roles being passed as a type argument, though, the wrapper struct implementation strategy has severe limitations. In the context of the example above, imagine the following (somewhat contrived) method:

public string[] ReloadPeople<T>(IEnumerable<T> people) where T : DataObject, IPerson
{
    var names = new List<string>();
    foreach (Person person in people)
    {
        person.Reload();        // method from DataObject;
        names.Add(person.Name); // property from IPerson;
    }
    return names.ToArray();
}

Customer[] myCustomers;
string[] names = ReloadPeople(myCustomers);
foreach (var name in names) WriteLine(name);

A wrapper struct wouldn't work here, because the type argument (which in the example is inferred to be Customer) needs to satisfy both the IPerson interface constraint and the DataObject class constraint, and a wrapper struct that implements the interface would only satisfy the former.

Also, without runtime participation, the method would expect an IEnumerable<Customer>, not an IEnumerable<DataObject>. But the caller, using type erasure, would at runtime actually have a DataObject[], so there'd be an argument type mismatch.

How can we make the runtime participate and make it work? Let's say that roles are actually represented in the runtime as a new kind of type, next to structs, classes, interfaces etc., rather than being compiled away "into something else". The runtime has roles!

Now when a role is passed as a type argument, the runtime can see what is happening, and "do the right thing". Specifically here, it can see that yes, the IPerson constraint is satisfied by the Customer role, but also it can "see through" the role to the type it's augmenting, and see that the underlying DataObject type satisfies the DataObject constraint. So far so good.

As for the IEnumerable<T> argument type, we need the runtime to understand that an IEnumerable<Customer> is really the same as a IEnumerable<DataObject> at runtime. That's only true because IEnumerable<T> doesn't rely on Customer to satisfy any of its constraints on T. So the runtime needs to be smart enough to distinguish generic instantiations where the role is integral from ones where it is irrelevant to the validity (and meaning) of the instantiation.

This is further explored below. The main point here is that there is enough information available that the runtime can do it.

Extension Interfaces

The extension everything proposal generalized the current extension methods to apply to a wide range of member kinds, including static ones. It fundamentally changes the extension declaration syntax to look more like a type declaration, containing additional members that are expressed as if they were in the body of the extended type itself.

Extension interfaces add to that the ability for the extension to implement interfaces on behalf of the extended type.

An example: implementing IEnumerable<T>

Even today it is easy to add a GetEnumerator method to any type, as an extension method:

public static class Extensions
{
    public static IEnumerator<byte> GetEnumerator(this ulong bytes)
    {
        for (int i = sizeof(ulong); i > 0; i--)
        {
            yield return unchecked((byte)(bytes >> (i-1)*8));
        }
    }
}

With the extension everything proposal we would have a different syntax for declaring extension methods, something along the lines of:

public extension ULongEnumerable of ulong
{
    public IEnumerator<byte> GetEnumerator()
    {
        for (int i = sizeof(ulong); i > 0; i--)
        {
            yield return unchecked((byte)(this >> (i-1)*8));
        }
    }
}

Note the use of this to represent the receiver in the method body. The new extension syntax makes it look much more like the members are actually declared inside of the type declaration of the extended type itself. Requiring the extension itself to have a name (ULongEnumerable here) may seem a little excessive, and whether that's required is certainly up for debate. It will however come in handy for disambiguation (similar to the name of the static class that holds extension methods today), and I also use it later in my implementation scheme. Also it makes for a good place to declare any type parameters, as we are about to do in the IComparable<T> example below.

Either way, though, this won't get you far: foreach is one of the few C# language features that expands to use instance methods but not extension methods, so the GetEnumerator extensioln method won't get picked up. We could (and should) fix that in the language, in which case you can write:

foreach (byte b in 0x_3A_9E_F1_C5_DA_F7_30_16ul)
{
    WriteLine($"{e.Current:X}");
}

However, while ulong would thus satisfy the enumerable pattern from a language/compiler perspective, it still wouldn't make it an IEnumerable<byte> in a type sense, so you couldn't for instance use it in a LINQ query.

Extension interfaces are here to fix that! If we let extension declarations implement an interface, then we are in business:

public extension ULongEnumerable of ulong : IEnumerable<byte>
{
    public IEnumerator<byte> GetEnumerator()
    {
        ...
    }
}

Declaring that an extension for a type "implements" an interface means that the type's members - including its available extension members - can be used to implement the interface. In this case, the extension GetEnumerator method is used to satisfy the interface.

The declaration means that wherever the extension is in force (probably via a using directive, as today), ulong is considered to not only have a GetEnumerator method, but also to in some sense implement IEnumerable<byte>. We should ponder what that means exactly, but it should at least allow for a given ulong to be passed as (i.e. converted to) an IEnumerable<byte>, e.g. in a method call:

const ulong ul = 0x_3A_9E_F1_C5_DA_F7_30_16ul;
var q1 = Enumerable.Where(ul, b => b >= 128);  // method argument
var q2 = ul.Where(b => b >= 128);              // extension method receiver
var q3 = from b in ul where b >= 128 select b; // query expression
// etc.

This should seem familiar to how roles allowed individual objects to convert to interfaces above. Indeed, below I'll propose implementing extension interfaces with the help of roles.

An example: implementing IComparable<T>

IComparable<T> is most commonly used as a constraint on generic types or methods that need to compare several values of the same type.

With extension interfaces we can make types IComparable<T> that wouldn't normally be. For instance, if we have an enum

public enum Level { Low, Middle, High }

We can make it comparable with the extension declaration:

public extension LevelCompare of Level : IComparable<Level>
{
    public int CompareTo(Level other) => (int)this - (int)other;
}

We can also extend only certain instantiations of generic types with an interface implementation. For instance, let's make all IEnumerable<T>s comparable with lexical ordering, as long as their elements are comparable. (This implementation also uses expected new C# features switch expressions and tuple patterns but feel free to ignore that, once you're done enjoying the clarity it allows in the logic 😉):

public extension EnumerableCompare<T> of IEnumerable<T> : IComparable<IEnumerable<T>> where T : IComparable<T>
{
    public CompareTo(IEnumerable<T> other)
    {
        using (var le = this.GetEnumerator())
        using (var re = other.GetEnumerator())
        {
            while (true)
            {
                switch (le.MoveNext(), re.MoveNext())
                {
                    case (false, false): return 0;
                    case (false, true): return -1;
                    case (true, false): return 1;
                }
                var c = (le.Current, re.Current) switch
                {
                    (null, null) => 0,
                    (null, _) => -1,
                    (_, null) => 1,
                    var (l, r) => l.CompareTo(r)
                };
                if (c != 0) return c;
            }
        }
    }

Now we can use an IEnumerable<T> as an IComparable<IEnumerable<T>> wherever this extension is in force, but only as long as the given T is an IComparable<T> itself. For instance, an IEnumerable<string> would be comparable to other IEnumerable<string>s because string is comparable, whereas an IEnumerable<object> would not, because object is not.

So with these two extensions in force, we can now compare two arrays of Levels:

using static Level;
Level[] a1 = { High, Low, High };
Level[] a2 = { High, Medium };
WriteLine(a1.CompareTo(a2));

Because of the LevelCompare extension on Level it satisfies the constraint on the EnumerableCompare extension on IEnumerable<Level>, which therefore in turns makes the Level[]s comparable!

Static interface members

Interfaces today can only require instance members in their implementing classes and structs. When interfaces are used as types, that is the only thing that makes sense, since we are talking about the capabilities of the individual objects of that type.

However, when interfaces are used as constraints, it makes sense for them to be able to specify other aspects of a given type argument; notably any static members it may have. This is so that those static members can be accessed directly on the type argument in generic code.

There's a question about which kinds of static members would make sense, but methods, properties, indexers and unary and binary operators should definitely be included.

An example: Numeric abstraction

Today C# does not offer a means for numeric abstraction, and cannot elegantly express generic numeric algorithms. This is quite a severe limitation for many computational workloads, such as for instance machine learning.

Here is a simple numeric abstraction, based on the mathematical notion of monoids.

public interface IMonoid<T> where T : IMonoid<T>
{
    static T operator +(T t1, T t2);
    static T Zero { get; }
}

This represents that a monoid over a given type T must provide a binary + operator as well as a static Zero property yielding a neutral element. The constraint where T : IMonoid<T> is there to morally satisfy the rule that an operator can only be implemented inside one of its operand types.

Given the abstraction, we can now write a simple generic numeric algorithm:

public T AddAll<T>(T[] values) where T : IMonoid<T>
{
    T result = T.Zero;
    foreach (T value in values) { result += value; }
    return result;
}

This generic method works over every monoid, yet is able to make use of operators (+) and static members (Zero) directly in code as if working on a concrete type for which these were defined. We have numeric abstraction!

Note that the constraint is what makes it possible for the compiler to search for the operator definition by looking in the operand type, just as how we do today for concrete operators.

Now let's combine this with extension interfaces:

public extension IntMonoid of int : IMonoid<int>
{
    public static int Zero => 0;
}

The declaration extends int with a static Zero property, but also makes it implement the IMonoid<int> interface. The interface is fulfilled jointly by the Zero property of the extension, and the + operator inherent to the underlying int type itself.

Bringing it all together, we can now apply our generic numeric algorithm to an array of ints:

int result = AddAll(new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 });

This infers int as the type argument to the generic method AddAll using normal C# type inference, and deems it to satisfy the constraint of IMonoid<int>, because the extension causes int to implement that interface. This only works if the extension is in scope! Elsewhere int and IMonoid<int> have nothing to do with each other.

To work properly, static interface members would have to be implemented in the runtime itself. This has previously been prototyped internally at Microsoft, so we know it can be done.

Extensions through roles

We can think about extensions as "extension roles". An extension declaration really declares a role for the extended type, and then applies that role to all occurences of the underlying type throughout the scope where the extension is in force.

An extension is a role that all instances of the extended type play within a given static scope!

For extension interfaces, specifically, this means that interface-implementing roles become the mechanism whereby the interface gets applied, when converting to the interface as well as when satisfying constraints.

For instance, the extension declaration from above:

public extension IntMonoid of int : IMonoid<int>
{
    public static int Zero => 0;
}

is really implemented as a role declaration:

public role IntMonoid of int : IMonoid<int>
{
    public static int Zero => 0;
}

And wherever the "monoidness" of the int is required, the role is used to achieve it. For instance, in the call

int result = AddAll(new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 });

The role IntMonoid is passed as the type argument to AddAll<IntMonoid>(...), so that the constraint is satisfied, and the runtime knows how to do IMonoid things with the incoming ints.

Disambiguation

The mapping of extensions to roles opens up an approach to disambiguation of extension members when more than one candidate is in force.

Today, extension methods are disambiguated by falling back to their underlying nature as static methods, and relying on the name of their enclosing static class.

With "extension everything" there is no longer a manifest "second nature" that extension members can fall back to. But the underlying roles could play that, hm, role. They would be allowed to be used directly as a role in the source code. Specifically, you could implicitly convert a given int to IntMonoid, and the IntMonoid members would then take precedence over those of both int and other extensions of int.

Additional considerations

Having a notion of how this feature set works, let's go deeper into some specific questions that are likely to come up quickly.

Type tests

If an extension is in scope, should it influence type tests? I.e. given the earlier extension of ulong to IEnumerable<byte>, if we write:

if (o is IEnumerable<byte> e) WriteLine("Yes!");

and at runtime o is a boxed ulong, should the WriteLine occur? Intuitively it could go either way. We could argue that in the static scope we should try our best to make it look "as if" ulong inherently implements IEnumerable<byte>. Or one could say that type tests are specifically for checking inherent type relationships, and extension interfaces are much more like user defined conversions, which already don't count in type tests today.

We probably can implement it so that the type tests work in a given scope. But it won't be cheap. The compiler would have to look at all extensions in scope "from the other side", noting which ones could cause IEnumerable<byte> to be implemented, then checking for all the types that are extended by those extensions. In other words, the above test would be expanded by the compiler to something like:

if (o is IEnumerable<byte> e || o is ulong u && DummyTest(e = u))...

So it isn't pay for play: an extension wouldn't just impose cost when it is being actively applied, but also on all tests against all target interfaces of all extensions that are in scope!

For this practical reason, I'd propose not to make the extensions apply in type tests. This would be one of the ways in which extension implementation isn't as full-featured as inherent implementation, and the "seams would show".

Explicit implementation

If extensions and roles can implement interfaces, then it would make sense to allow them to implement interface members explicitly, so that the members don't show up on the extended type or role itself, but only when they appear "as" the interface through boxing or generic constraint.

For instance, using standard explicit implementation syntax for the Zero property, the extension of int to IMonoid could be written as:

public extension IntMonoid of int : IMonoid<int>
{
    static int IMonoid<int>.Zero => 0;
}

You would then not be able to say int.Zero, but the member would be there on e.g. the type parameter in the body of the generic AddAll method, since it is constrained by IMonoid<T>.

This is exactly how explicit implementation works today. There doesn't seem to be any additional semantic or implementation challenges with allowing explicit implementation in extensions and roles, and it seems useful and tidy to be able to implement extra interfaces without polluting the type itself with extra members.

Eagerness of "Witnessing"

When a value of an extended type (or a value playing a given role) is converted to an interface that it doesn't inherently implement, there needs to be some sort of "boxing" to an object that implements the interface to "witness" how it implements the interface. There are a couple of questions you can ask about that.

Should we eagerly "witness" on suspicion, even when "boxed" to a type (such as object) that doesn't necessarily require it? It would certainly help with implementing type rediscovery later, should we choose to want to do that. In a sense it seems a bit odd if these two operations:

IFoo f1 = myBar;
IFoo f2 = (IFoo)(object)myBar;

result in different outcomes. (The latter would fail at runtime.)

I think eager boxing is unrealistically expensive, and moreover I would fear that it would lead to "pollution" of objects with chains of "witnesses" wrapped all around them. Chaining is already an issue we have to discuss, but hopefully, "witnessing" only when directly statically necessary will limit this to a manageable level.

Object identity

A follow-up question to consider is object identity. Should a "witnessed" reference type be able to compare reference equal to an "unwitnessed" cousin?

IFoo f = myBar; // witnessed through extension
object o = myBar; // directly assigned
if (f == o) WriteLine("Equal!");

For f and o to compare reference equal above, the runtime would need to be in on it, treating "witnesses" specially by "seeing through" them to the core identity beneath. This is certainly doable, but costly, probably adding the cost of an additional check to all or most reference comparisons in a program!

I believe that it is probably fine to not try to retain object identity of "witnessed" objects. We should think of a "witnessing" conversion as a representation-changing one, just like boxing of value types is today.

Identity conversions

I mentioned earlier that there would be identity conversions both ways between a role and its underlying type. This means that from the type system's point of view they are the "same" type in most respects. For instance, you cannot overload a method on one type versus the other.

There are currently three cases in the language where there are identity conversion between types that are a little bit different, but share a runtime representation. One is between dynamic and object, the other is between tuple types with identical types in identical positions, but with different tuple element names. A third one is between constructed types which differ only by type arguments which recursively are themselves different but identity convertible. In all of these cases the identity conversion is transitive: if there is an identity conversion from A to B and from B to C, then there is also one from A to C.

With roles it would be nice to have identity conversions between them and their underlying types. After all both directions are representation preserving and will work without exception. However, it seems desirable to avoid identity conversions between different roles of the same underlying type. This means that the identity conversions would not be transitive: for two roles R1 and R2 of a type T, there would be identity conversions from R1 to T and from T to R2, but not directly from R1 to R2.

I cannot think of any aspect of the language that relies on transitivity of identity conversions, but there may be some. This is certainly something to look into.

Roles as type arguments

On a related note we have to think about conversions of constructed types where the type arguments are roles. There is something different going on, depending on whether the constructed type "relies on" the role implementing a required interface. Consider these declarations:

/* Assembly 1 */
public interface IRecord
{
    public int ID { get; }
}
public class Register<T> where T : IRecord
{
    Dictionary<int, T> records = new Dictionary<int, T>();
    public void Add(T record) => records.Add(record.ID, record);
    public bool Contains(IRecord record) => records.ContainsKey(record.ID);
    ...
}

/* Assembly 2 */
public class Person
{
    public string Name { get; }
}

/* Assembly 3 */
public role Employee of Person : IRecord
{
    int IRecord.ID => Name.GetHashCode;
}

We have a registration "framework", and an independently declared class Person that gets adapted by a third party to the framework's currency type IRecord with the help of a role Employee.

On the one hand, for a stock collection type, say List<T>, we want List<Person> and List<Employee> to be identity convertible. In a sense, List<Employee> is just a "view" on a List<Person> and we want to freely convert between them. I make use of this kind of conversion for instance in this member declaration from the first role example above:

    public IEnumerable<Order> Orders => this["Orders"].AsEnumerable();

where an IEnumerable<DataObject> is implicitly converted to an IEnumerable<Order>.

On the other hand, a Register<Employee> is clearly not the same as a Register<Person>. In fact the latter does not even exist, since Person - unlike Employee - does not satisfy the IRecord constraint on T in IRegister<T>.

Clearly the difference between List<Employee> and Register<Employee> is due to the fact that the role is integral to satisfying the constraint in the latter. Somehow we need to make the runtime aware of this difference!

There are a couple of ways you can imagine this. One way is that whenever the runtime sees a role as a type argument it will agressively erase it to the underlying type, unless it is necessary for the constraints. Another way is that the runtime understands when the conversions are there (as with List<Employee>) and when they aren't (as with Register<Employee>).

The whole thing gets even a bit more complicated if we allow roles to reimplement an existing interface that the underlying type also implements. Say that Person itself also implemented IRecord, but differently from the Employee role. Now both Register<Person> and Register<Employee> are legal constructed types, but different! One makes use of Person's implementation of the ID property, the other of Employee's implementation. So even when both instantiations are legal, they may or may not be identity convertible to each other. We may be able to avoid this by forbidding reimplementation by roles, but I suspect that is hard: once you get generic enough you may not know that you are reimplementing an existing interface.

So one way or another, the runtime has to understand when a role as a type argument makes the constructed type different from using the underlying type, and when it doesn't.

Comparison to previous proposals

"Concept C#" tries to add Haskell-style type classes to C#.
Haskell, being a functional language, is all about top-level functions being applied to values. Type classes allow you to abstract over the set of functions that apply to a given type of values. Thus, when you write a generic function and constrain the type parameters with type classes, you know which functions are available for the type parameters in the body of the method, even if you don't know which implementation of those functions to use (those are supplied when the generic function is instantiated).

Concept C# tries to apply a similar mechanism to C#, by allowing a) the expression of such abstract "groups of functions" into what is called "concepts", as well as a means of declaring the function implementations for a given type ("instances"). Generic functions using concepts explicitly take an extra type parameter for communicating the choice of function implementations to the generic method. The caller rarely has to supply those extra type arguments explicitly, though, as type inference will usually figure them out from context - the "static scope" within which the concept is in force.

There is some quite impressive type inference and some neat implementation tricks going on to make this work. However, it arguably doesn't feel very C# like, based as it is on "outside functions" rather than the more object-oriented "inside methods".

"Shapes" try to make the whole thing a little more object-oriented, and fit closer with the existing mechanisms of C#. It still has a separate new abstraction mechanism called "shapes", but instead of abstracting over groups of "outside functions" it specified groups of "inside members" - static as well as instance - that are required for a type to implement the shape.

Shapes still aren't types. They are still a form of "named constraints", that can only be used for generic abstraction, not subtype polymorphism.

The shapes proposal unified the post-hoc application of shapes with "extension everything". It still needs an extra type parameter on generic methods that use the shapes as constraints, but the compiler generates that extra type parameter and it remains hidden at the language level.

The two major remaining downsides of the shapes proposal are: a) it still introduces a whole new abstraction mechanism, that in many ways competes with interfaces for expressing contracts. And b) for this reason, there is no way of adapting types to existing generic code that uses interfaces, not shapes, as constraints.

The limitation of the shapes proposal to work only for constraints, not conversions, may be seen as an upside or a downside depending on how you look at it. But it certainly limits the scope of the feature, and may put pressure on the style of libraries in the future to rely more on generics and less on subtyping, arguably leading to more complex signatures and a greater reliance on type inference to keep consuming code readable.

The purpose of the current proposal is to try to address those remaining downsides and suggest a feature that leverages the current interface abstraction of the language, while fully integrating with how existing generics work. The trade-off is that: a) it needs to own up to interfaces being types, and facilitate the use of extensions for conversions to such interfaces. b) it needs to be able to pack the information for which previous proposals used two type arguments (one for the type itself, one for its implementation of the concept/shape) into one type argument (the role), that somehow "just works" even with preexisting generic methods and types that use interface constraints. Thus, the runtime needs to be "in on it", where the other proposals could be "compiled away" to the existing runtime.

The previous proposals don't really have a concept of "roles". They only allow the extension of all values of a type with extra "constraint satisfaction power", not selected ones, corresponding to the "extension interfaces" of the current proposal. You can certainly imagine cutting roles as a language-level construct from this proposal, and only doing extension interfaces. However, the underlying mechanisms to implement it, including in the runtime, would be very similar to roles. This is because extension interfaces rely on a notion of static scopes that make no sense to the runtime. So they need to be transformed from a mechanism that is in force due to code location, into one that is specifically applied when types and values are passed at boundaries. That is exactly what roles do.

I personally think that the role feature has independent value, and also provides good, consistent and natural answers to many design questions that otherwise arise with extension interfaces.

[dfkeenan]

Lot's of interesting stuff there. 😄

I was curious if a part of "static interface members" you considered including something like"constructor interface members"? Allowing generic methods to have more options than new().

[MadsTorgersen]

@dfkeenan constructors are interestingly different. A base class can have a constructor without a derived class having it. So if an interface could specify a constructor, then a base class could satisfy it as a constraint while a derived class couldn't. Not saying that's bad, just - new (no pun intended! 😉)

We should look at it for sure, but I didn't want to get into those complications here.

[Joe4evr]

3. Static interface members: Allow static members to be specified in interfaces. A class or struct implementing the interface must implement a corresponding static member.

I don't know how much I like this. I was getting pumped for using a static member in an interface to put a fallback implementation in once #52 is available:

public interface IFooConfig
{
    public static IFooConfig Default { get; } = new DefaultFooConfig();

    bool CanDoBar { get; }

    private sealed class DefaultFooConfig : IFooConfig
    {
        public bool CanDoBar => true;
    }
}
//....
public FooService(IFooConfig? config = null)
    => _config = config ?? IFooConfig.Default;

This already compiles fine on the feature branch.
I don't like how the static member would suddenly be part of the contract with the implementer. I understand that it could solve some problems, but then we will also need something to indicate that some static members are not part of that contract.

[TheGrandUser]

For type testing to roles/extension interfaces, why not something like
if (o is also IEnumerable<byte> e) WriteLine("Yes!");
so you can opt into the higher costing check if you really need to.

[ghord]

One question about the use case of arithmetic abstractions: Are static extension methods going to be as performant as unabstracted code? Right now interface calls are slower than normals calls, and it would be shame if the feature was not a good fit for high performance code.

[xoofx]

I like the concept a lot, a role allowing a kind of super constrained typedef.

Though, maybe not sure about the name "role"... Maybe view as used in the description of a role would be an easier name to associate with the underlying concept...

So If I understand correctly, a role would require the runtime to treat it its generic instantiation as we already do for structs right? So even if the role is based on a class type, it would require to instantiate it (unlike generic instance for classes that are shared)

Also just to make sure, we can have role of roles right?

[iam3yal]

@xoofx

Also just to make sure, we can have role of roles right?

Just to give a scenario, someone might want to have a role of say Time and then use this for units of time such as Hour, Min and Sec so can we say role Time of int and then do role Hour of Time? guessing here but if roles are actual types I see no reason why it wouldn't be possible.

[amis92]

All the emoticons/reactions cannot express the joy of considering CLR update for all that goodness. It can and will take time, but I'm really excited about even the concept 😎 of CLR 5.0

This type system extension along with data-type handling and generation proposed in #1673 and #1667 (and maybe source generators - one can dream!) would, in my opinion, make a perfectly good excuse to follow Windows release naming (after C#8 skip 9 and go straight to C#10). 😇

[orthoxerox]

I don't know if making extensions types, not just type constraints, is a good idea. It opens a huge can of worms with equality, type testing, etc. That's why I like shapes more than this proposal.

The starting example of wrapping dynamic objects in a fake static shell is really compelling, though. I must think more about it.

[xoofx]

Just to give a scenario, someone might want to have a role of say Time and then use this for units of time such as Hour, Min and Sec so can we say role Time of int and then do role Hour of Time? guessing here but if roles are actual types I see no reason why it wouldn't be possible.

Oh, indeed, I have plenty of scenario as well, that's why I'm asking! 😉 Note that roles are not really actual runtime types, only compiler time type (not saying you implied this, but I emphasizing this for a casual reader)

[Kukkimonsuta]

A follow-up question to consider is object identity. Should a "witnessed" reference type be able to compare reference equal to an "unwitnessed" cousin?

Do "witnessed" references preserve own identity for given instance?

IFoo f1 = myBar; // witnessed through extension
IFoo f2 = myBar; // witnessed through extension
if (f1 == f2) WriteLine("Equal!");

Use case:

interface IFoo { }
class Bar { }
public role BarFoo of Bar : IFoo { }

private HashSet<IFoo> _registry = new HashSet<IFoo>();
public void Register(IFoo foo)
{
    if (_registry.Contains(foo))
    {
        throw new InvalidOperationException("Cannot register twice");
    }
    _registry.Add(foo);
}

var instance = new Bar();
Register(instance);
Register(instance); // this must throw

[iam3yal]

@xoofx They may actually be a runtime type. :)

How can we make the runtime participate and make it work? Let's say that roles are actually represented in the runtime as a new kind of type, next to structs, classes, interfaces etc., rather than being compiled away "into something else". The runtime has roles!

[xoofx]

@xoofx They may actually be a runtime type. :)

How can we make the runtime participate and make it work? Let's say that roles are actually represented in the runtime as a new kind of type, next to structs, classes, interfaces etc., rather than being compiled away "into something else". The runtime has roles!

Ha, good spot... missed that part... but I'm not sure this is good. I was expecting the role information to be accessible at JIT/AOT time (via metadata), but not to create an entire new (reflectionable) type. The changes required to the runtime could be a significant burden and showstopper.

[xoofx]

This is why the (incomplete?) example of @Kukkimonsuta could be misleading:

You should not be able to pass (BarFoo) to the Register(IFoo) method. Only through a generic constraint:

public void Register<T>(T foo) where T : IFoo
{

And in case of a role being use through a generic, the code would be "instantiated". No interface calls would be generated at all.

Otherwise we are going to have a trait pointer like rust that could be 2xpointers (a pointer to a implementation through the interface IFoo and a pointer to the object instance)... I don't think this is sustainable at this stage of the existing "legacy" of the .NET runtime....

[JesperTreetop]

As a long-time C# developer, I think Swift's approaches, which witnesses and shapes pretty much allow, are good examples. The reason I like these corners of the language being explored is to eliminate friction and make some things possible.

In Swift, it's completely possible to invent a new protocol to mean, for example, "encodable in this form", and then add extensions to existing objects to show how they implement this protocol. That makes the problem solving more regular, since even the system types can now be handled in the same way as your own types.

While I like duck typing in languages that are dynamic, I like the simple and direct approach: yes, you do have to declare your conformance but anyone can add an extension and add that conformance. It seems like a better way to remove the tension. It also mostly removes the "nominal typing" tension: not everything with these properties will conform, but it's five seconds of work to declare conformity and express that intent - which is what static typing is all about. And you don't have to spend time wrapping things in a potentially semantics-changing, memory-impacting way.

This would be a problem if interfaces were supposed to be closed, like something anyone couldn't implement willy-nilly, but that's not how it is in practice in C#. If you can't see an interface because of visibility, you can't implement it either, so the boundary remains very clear. However, implementing known interfaces could still break assumptions in other code, but that's the case with making your own types implement new interfaces too, to some degree, so it's not a whole new class of problems.

The exploration in this issue is like music to my ears as someone who wants to get stuff done in my code. I might think differently if I thought C#'s all decisions should remain the same way from C# 1. With this, there's a new distinction to make: what does a type naturally do, vs what has it been tarted up in the current environment to do. This is a long debate with reasonable arguments on both sides, and all I know for sure is that if C# goes this way (which I personally hope), it shouldn't go there half-assed. If it breaks the tradition, it should at the very least let us do the new things that you would want to do (like implementing other people's interfaces no matter what they think about it).

Extension methods have been the toe in the water for a long time and the reason for this issue is that the community wants more power. (And even the C# 2.0 developers wanted generic operators and "INumeric".) Maybe there should be a way to see the type without extensions, but it's hard to do that without introducing confusing forks in all consuming code. It seems better to me to just jump in.

[JiriZidek]

@Kukkimonsuta But you see how awful it is. :-) When I look to our codebase, I see lot of "role" or "Role" names. I don't remember any "async" name before C#5. Its because async is adjective, and role is a noun. And adjectives are rarely used as names.

[Konard]

Just give me static extensions to extend static classes such as Assert class.

[Serentty]

I like this new proposal a lot more than shapes. Reusing interfaces is nicer than competing with them. I would like to see this implemented in a way that doesn't get ugly with reflection, even if it means that it has to wait longer for CLR changes.

[Serentty]

Also, I agree that interop should be a priority in the design, and that it should map nicely to CLR types. Right now it's mostly just F# to worry about since VB isn't being kept up with changes anymore, but keeping things open for new and small .NET languages to expand the ecosystem is a very good thing.

[Serentty]

@juepiezhongren I agree that roles are good enough to cover the use case of straight-up extension, and that the latter probably isn't necessary if its only purpose is just to save you the effort of creating a role. Especially because it seems to me like it complicates reflection. It would be nice if, provided that you didn't convert it to the underlying type, reflection could tell you which role you were looking at and which interfaces it added. I don't see how that could be done with straight-up extensions.

[still-dreaming-1]

Since this is talking about roles, I wonder if this would help make DCI more naturally expressed in C#, or rather I wonder if it would result in using DCI within C# making your code more clear and less bug prone than not using DCI in C#. DCI was invented by the inventor of MVC. DCI promoters claim "roles" are the missing piece in object oriented programming. When done properly, DCI is supposed to make code read, write, and behave like technology that is a natural extension of our mind. It is supposed to help the programmer think like an end user, and help the program work the way the end user thinks it will work.

https://www.artima.com/articles/dci_vision.html
https://github.com/jcoplien/trygve

[Iron-E]

We've been talking on the discussion about what it would look like if interfaces stayed the same but a parameter keyword was added to indicate the interface's contract is fulfilled with duck typing (i.e. through shapes).

For example:

private void Foo(IList<string> bar) {…}

// use shape behavior
Foo(implicit x);
// use classic `implements` behavior
Foo(y);

That way there's still no second type of interface (at least not for the C# user), and the feature will only be used when a developer explicitly signifies that a value should be passed with this 'shape' behavior. That will help save on performance, since it can also be that values are not implicit for a function call (when developers can implement an interface on a class they control).

[mharthoorn]

Regarding the syntax for extension interfaces: the current syntax in this proposal is "along the lines of"

public extension LevelCompare of Level : IComparable<Level>
{
    public int CompareTo(Level other) => (int)this - (int)other;
}

I have been thinking for quite a while what I don't like about this syntax and realized
we can make this in line with existing C# subclass syntax: an extension interface is
very very similar to a sub class, with the difference that it is still the same type.

Take the following example, where the sub class implements a new interface on an
existing class.

public class Cat
{
    public void Miauw();
}

interface IAnimal
{
    void Sound();
}

public class AnimalCat : Cat, IAnimal
{
    public void Sound() => Miauw();
}

This is very readable syntax imho. And just as a sub class can only derive from one class, but
multiple interfaces, so can an extension or role only be about one class and one (maybe more) interfaces.

We only have to replace class here with extension, shape or role etc.:

public extension AnimalCat : Cat, IAnimal
{
    public void Sound() => Miauw();
}

Or take the original example:

public extension LevelCompare : Level, IComparable<Level>
{
    public int CompareTo(Level other) => (int)this - (int)other;
}

[0x0737]

@mharthoorn

This will probably work with extensions, but not with roles if they will have the ability to form hierarchy.
But I'm not sure if something like that is considered. But it could be a nice addition if I've understood everything right.

public role RoleA of Class : InterfaceA { }
public role RoleB : RoleA of Class : InterfaceB { }

Also, the casting could work this way:
If we have

public role RoleA of Class : InterfaceA { }
public role RoleB : RoleA of Class : InterfaceB { }
public role RoleC : RoleA of Class { }

We could do the following conversions:

Class <-> RoleA
Class <-> RoleB
Class <-> RoleC
RoleA <-> RoleB
RoleA <-> RoleC
RoleA <-> InterfaceA
RoleB <-> InterfaceB
RoleC <-> InterfaceA

But we couldn't do

RoleB <-> RoleC

since sideway conversions are not allowed

Moreover, roles derived from ClassA could use any class derived from ClassA as their underlying type:

public class ClassA { }
public class ClassB : ClassA { }

public role RoleA of ClassA : InterfaceA { }
public role RoleB : RoleA of ClassB : InterfaceB { }

This will probably require full virtual static members support for implementation, but I'm not sure

P.S. The complexity lies in conversions between roles that override the underlying type of base role. What I've said may even be impossible, more research is needed.

[ronnyek]

I was pointed at this issue in asking for something along the lines of typescript's Partial<T> utility type.

I'm confused about the intention of this? Is this concept of roles to be able to deal with subsets of functionality / or data while still maintaining OO functionality?

Eg, my primary use case would be similar to the first example where you have some object that represents "everything". I'd want to have "views" or partial's of that type where I could conditionally include certain members (for filtering, or composing types specific to how they are being used), but still maintaining type safety etc.

I feel like in the grand scheme of things, having an object that represents your data model, and then 20 diff classes that represent subsets of the same data with the same field names etc, and having to copy data around back and forth is overhead, maintenance, and frankly just silly.

I'm trying to understand if this proposal would address that concern.

In the first example it seems to treat the original data object as the "source", but does this concept of Roles depend on the official source being effectively a dictionary of values?

public role Customer of DataObject
{
    public string Name => this["Name"].AsString(); //<-- couldn't this effectively happen today?

In the proposed Role functionality, would the underlying type (dictionary of values) contain any type information about its properties or would we effectively be casting/converting 100% of the time.

[theunrepentantgeek]

The underlying type of a shape can be any kind of struct or class, it's not restructed just to dictionary style property bags.

Shapes (as described here) are more akin to interfaces - but with the very useful wrinkle that, unlike interfaces, the underlying type doesn't need to explicitly opt-in to doing so. When the compiler assesses whether a particular type conforms to a particular shape, extension methods (and potentially extension properties etc) are used as well.

Mads' original post on this issue goes through all of this in quite some detail, it's worth the time to read it in detail.

[En3Tho]

What could you say about an approach like this?
The idea is using as many existing features as possbile, like interfaces and extension methods and do not add a new language construct like shapes.
It requires new (static) extension property/methods syntax though.
No need to create an explicit shape or explicit witness.

/// define an iaddable interface
interface IAddable<T>
{
    static IAddable<T> operator +(IAddable<T> left, IAddable<T> right);
    static IAddable<T> Zero { get; }
}

/// define a method that consumes a shape of IAddable, e.g. it can be any type that either implements IAddable directly or just has "shape" of it
public TAddable Add<TAddable, T>(TAddable left, TAddable right)
    where TAddable: shapeof IAddable<T>
{
    return left + right;
}

/// now create a type with only + operator defined
public struct AlmostIAddable
{
    private int value;
    public AlmostIAddable(int val) => value = val;
    public int Value => value;
    public static AlmostIAddable operator +(AlmostIAddable left, AlmostIAddable right) => new AlmostIAddable(left.value + right.value);
}

/// AlmostIAddable is missing Zero static property
/// so just extend that struct via extension static property
public static class AlmostAddableExtensions
{
    public static AlmostIAddable AlmostIAddable.Zero => new AlmostIAddable(0); // static property syntax

    // small ideas for more extension methods syntax
    // static extension method for a type
    public static AlmostIAddable AlmostIAddable.Foo(int a, int b) => new AlmostIAddable(a + b); // static method syntax    

    // instance extension property syntax
    public static AlmostAddable PlusOne => new AlmostIAddable(this.Value + 1); // implicit this
    public static AlmostAddable PlusOne { get => new AlmostIAddable(this.Value + 1); } // implicit this
    public static AlmostAddable PlusOne { get => new AlmostIAddable(this.Value + 1); set => new AlmostIAddable(this.Value + value) } // implicit this and value from setter
}

var left = new AlmostIAddable(0);
var right = new AlmostIAddable(1);

Add(left, right); // witnessed with static Zero extension property, no new types needed, compiler will choose a native implementation if there are conflicting methods or issue an error if there are only conflicting extension methods (I believe it should be a very rare ocasion)

[HamedFathi]

@MadsTorgersen

Does it cover extension for static classes too? like Console, Path and ...

[AlexanderMorou]

@MadsTorgersen
I'm a bit confused and concerned about this concept. Perhaps you could clear a few things up for me?

Your simple examples show introducing a language feature to generate what amounts to type-checked interfaces that are erased with the lambda implementations dropped in place.

My biggest concern with your initial sample of:

public role Order of DataObject //...

Can't the example shown be as simply suited using a purpose built library that dynamically generates interfaces? The sample you provided of all string-properties using .AsString() or .AsWhateverType() can just as easily be implemented in a library.

I only know this because I tried to implement one.

Since you follow a pattern, it may actually be preferable to implement it in this manner (using a library vs a language feature). The library approach could allow you to tailor optimizations specific to the implementation, for instance:

var orders = customer.Orders;
for (int i = 0; i < orders.Count; i++)
{
 //Do something with each order.
}

In the implementation of the list-based element, I would do something like so:

if (typeof(TGeneric).GetGenericMatchFor(typeof(List<>)) is GenericTypeMatchInformation genericInfo
    && genericInfo[0] == typeof(TElement)
    && typeof(TElement).IsInterface)
{
    var propertyName = propertyInfo.Name;
    Lazy<Func<JToken, TElement>?> lazyFactory =
        new Lazy<Func<JToken, TElement>?>
        ( () => {
                    if (typeof(TElement).HasRole())
                        return RoleExtensions.GetFillRoleFactory<JToken, TElement>();
                    return null;
                });
    return x => 
    {
        var jTokenTarget = x[propertyName];
        if (jTokenTarget == null)
            return default;
        if (jTokenTarget.Type != JTokenType.Array)
            return default;
        var annotation = jTokenTarget.Annotation<List<TElement>>();
        if (annotation == null
            && lazyFactory.Value is Func<JToken, TElement> factory)
            jTokenTarget.AddAnnotation(annotation = new List<TElement>(x[propertyInfo.Name]!.Select(x => factory(x))));
        return (TGeneric?)(object?)annotation;
    };
}
throw new NotImplementedException();

The main things I can't implement are interface statics. As they are defined today, at least.

As a type-erased language feature, there are certain things you can't do as effectively. You could write the above for each and every property that needs a List, but it would be arduous and messy fast.

The above shows using annotations to add a makeshift instance-level cache to avoid the hit of recreating the wrapper. Yes, in scenarios where you are only planning on sifting through the data once and done, it would be a performance hit, instead of a boon. But you tend to tailor solutions to your needs.

For the case of members that need to be directly on the object, like Reload, I'd annotate the interface with a 'AsDuckType' attribute and it would pass it through as a direct call to the type, bypassing the role logic.

As neat as being able to use AddAll on a type is, I think it would be better suited to extending the use of type-parameters, instead of implementing a type-erased feature.

For instance:

where t looks like CThisConstraint

CThisConstraint is a constraint type used in places where you have several constraints used in many parts of the application. It'd basically do little more than define constraints. Such as .ctors, operators and the like.

There's a thousand ways to do the same thing, my concern with the feature as it is today is the erasure. This is due to what's lost on compilation. Java went that route with type-parameters, and I think C# is better for its retention of that information.

[JesperTreetop]

The type-parameter-as-constraint part at the bottom assumes that everything has the right shape from the beginning. The point of extensions and roles is that you can adapt instances of objects that already exist or that can't be changed to have the right shape to your code. That can be realized by creating wrapper objects ("witnesses"), but it could also be optimized away in some cases by the compiler by flowing sufficient information where it is in control of both the caller and the callee, which is going to be a lot of the time when it's your own extensions. If you're using a library to do this, you're producing code which purpose is to do the wrapping contortions, and the compiler is going to have a much harder time to prove that this is what you're trying to do and optimize it the whole way, and it's probably never going to even try. A language feature gives you a fighting chance. The library method producing code is essentially erasure of a similar kind to what you're mentioning; erasure of intent known to the compiler.

As a concrete albeit dull example, you could make a generic extension interface IIndexable<out T> for an indexable collection, add extensions to IReadOnlyList<T> and collections that implement indexers but not IReadOnlyList<T>, and then have a method that takes params IIndexable<T>[] indexables which could transparently take either type as an element.

If there's no adaptation by the extension at all but merely pointing through to an indexer you know is there, sure, that's doable with just duck typing, and I would have really liked duck typing as an intermediate step on the scale that begins with early binding with nominal types and ends with dynamic. But that's not what extensions are used for in languages that have them; they are used to control layering and to reshape things that aren't by construction similar, so that you can take advantage of their similarity without breaking the isolation of the original type. If they're putting all this time and effort into bringing it to C#, I definitely think they should be shooting for going all the way.

[AlexanderMorou]

@JesperTreetop That's the thing, though. The Constraint type would exactly represent the expectation in a very clear format that can be reflected upon.

Without the ability to dig into the data, you limit developers on what they can do with a given feature. Just like the interfaces and the dynamic types, should constraints allow dynamic solutions, many of the role-based solutions presented could have libraries built around them.

In a situation where that data is erased during compile, you turn it into a black box.

[ronnyek]

Correct me if I'm wrong, but this would potentially allow me to have a sort of underlying "generic" (not language generics per say) data object, and have projections of that data as strongly typed subsets of that data? Theoretically I assume this could potentially allow use to avoid DAO's and avoid usage of things like automapper etc?

If so, I think this is solving one of the biggest problems I encounter on a daily basis (and probably the thing about c# I complain about most).

If not, is there a simple summary of what problem it solves? I'm excited if it's solving the problem it seems like its solving.