[Proposal]: Roles and extensions

[MadsTorgersen]

Roles and extensions in C#

• Proposed
• Prototype: Not Started
• Implementation: Not Started
• Specification: Not Started

Motivation

In C# today, one assembly can use extension methods to augment types of another assembly, effectively adding new instance methods to it. This proposal expands on that principle so that:

1. All kinds of members, including static ones, can be added from the outside, as long as they don’t add instance state
2. Augmentations can be “roles” which selectively augment individual values, as well as “extensions” which augment all values of a type throughout a scope (like today’s extension methods)
3. Roles and extensions present as first-class types, with names and conversions
4. Stretch goal: Roles and extensions can implement interfaces on behalf of the underlying types and values, acting as adapters

Common to these is that they don’t require involvement of the underlying type, and don’t intrude upon its inherent semantics, thus facilitating new degrees of decoupling and separation of concerns.

The main purpose of the proposal at this point is to lay out the motivating scenarios and a comprehensive set of ideas for addressing them. The terms, concepts and syntax are all up for discussion. From a language design perspective there is a lot of new ground here, so it's ok - even necessary - for us to spend some time and effort making sure we end up giving it the best embodiment that we can.

Syntax

Acknowledging the design latitude, here’s the style of syntax I’ll use in the following:

role Name<T> : UnderlyingType, Interface where T : Constraint
{
    // Member declarations
}
extension Name<T> : UnderlyingType, Interface where T : Constraint
{
    // Member declarations
}

Extensions are just roles that apply automatically to their underlying type when brought into scope with a using directive. Roles and extensions have a name, can be generic, can implement interfaces and can declare members that don’t introduce instance state. Unlike base types elsewhere, the underlying type can be pretty much anything, including value types, type parameters, pointer types etc.

A much more detailed description of the syntax and semantics follows under "Detailed design" later on.

Motivating examples

These examples are meant to present motivating "archetypical" uses of roles and extensions. There are some bigger examples near the end of the document.

Throughout these examples I will make use of a simple DataObject type meant to represent weakly typed, semi-structured data, e.g., wire data:

public class DataObject
{
    public DataObject this[string member] { get; set; }
    public string AsString() { }
    public IEnumerable<DataObject> AsEnumerable();
    public int ID { get; }
}

None of the examples modify the type itself, but instead enhance it in various ways with roles and extensions.

Layering with extensions

API layering often conflicts with discoverability. We want to create APIs with only a one-way dependency, but members relating to the depending layer would be most naturally found in the depended-on layer. With extensions the depending layer can add members to types in the depended-on layer:

public extension JsonDataObject : DataObject
{
    public string ToJson() { … this … }
    public static DataObject FromJson(string json) { … }
}

Here the DataObject type is extended by a Json conversion API to have a static and an instance method, converting to and from strings representing Json.

The instance method of the extension can be considered a new and better syntax for the extension methods we already have in C# today. Bodies of instance extension members may refer to the extended object with the this keyword.

The name of the extension can be used for disambiguation purposes, similar to the static class containing extension methods today. We might consider allowing extension declarations without names.

Extensions can be brought into scope the same way as extension methods today: With a using static directive on the extension type or a using on its namespace:

using JsonLibrary;

var data = DataObject.FromJson(args[0]); // Extension static method
WriteLine(data.ToJson());                // Extension instance method

Lightweight typing

Wire data is most naturally represented with weakly-typed dictionary-like types such as DataObject, but those don’t lend themselves well to helpful tooling and elegant program logic. However, conversion to strong types is expensive and lossy, and all-out wrappers still introduce a level of indirection. Roles can provide strongly typed augmentation with a lighter touch:

public role Order : DataObject
{
    public Customer Customer => this["Customer"];  
    public string Description => this["Description"].AsString();
}
public role Customer : DataObject
{
    public string Name => this["Name"].AsString();
    public string Address => this["Address"].AsString();
    public IEnumerable<Order> Orders => this["Orders"].AsEnumerable();
}

Roles are transparent wrappers. They are considered identical to their underlying types and have implicit conversions in both directions. The conversions are free at runtime, because the underlying representation of the object doesn’t change. The Order is the DataObject. As the Orders property shows, even constructed types over roles convert freely. So now strongly typed, IntelliSense-aided code can be written without clunky indirection or runtime penalty:

IEnumerable<Customer> customers = LoadCustomers();

foreach (var customer in customers)
{
    WriteLine($"{customer.Name}:");
    foreach (var order in customer.Orders)
    {
        WriteLine($"    {order.Description}");
    }
}

It is easy to imagine roles for wire data being automatically generated, e.g. by a source generator based on a schema or sample. This would address a similar scenario to F# type providers, but with named types and generated source code you can debug through.

While there would be nothing technically difficult about allowing identity conversion between two different roles with the same underlying type, I think it would be useful to disallow it – or at least warn about it – making people explicitly take the indirect route through the underlying type. That way the roles provide some protection against accidentally converting things across the role hierarchy.

Adaptation to interfaces

We can imagine using roles and extensions to implement additional interfaces on values of the underlying types.
Say there’s some framework that you plug into by implementing interfaces such as this:

public interface IPerson
{
    public int ID { get; }
    public string FullName { get; }
}

We would like some of our person-like roles over DataObject to plug in to this framework. We can do this by expanding our declaration of Customer to:

public role Customer : DataObject, IPerson
{
    string IPerson.FullName => Name;
    // … plus same members as before
}

The declaration of Customer explicitly maps FullName to Name, but does not need to contain an implementation of ID because it “inherits” a suitable one from the underlying DataObject type.

Now the Customer role can be used to treat DataObject instances as implementing IPerson! It will satisfy IPerson as a generic constraint, and there would be an implicit conversion – likely a boxing conversion – from Customer to IPerson.

Combining roles and extensions

In the previous example, the developer (or source generator!) who declared Customer may not be aware of the person framework or interested in integrating with it. Again, this can be solved with an extension. The Customer role itself can be extended to fulfill the interface!

public extension CustomerAsPerson : Customer, IPerson
{
    string IPerson.FullName => Name;
}

Whoever imports this extension can then use unsuspecting Customers – who are themselves unsuspecting DataObjects – as IPersons!

Generalized aliases

Simple roles without bodies can serve as a better form of aliases, essentially rendering using aliases obsolete:

public role DataSet<T> : Dictionary<string, T>;

Unlike using aliases, such a "role alias" can be public (or any other accessibility), can be generic, can participate in the file's declaration space of types, can be recursive, etc. They are real type declarations, but still work as if they are just synonyms for the underlying type.

Detailed design

Role declarations are a new form of type declaration:

role_declaration
    : attributes? struct_modifier* 'partial'? role_keyword identifier type_parameter_list?
      ':' type role_interfaces type_parameter_constraints_clause* struct_body ';'?
    ;
    
role_keyword
    : 'role'
    | 'extension'

role_interfaces
    : (',' interface_type)*
    ;

A role is in some ways like a struct and in some ways like a class. Like a struct it cannot be derived from, and therefore cannot be abstract or sealed. It also cannot have virtual or overriding members (except perhaps allowing it to override the members of object similar to structs). It can implement interfaces, using a combination of its own members and ones inherited from the underlying type. Just like a struct, the role has a boxing conversion to those interfaces (though not directly to object).

Somewhat like a class, however, a role specifies a kind of "base type" which we call the underlying type. Unlike a base type the underlying type is mandatory. On the other hand it can be any type, not just a class type. (We need to think about syntactic ambiguities in this position for tuples, pointer types etc.) At compile time the role inherits the members of the underlying type, just as a derived class inherits from its base class.

(Open design question: A role could be allowed to specify additional base roles. They would need to be applicable to the specified underlying type, and would provide a way to combine roles through a form of "multiple inheritance".)

The role transparently "wraps" the underlying type in such a way that values of the role and of the underlying type have identical runtime representation. For this reason a struct_member_declaration in a role body is prohibited from declaring additional instance state, either as a field, an auto-implemented property or a field-like event, on top of what's inherited from the underlying type.

The underlying type of a role can be any type, even an interface, a value type, an array type, a type parameter, the dynamic type, a pointer type or another role. Thus a role can be used to enhance types that cannot normally be extended with new functionality. Certain kinds of underlying types are subject to restrictions that limit how the role can be defined or used. For instance, a role on a ref struct cannot specify interfaces, and does not have a boxing conversion. A role on a pointer type or ref struct cannot be used as a type argument or an array element type.

In general, operations on a role are resolved exactly like those on a derived class: try to use the declarations in the role first, and if those aren't found, use the underlying type. The concrete mechanics of this for each operation needs to be specified in detail, but should generally be as similar as possible to how the corresponding operations are handled in a derived class.

The identical runtime representation of a role and its underlying type enables identity conversions to exist both ways between them. This in turn leads to the existence of identity conversions between constructed types where roles and their underlying types respectively are used as type arguments, as well as between array types where the element types are roles and their underlying types respectively.

Identity conversions in C# are generally transitive, which would allow them to apply not just to and from the roles and their underlying types but also between roles that have the same underlying type. However, it might nevertheless be reasonable to discourage direct conversion between unrelated roles on the same type, e.g. with warnings or even errors, so as to enable the enforcement of a "shadow type hierarchy" of roles. We have precedence for such diagnostics on identity conversions between tuples using different element names.

In addition, there is an implicit boxing conversion between a role and each of the interfaces it implements. This means that a new object is created whenever the role is converted to the interface, and the result will thus not be reference-equal to the underlying object, even if that is a reference type.

Any reference conversions, implicit and explicit, to and from the underlying type, are inherited by the role, although its own identity and boxing conversions take precedence, along with any user-defined conversions declared in the role.

The identity, reference and boxing conversions on a role can be used to satisfy generic constraints when the role is used as a type argument. This is the source of much of the feature's expressiveness, because roles can be used to "bridge" or "adapt" an existing class or struct (or other type) to an existing interface without modifying either, in such a way that values of the existing class or struct can be passed to generic methods using the existing interface as a constraint.

An extension declaration is simply a role declaration with a different keyword. In addition to all of the above, when an extension is in scope - either directly or via being brought in through a using or using static directive - it automatically applies as a "fallback" role for the underlying type. Similar to existing extension methods in C#, if a lookup fails on the underlying type, it is tried again against the extension. If multiple extensions apply, ambiguities between applicable members are resolved in the same manner as for extension methods.

In addition to member lookups, extensions can apply when the underlying type is used as a type argument, allowing interfaces implemented by the extension to help satisfy constraints on the corresponding type parameter.

Implementation strategies

It is a cornerstone of the proposal that there is an identity conversion between a role and its underlying type, and that just like other identity conversions in the language it is essentially free – there is no representation change at runtime.

One implementation strategy is to use “erasure”, just as we do with dynamic vs object, with tuple names and with nullability annotations. Here, in the IL we generate, we really do use the same type (the underlying type in this case) and then use other metadata (attributes etc.) to communicate the extra information about which language-level "embodiment" of the type is meant here. In this way, at runtime, the value doesn’t change its type at all and the identity conversion is trivially “free”. Even for constructed types, e.g. arrays, the conversion is free because the underlying type is the same:

DataObject[] dos = new Customer[10]; // Erasure: Customer[] is really DataObject[]

Of course the added members still need to be encoded somewhere; it is likely that we would generate some form of type declaration under the hood to hold them.

Unfortunately, and erasure strategy will come up short for the scenario of roles and extensions implementing interfaces. Consider this generic data structure:

class Rolodex<T> : IEnumerable<T> where T: IPerson
{
    Dictionary<string, T> people = new ();
    void Add(T person) => people.Add(person.FullName, person);
    T Find(string name) => people[name];

    public IEnumerator<T> GetEnumerator() => people.Values.GetEnumerator();
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

Assuming Customer implements IPerson, we should be able to create a Rolodex<Customer>:

var rolodex = new Rolodex<Customer>();

We can’t just “really” pass DataObject as the type argument because it doesn’t satisfy the constraint. There needs to be a real Customer type at runtime so that the constraint can be satisfied and the call to person.FullName in the body of the class can be resolved. Values of the Customer type must have identical runtime representation to their underlying DataObject, and the runtime must be able to understand that it can freely convert between them as a no-op, using the same bits. Essentially it needs to be like a wrapper struct: The bits are the same, the type is different.

Furthermore, the returned Rolodex<Customer> implements IEnumerable<Customer> and DataObject is identity convertible to Customer. So Rolodex<Customer> must be implicitly reference-convertible to IEnumerable<Customer>! The runtime needs to understand and allow this conversion:

IEnumerable<DataObject> dataObjects = new Rolodex<Customer>();

In short, there are things the runtime will need to understand about roles if we embrace their ability to implement interfaces.

More examples

Implementing IEnumerable<T> with an extension

Say we want to view a ulong as an IEnumerable<byte>. We can do that with an extension implementing that interface:

public extension ULongEnumerable : ulong, IEnumerable<byte>
{
    public IEnumerator<byte> GetEnumerator()
    {
        for (int i = sizeof(ulong); i > 0; i--)
        {
            yield return unchecked((byte)(this >> (i-1)*8));
        }
    }
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

Now, wherever the extension is brought in you can iterate over the bytes in a ulong:

foreach (byte b in 0x_3A_9E_F1_C5_DA_F7_30_16ul)
{
    WriteLine($"{e.Current:X}");
}

You can also use its boxing conversion to IEnumerable<byte> to access LINQ functionality:

const ulong ul = 0x_3A_9E_F1_C5_DA_F7_30_16ul;
var q1 = Enumerable.Where(ul, b => b >= 128);  // method argument
var q2 = ul.Where(b => b >= 128);              // extension method receiver
var q3 = from b in ul where b >= 128 select b; // query expression
// etc.

Implementing IComparable<T>

IComparable<T> is most commonly used as a constraint on generic types or methods that need to compare several values of the same type.

With extension interfaces we can make types IComparable<T> that wouldn't normally be. For instance, if we have an enum

public enum Level { Low, Middle, High }

We can make it comparable with the extension declaration:

public extension LevelCompare of Level : IComparable<Level>
{
    public int CompareTo(Level other) => (int)this - (int)other;
}

We can also extend only certain instantiations of generic types with an interface implementation. For instance, let's make all IEnumerable<T>s comparable with lexical ordering, as long as their elements are comparable:

public extension EnumerableCompare<T> : IEnumerable<T>, IComparable<IEnumerable<T>> where T : IComparable<T>
{
    public int CompareTo(IEnumerable<T> other)
    {
        using var le = this.GetEnumerator();
        using var re = other.GetEnumerator();
        while (true)
        {
            switch (le.MoveNext(), re.MoveNext())
            {
                case (false, false): return 0;
                case (false, true): return -1;
                case (true, false): return 1;
            }
            var c = (le.Current, re.Current) switch
            {
                (null, null) => 0,
                (null, _) => -1,
                (_, null) => 1,
                var (l, r) => l.CompareTo(r)
            };
            if (c != 0) return c;
        }
    }

Now we can use an IEnumerable<T> as an IComparable<IEnumerable<T>> wherever this extension is in force, but only as long as the given T is an IComparable<T> itself. For instance, an IEnumerable<string> would be comparable to other IEnumerable<string>s because string is comparable, whereas an IEnumerable<object> would not, because object is not.

So with these two extensions in force, we can now compare two arrays of Levels:

using static Level;
Level[] a1 = { High, Low, High };
Level[] a2 = { High, Medium };
WriteLine(a1.CompareTo(a2));

Because of the LevelCompare extension on Level it satisfies the constraint on the EnumerableCompare extension on IEnumerable<Level>, which therefore in turns makes the Level[]s comparable!

Conclusion

Roles and extensions support proper separation of concerns, lightweight typing and adaptation, reducing dependencies and the costs of wrapping and conversion. There's a good amount of design work left to do, and in their full generality they are probably a fairly big feature to implement, requiring the participation of the runtime.

[HaloFour]

It's awesome to see some movement in this space.

I have some concerns of the loose typing with roles. I understand that it's intentional but it feels like it could very easily lead to situations where you're treating an instance as the incorrect role with undefined results. To me it also begs for a way to pattern match the underlying type to a given role, perhaps through a conditional member declared on that role. Otherwise how could you ever tell that a DataObject is a Customer vs an Order?

I'm also curious as to where roles/extensions fit into the conversation around shapes and structural typing. Assuming that interfaces remain the mechanism through which you define the constraint and this feature does hit the stretch goal of supporting extension implementation does it require manually writing a bridge/witness from an arbitrary type to that interface or can the compiler provide one given the appropriate extensions are in scope? I recall there was also a conversation about runtime changes necessary in order to avoid boxing the witness?

[MgSam]

I'm glad to see this thread (structural typing) being picked up again.

A few initial impressions:

• role and extension feel extremely similar to the point that it seems like there should be a way to marry the two concepts
• I think finding a way to "extend" interfaces onto existing types is where 99% of the value is and should be a core, not a stretch goal. I say this with an extensive background in TypeScript enjoying the benefits of structural typing. The ability to just use data as it is without copying it all over the place is really what this proposal is trying to achieve.
• I still feel that "no new instance state" for extensions/roles is too harsh of a restriction. The language and framework have the tools to do this (ConditionalWeakTable), choosing to make them hard to use because it might have performance implications if used incorrectly is too harsh of a decision. You guys did add dynamic into the language, after all. It doesn't come with a warning telling you not to use it because its expensive; it's just something you learn to use appropriately as a C# developer. As with any sharp tool, you need to know how to use it to not cut yourself. Instance state in extensions should be the same way. Give us the tools, let us make the decisions as to when to use them.
• I'm curious as to why the shapes proposal was apparently jettisoned wholesale. I don't recall it being discussed heavily in LDM but maybe I'm misremembering

[HaloFour]

@MgSam

I'm curious as to why the shapes proposal was apparently jettisoned wholesale. I don't recall it being discussed heavily in LDM but maybe I'm misremembering

I believe that was covered in the previous roles discussion that detailed the "Comparison to previous proposals".

The two major remaining downsides of the shapes proposal are: a) it still introduces a whole new abstraction mechanism, that in many ways competes with interfaces for expressing contracts. And b) for this reason, there is no way of adapting types to existing generic code that uses interfaces, not shapes, as constraints.

I chalk it up to being spaghetti thrown at the wall.

[qrli]

Inspired by the discussion: Is it possible to have roles implemented as interfaces attached to target type (similar to default interface implementation)? In that way:

• There will no new Role type, but only a new way to attach interface implementation to classes/structs.
• All roles will be interfaces (with implementation), thus will work with existing interface constraints.
• The typing & type conversion of roles may follow pretty much of how interfaces work.
• When roles implement interfaces, it would be the same as interface implementing other interfaces.

[YairHalberstadt]

The following two design aims seem difficult to reconcile, both semantics wise and implementation wise:

1. If R is a role/extension of T, there should be an identity conversion from a constructed type of T to a constructed type of R.
2. If R/T implements an interface, it should be able to be used as a type parameter which is constrained to that interface.

Here's a couple of examples where this doesn't make sense:

Example 1

Since there is an identity conversion between List<DataObject> and List<Customer>, this can only work by erasure - if you have to wrap or copy the instance of List<DataObject> to get that to work that's no longer an identity conversion.

This leads to the following unexpected behavior:

var dataObjects = new List<DataObject>{do1, do2};
var customers = (List<Customer>)new List<DataObject>();

Console.WriteLine(customers.GetType()); // prints "System.Collections.Generic.List`1[DataObject]"
Console.WriteLine(((object)customers) is List<Customer>); // prints "False"

Meanwhile if I use a role to fulfill a constraint this can't work through erasure. So if I have the following:

public class PersonList<T> : List<T> where T : IPerson

var customers = new PersonList<Customer>{d0, d1};

Console.WriteLine(customers.GetType()); // prints "PersonList`1[Customer]"
Console.WriteLine(((object)customers) is List<Customer>); // prints "True"

So there's this weird situation where the runtime type of a constructed type of a role/interface sometimes reflects that it's the role/interface and sometimes doesn't which I expect to be highly confusing.

Example 2

Consider the following:

class A {}

class NamedWrapper<T> where T : INamed
{
     public void PrintName() => Console.WriteLine(T.Name);
}

interface INamed
{
    public static string Name { get; }
}

namespace N1
{
    extension E1 : A, INamed
    {
        public static string Name => "Monty";
    } 
}

namespace N2
{
    extension E2 : A, INamed
    {
        public static string Name => "Python";
    } 
}

namespace N3
{
    using N1;
    public NamedWrapper<A> GetNamedWrapper()
    {
        var namedWrapper = new NamedWrapper<A>();
        namedWrapper.PrintName() // prints "Monty"
        return namedWrapper
    }
}

namespace N4
{
    using N2;
    using N3;

    public static class Program
    {
        public static void Main()
        {
             var namedWrapper = GetNamedWrapper();
             namedWrapper.PrintName(); // prints "Monty"
             var namedWrapper2 = (NamedWrapper<E2>)namedWrapper; // identity conversion
             namedWrapper2.PrintName(); // Should print "Python", but will in fact print "Monty" because it's an identity conversion
        }
    }
}

I think that the idea of having identity conversions between constructed types to constructed types of roles and interfaces just isn't really doable.

[En3Tho]

I agree with @MgSam about the ability to implement interfaces through type extension. I believe this should be the main goal. Because then it will allow decorating any type (and this js like type too) with static typing anyway (just declare an interface and extend that DataObject or dictionary or anything). But not vice versa.

[TahirAhmadov]

Wire data is most naturally represented with weakly-typed dictionary-like types such as DataObject, but those don’t lend themselves well to helpful tooling and elegant program logic. However, conversion to strong types is expensive and lossy, and all-out wrappers still introduce a level of indirection.

I disagree with this statement. The hypothetical DataObject has a dictionary of some sort behind the scenes. In the typical deserialization scenario, be it XML, JSON or binary, properties will be added to the dictionary. This incurs the standard dictionary costs: key lookup and behind the scenes allocations, as needed; plus the overhead during get operations. On the other hand, a properly implemented deserializer which creates an optimized type-specific method (using something like LINQ compilation or IL emit) can output code which sets target properties directly - no dictionary overhead; and the result is a strongly typed DTO, whose properties can be accessed natively with no overhead. I think it's a bad motivational example for the roles proposal.

[TahirAhmadov]

Looking at this more, it looks like the extension part of this is great, especially including the interface "adapters".
On the other hand, the role part is counter intuitive, introduces runtime difficulties, and is motivated by anti-patterns, IMO.
Even if we forget the DataObject scenario for a moment and look at strong types, role Customer for class Person has a code smell. If you have people, some of whom are customers and others are staff/vendors/etc., this needs to be stored somehow, both long term and in memory. These "roles" will have their specific attributes; for a customer, it may be shipping address, for an employee, it may be salary, etc. Therefore, these "roles" should really be their own types; trying to re-purpose Person as Customer just isn't a good solution.
And this is before you go full "enterprise" where a customer can be a company or a person, etc. - completely incompatible types.
In short, thumbs up for extensions and interfaces; thumbs down for roles.

[iam3yal]

I really like it but I do feel like it should be a single concept instead of having roles and extensions an extension should either extend an existing type or wrap an existing type what if in the example above I want to use JsonDataObject on Customer?

Here is an example of what I mean.

// Customer.cs
namespace Data;

public extension Customer : DataObject // Wrapper type
{
    public string Name => this["Name"].AsString();
    public string Address => this["Address"].AsString();
    public IEnumerable<Order> Orders => this["Orders"].AsEnumerable();
}

// JsonDataObject.cs
namespace Data;

using JsonLibrary;

public extension JsonDataObject : DataObject // Extension type
{
    public string ToJson() { … this … }
    public static T FromJson<T>(string json) { … }
}

// Program.cs / Main method
using Data;
using Data.JsonDataObject; // Importing a specific extension type
using Data.*; // Importing all extensions types in the namespace Data

var customer = customer.FromJson<Customer>(args[0]);
WriteLine(customer.ToJson());

[CyrusNajmabadi]

I want to use JsonDataObject on Customer?

I don't see why you wouldn't be able to. a Customer is a DataObject (both in terms of type hierarchy, and in terms of actual raw erased value). So all of JsonDataObject would apply to it.

[iam3yal]

@CyrusNajmabadi Okay thank you but still I think and feel like extensions and roles can be a single concept but applied differently but then that's just my opinion.

[CyrusNajmabadi]

@eyalalonn We just had an LDM meeting on htis. Trust me that that sentiment is understood and we're definitely going to be thinking heavily about these concepts and hte best way to expose them :)

[333fred]

Sorry everyone, I pressed convert on the wrong github issue. I converted this discussion back to a full issue, but it didn't move the comments back. The new issue is #5497.

[Xyncgas]

it's way better than declaring static extension methods, naming is hard

[Rekkonnect]

You still need to name your extensions and roles. Perhaps they could make it so that you could declare anonymous extensions and roles for a type, which would be great.

[juepiezhongren]

with role appearing, openxml could be reimplemented, with data being simple xelements and no more overhead.

[TahirAhmadov]

Actually that would be a bad idea. Different nodes have different properties. Roles would not work here.

[juepiezhongren]

in openxml, all properties are in fact backed or at least synced with xml attribute, i can imagin the performance improvement and the decomplexation once role-based api r available

[TahirAhmadov]

Yes but there is no reason to treat a "para" node as a "run" node. It opens the door for bugs. Strong typing ensures that only relevant properties can be set. The fact that behind the scenes it's all XML attributes doesn't negate the need to differentiate between different types of nodes.

[juepiezhongren]

role and extenaion are basically the same, shall we make a way to allow them inter transformable?

[Phantonia]

Maybe roles (not extensions) can enable something like constrained types, where instances of the role are just instances of the underlying type but guaranteed to fulfill a condition.

An example:

role EvenInteger : int
    where this % 2 == 0
{
    public int Halve() => this / 2;
}

Any cast int -> EvenInteger would need to be explicit and throw an exception if the condition is not fulfilled.

int x = 4;
EvenInteger evenX = (EvenInteger)x; // works

int y = 3;
EvenInteger evenY = (EvenInteger)y; // throws

Other examples:

role NumericString : string
    where this.All(c => char.IsDigit(c))
{
    public int ParseAsInt() => int.Parse(this);
}

role ValidEnum<T> : T
    where T : struct, Enum
    where Enum.IsDefined(this)
{
}

[0x0737]

This is very useful if supported inside is/as

[En3Tho]

This might be an interesting idea. Basically it's like F#'s aliases or actually closer to types with units of measure attached that are erased in run time but with even stronger contracts.

I guess this sorta has next pros compared to struct wrapper:

1. Well, you don't need a wrapper.
2. Can be used to change collection "types" with no cost, e.g. from EvenInt[] to int[] because it's the same type.
3. Can in theory be combined with other roles without any new type/wrapper creation.
4. Can in theory extend pattern matching akin to active patterns, value could be compared with different roles at the same time or with one combined role. Could be flexible. Eg.

if 10 is EvenInt and PositiveInt value { ... }

But the next cons still exist:

1. You still can make an improper default value even with roles, similar to struct wrapper.
2. Casts from int[] to EvenInt[] or from one role to another are "unsafe", as with struct wrappers.
3. Harder to use with reflection and code generation as there is no type actually (but I may be wrong and this could be extended with new reflection apis that respect roles)
4. Interop with other .Net languages could be worse than with struct wrappers because at least struct wrappers share a common contract in all .Net (default defeats everyone otherwise it's ok) while with roles it's C# only feature. Not sure how will older versions of C# consume roles (as "unroled" underlying type I guess).
5. If compiler somehow "forgets" or you make compiler lose information that your value has a role eg you pass this numeric string into a method that accepts a string, then this might result an additional checks next time. I guess this is again the same with struct wrappers. But in case of struct wrapper you can make an additional method to "force" object creations. Not sure if roles will provide similar functionality.

Also, in your example you've used an expanded contracts which are not part of this proposal but maybe this could move them forward too.

[Phantonia]

Couple things I would like to note here. To respond to some cons:

1. That would indeed be problematic. I'd assume the compiler would generate the condition as a public method IsValid to manually check. This would probably also benefit from work in the field of non-defaultable structs? See Championed: Non-defaultable value types #146, Defaultable Value Types #5337.
2. Imo the mistakes C# 1 did with array variance should be entirely avoided by making array variance with any type and a corresponding constrained role illegal.
3. Assuming roles are implemented with runtime support, reflection APIs would need to be extended anyway, imo. So reflection for constrained roles would only be natural.
4. I'm not sure how any other .NET language would deal with roles in general. Unless their respective compilers would be modified to understand roles (assuming they are implemented using runtime support) they would break with any role, not just constrained ones, or am I missing something here?

[En3Tho]

Well. Personally I'm still not sure what roles are (valid full fledged types or a compile time "masks"). I actually see the potential in latter as I personally use TypeAliases in F# quite frequently. They can give an additional context to your code without creating new types which is a good thing (especially when trying to refactor old code bit by bit or make code more readable and understandable) but at the same time they do not protect you from misapplying values as they are just aliases.

Roles faintly resemble this approach but I believe they are thought to have much broader functionality. Eg either enriching a value with additional behaviour or constraining, validating, matching value etc.

Maybe they coils even help with using unsafe types by not creating additional types but instead masking those unsafe types with safe operations.

All this is very similar to struct wrappers but I still believe that roles could be designed to have lesser typing overhead eg. creating DDD style typed ID's from Guid values, maybe reducing startup times by reducing types that need to be loaded.

I guess I would like to play with this kind of thing first. And my thought of what roles might be can be very different to what design team have in mind.

[WispArchitecture]

I'm not sure I understand this. But I think I'm hearing:

Currently, if I have object A, I can make instances of it act like X by using a file that has extensions on A.
A key limit on this is that my data scope for A is restricted to A's public surface.
Discussed versions of this proposal let me name an "enhanced" extension (a Role) and potentially add "non-state" attributes.

1. I can then "shape" or mod an object by using Roles.

But doesn't using apply to the whole file? Might this introduce unintended side effects?
What happens when two using have conflicting extensions?
Will these mods be available on Nexus and can I use MO2 to manage them?

2. Or Role is a wrapper you can declare in an object to type and constrain data? (Business Record?).

Mostly, I see a lot of infrastructure required to manage rules and assets around this. Without the right tooling, current extensions can seem like "Ghost Modifiers" - Class instance A in File1 does this. But an instance of the same class in File2 acts differently. Sure, we already deal with this, but for fun try writing a fair-sized app using only extensions. They become difficult to manage. (Pasta) One reason is they don't naturally align with the &$%@# "DirectoryPath = namespace" approach we default to.

Personally, I think this is solving a problem that shouldn't exist. We commonly treat objects as (relational) data and not classes or objects (in the classic OO sense). So we don't get the benefits of OO. Even there, the Data-Procedure OO fork we mostly use is outdated. The State-Message fork (coincidentally championed by the same people designing ARPANet) is better suited to modern async/distributed/container-component applications. Parts of this seen to entrench this more, and lead to Ghost Classes (static class extensions). (Which I really want even though I know better lol)

In some ways I'm starting to identify with people that feel the language is getting too "big". I like new features. I can choose to use them or not. But I don't like ones that break things. Or ones make the language internally inconsistent. Try to build an app where your classes don't have a parameterless constructor. Whole sections of the environment won't work - "...requires a parameterless constructor". K. Added. "...not nullable field must be initialized before leaving constructor."

[najak3d]

This is one of the most needed/brilliant ideas I've seen for C#. It's an efficient/natural way to avoid having to "wrap an existing Class in another class to make it implement an interface (even if the existing object already complies with the interface). I agree with the idea of both having roles and extensions -- as one is done explicitly vs. the other done implicitly. I can think of instances where I would want it to be explicit vs. implicit, so it's good that the spec is aiming to provide us with both.

I love that we'll be able to add class static method extensions as well. As it puts the static method right where it makes the most sense. For example, if I wanted to create a UI Button factory method -- I don't have to invent a whole other class name to contain the factory (e.g. ButtonFactory.cs). Instead I could just add factory methods to button like this:

Button.Create(name, position, clickHandler);

Which is clear, and doesn't require the introduction of another artificial class name to add this Create(...) method.

I think we could nick-name this feature "Extensions done right".

[nitz]

Did this topic ever get any discussion in a LDM? I saw it was on the docket to get discussed back on March 2, but that note got removed and I didn't see anywhere if it was discussed or not.

I really hope for this feature to get some traction, so I figured I'd at least see if anyone had discussed it!

Cheers

[CyrusNajmabadi]

Yes. It is being actively discussed. You can see updated in our published notes here: https://github.com/dotnet/csharplang/tree/main/meetings/2022

[jnm2]

@CyrusNajmabadi I don't see it.

[yaakov-h]

It looks like it was last discussed on 1st December 2021?

[nitz]

@yaakov-h That's the last I see, too. I had hoped there was some recent movement based on the listing it temporarily had in the LDM topics, but I don't see anything since 2021 Dec 1.

[0x0737]

https://github.com/dotnet/csharplang/blob/main/meetings/working-groups/roles/roles-2022-11-10.md
Are there any problems or downsides in changing the extensions lookup order to be "role members first"? I think that it would be useful to be able to hide instance members behind extensions' ones to augment their behavior. Maybe I'm missing something important here. Also, modopt implementation looks really weird to me because of the signatures problem. That's just illogical to not be able to overload on roles since it's actually a type. Just like structs with implicit mutual conversions between them.

[333fred]

1. Absolutely. We don't think extension methods should override existing functionality, as there'd be no way to get back to the original functionality of the type. When you've converted something to a role, though, you've explicitly opted into that.
2. @jcouv, I'm not sure what the overload section in the notes is saying. The modopt version would allow overloading, it's the attribute version that would not. Was there a typo?

[0x0737]

@333fred
1: Ah, ok. For some reason I thought you can do ((OriginalType)x).Method() to use original functionality... and now I realise that this is the same type :)
2: Let's wait what the roles team says, but if it's really a typo, then I just don't see any reason to go with attributes. Especially if interfaces are planned for phase 2, maybe it will even end up in a new runtime type kind. I have a weird idea on how this would work. When applied as a wrapper to a type to fit into some interface constraint, a role could change the type of an object to an "augmented" type, preserving the original name, but different TypeHandle, different method count etc (details on possible impl later). There should also be a feature so that the reference comparison works fine on values of augmented types. So other code wouldn't break. Augmented types could keep some metadata flag that says it is an augmented type and a field with the original type. Add two bit fields to MethodTable, if it is needed. One is "IsRole" and the other is "IsAugmented". And two fields (or reuse existing) for the original type and role type MT pointers.
Reference comparison would have to ignore augmentation completely.

Not sure about System.Type equality though.
If typeof(A) and augmented typeof(A) should be equal (they probably should), then to compare augmented types' roles, you'd have to use some Type.Role property to get the role applied to an augmented type.
Things are getting more complicated for 'is' pattern.
If I got it right, to evaluate 'A' is 'B' (where A is the type of the object that is being tested and B is the type we are testing against) the runtime has to:

1. XOR their IsRole bits (result "X")
2. If type A or B has IsAugmented == 1 AND X == 1, replace it with its role type
3. If type A or B has IsAugmented == 1 AND X == 0, replace it with the original type
4. Do evaluation as usual. Roles have their parent roles too. Roles could be implemented directly by classes. If they can be, search for a role like for an interface in class' parents.

Not sure if this is a performance hit.

What wonders me very much is how the type identity is preserved when passing a role to a method that expects an interface.
It currently expects a direct reference to an object, but we can't pass the same object with different MT without duplicating its data. So... Maybe there is a new kind of boxing? An "interface box", that is basically a pointer to a heap or stack allocated struct with a pointer to an MT and the original object reference. And every interface-type variable is basically a pointer to an interface box. This would have a huge impact on codegen, so probably there should be a way to restrict a specific interface parameter to non-role impls through an attribute like [ClassImplsOnly]. Not sure about performance, but if it is good, then why worry? We have CIL, so no ABI breaks.
So, a variable of type Role would store a pointer to an interface box with the Role MT and a variable of type Interface would store a pointer to an interface box with the Interface MT and every virtual instance method call will be rewired to the MT stored in the box.
All this means roles might be real wrapper types and be able to override or hide existing functionality through the 'override' or 'new' keywords respectively (in addition to having their own hierarchy with virtuals and overrides). In this imaginary world it would be also possible to cast a type to a role and store it in a variable of a more generic type (object, for example) and the info that this is a role would be preserved via an interface box. Issues might show up with boxing/unboxing complications. For reference comparisons, we just compare the object pointers. Either directly that one from the variable or the one stored in the interface box.

I don't know much about the runtime and I can't think of all complications this implies right now, so all this is purely my imagination and not a real implementation proposal.

[333fred]

Let's wait what the roles team says, but if it's really a typo, then I just don't see any reason to go with attributes

I'm on the roles team 🙂

[FaustVX]

I've discussed about the roles in another discussion #6782 (reply in thread) with @333fred
And to me, the fact that role are just a view on an object seem not that useful
The differences between role and extension are almost non-existant.
I would like some sort of Validation (allow to cast an object to a role only if that object has certain values) ant type differentiation

// Validation
role Even for int where this % 2 is 0 { }
role Odd for int where this % 2 is not 0 { }

// Type differentiation
role TempCelcius for double
{
  public TempKelvin AsKelvin => (TempKelvin)(double)this + 273.15;
}
role TempKelvin for double
{
  public TempCelcius AsCelcius=> (TempCelcius)(double)this - 273.15;
}

var c = (TempCelcius)5;
var k = (TempKelvin)5;
c != k; // definitely not the same temperature
// and if I want to compare the underlying value
(double)c == (double)k;

[HaloFour]

And to me, the fact that role are just a view on an object seem not that useful
The differences between role and extension are almost non-existant.

The difference is that the view of the role limits the extension members. They're not intended to be an alias for the underlying type, either typesafe or not. They are intended to allow more narrow application of extensions.

That said, I personally would also like some validation, and my preference for doing so would be through custom patterns:

int value = 123;
if (value is Even even) {
    even.SomeMember(); // access members of the Even role here
}

[FaustVX]

The pattern matching is really nice.

The difference is that the view of the role limits the extension members. They're not intended to be an alias for the underlying type, either typesafe or not. They are intended to allow more narrow application of extensions.

Yes, that's why I've included the AsKelvin properties, only the TempCelcius object can use that.

[T0shik]

I don't know what was the original motivation/influence for this feature, but here are my 2 cents, when roles are being mentioned I think of "traits" and DCI.

I think there is a BIG potential to bring a new programming paradigm to C# with roles - DCI (Data Context Interaction) it's purpose is to improve code readability and shift developer focus on the use case (rather than classes). To summarise dci - objects are dumb data (D), when objects are in a context (C) they interact with each other (I). Their interaction are described by roles, any object can play any role (obv constrains in a strongly typed language)

There is prior art that existed for a long time starting in small talk (c# version over 10 years now), a programming language called Marvin (C# with DCI capability) and it's Java extended counter part trygve

Example of a use case implementation for transferring money between two accounts in Marvin. (source)

public class MoneyTransfer
{
    public MoneyTransfer(Account source,Account destination, decimal amount){
        Source = source;
        Destination = destination;
        Amount = amount;
    }

    role Source{
        void Withdraw(decimal amount){
            Source.DecreaseBalance(amount);
        }
        void Transfer(decimal amount){
            Source.Withdraw(amount);
            Destination.Deposit(amount);
        }
    }
    
    role Destination
    {
        void Deposit(decimal amount){
            Destination.IncreaseBalance(amount);
        }
    }
  
    role Amount{}
    public void Trans(){
        Source.Transfer(Amount);
    }
}

As an Engineer my 2 big points with DCI are:

• if a business person talks about a capability I can draw a circle around it in code.
• it's a reflection of the conceptual model in my head - if I imagine these objects this is how the interaction happens.

as the proposal currently stands roles and extensions are decorators on steroids, which sounds like boiler plate code reduction rather than something of substance.

• heavy reliance on classes/interfaces, if you have 3 classes (A,B,C) and 3 roles (X,Y,Z) if A and B can play X, B and C can play Y, A and C can play Z, you can imagine the interface extravaganza that you need to jump though to achieve this.
• people want to do it semi-dynamically, exhibit a: examples in this very proposal relying on a property like public DataObject this[string member] { get; set; }, exhibit b - duck typing

I think this is a step in the right direction, but it could be better focusing on interactions between objects rather than hierarchies and composition of classes, what OOP was supposed to be about.

[thinker227]

I think the primary intention/inspiration behind this feature was proper typeclass support (a la Haskell and the other big functional langs), eliminating the need for wrapper types and allowing users to implement their own interfaces on pre-existing types.

[sWozzAres]

You can implement this at the moment, but it's not ideal. The roles are classes, taking references to the containing class (context) and the object (self) in a constructor. This is made concise using c# 12 primary constructors. You can then use the 'context' and 'self' references to get at the context and object. It would be nice to eliminate these references along with the constructor.

[T0shik]

roles cannot be classes, you lose object identity and you can't have multiple inheritance so no class will be able to play multiple roles, and the primary constructor as a feature to a language is close to irrelevant for the problems being solved these days, it's more to make it hip for the new comers.

The good way to do it now is to use default interface implementations and add interface specific implementations for context properties, so there's still a lot of manual wiring. One thing that's not possible with this approach is have primitives play roles.

late binding is what you want, almost like an implicitly inferred interface base on properties which typescript already does.

[sWozzAres]

default interface implementations do not work, there is no way to initialize them with a reference to the context, which they need in order to access the contexts members (other roles + data)

[T0shik]

yeah no way to force the initialisation, but you can set it explicitly when setting things up; so they work kinda

I've played about with code generation to get something like this working:
https://github.com/T0shik/Shpec/blob/main/test/Playground/UseCases/Roles/dci_money_transfer.cs

if you slap enough syntax sugar on it, it can be done.

[HamedFathi]

How can we use this pattern for extending static classes (not members) like File or Path in C#? Could you please elaborate on it?

[ds5678]

public implicit extension FileExtension for File
{
    public static void WriteBytes(string path, ReadOnlySpan<byte> data) { }
    //etc
}

[0xced]

In case you missed it:

333fred on Dec 2, 2021

Sorry everyone, I pressed convert on the wrong github issue. I converted this discussion back to a full issue, but it didn't move the comments back. The new issue is #5497.

(That was a long journey to follow from #2505 → #1711 → #5485 → #5497) 😅

[Richiban]

Will this feature allow us to add extension members (such as operators) to certain generic instantiations of types?

For example, I'd like an Option<T> to support + only if T supports +, for example:

Option<int> a = new(), b = new();
        
Option<int> c = a + b; // Works

Option<Person> a = new(), b = new();
        
Option<Person> c = a + b; // Compiler error

[ds5678]

Yes.

extension NonGenericExtension for Option<int>
{
}
extension GenericExtension<T> for Option<T>
{
}

[Serdan]

The proposal has constraints, so

implicit extension NumberOption<T> : Option<T>
    where T : INumber<T>
{
    public static Option<T> operator +(Option<T> lhs, Option<T> rhs) => // depends on implementation
}

Could possibly even implement INumber on behalf of the underlying type with that constraint.

[markm77]

Saw this upcoming feature in the .NET blog: https://devblogs.microsoft.com/dotnet/dotnet-build-2024-announcements/#extension-types

Implicit extensions look great and will be useful for both extending types across assemblies and organising them within assemblies.

I think implicit extensions offer significant benefits over e.g. using the partial keyword to organise classes. For example, they allow a class to be split into primary and named secondary parts (extensions) which should be great for IDE navigation compared to now with partial. They also ensure different interface implementations can be compartmentalised properly in individual extensions. I presume they will also allow source generators to easily generate extensions for non-partial classes?

I am interested to watch the upcoming talk to learn about the benefits of explicit extensions. My immediate feeling is they look a bit like a back-door way of doing inheritance (with slightly different rules such as no extra state) but am interested to understand the motivation more.

[CyrusNajmabadi]

although I don't see that mentioned in the blog and I'm not sure where that part of the design currently stands.

Currently, unlikely for c# 13. We're exploring that side with specific runtime improvements to make this as good as possible.

[HaloFour]

Currently, unlikely for c# 13. We're exploring that side with specific runtime improvements to make this as good as possible.

I'd rather wait and find a solution that is as close to a zero-cost abstraction as the runtime would allow. :D

[markm77]

Just for interest, here are Swift's rules on private member access in extensions: https://docs.swift.org/swift-book/documentation/the-swift-programming-language/accesscontrol#Private-Members-in-Extensions. I realise of course in C# we have partial to organise types.

One nice thing about the C# extensions proposal compared to both Swift extensions and partial is that extensions are named! Anyone who has navigated to the definition of a class with multiple partials in a large project will know how difficult it is to know which one they are looking for. C# extensions will also (presumably) mean an interface included in the declaration of an extension cannot be implemented in another (as is the case with partial classes where interface implementations can be anywhere).

So I definitely think, as a future development, exploring allowing extension access to private members within an assembly would be worthwhile.

[HaloFour]

It sounds like Swift extensions are very similar to what is being proposed here, except that when declared within the same file as the extended type that it works more like C# partial types.

[reitowo]

This looks way much better

[zms9110750]

I recently came across a video introducing this feature.
The syntax he uses is 'implicit extension PersonExtension for Person '
Is there any room for negotiation in grammar? I don't like 'for', using ':' feels great.

And,can extensions implement interfaces? Can extensions be inherited?

[reitowo]

I think it just like where T : new(), reuse a for keyword to describe such extension.

[vladd]

There are other syntactic possibilities as well, e. g.

extension Extension(int) : IInterface;

This reads nicely as "extension of int".

[TahirAhmadov]

@vladd see this
#5497 (comment)

[markm77]

One way to allow for simpler syntax might be to allow nameless extensions (like in Swift) for simple extensions, e.g. support both

extension PersonExtension for Person : INewlyImplementedInterface {} // extension of Person with name PersonExtension
extension Person : INewlyImplementedInterface {} // extension of Person without name

Although I quite like the idea of an extension name (but am a bit unsure about it becoming some kind of a "type" rather than just a "label" for navigation etc).

Edit: just to clarify, talking about "extensions" not "roles" here....

[nathan130200]

Maybe they used for instead : in case we can extend an extension and inherith with another extension. I don't see any problem using for keyword there to describe an extension type.

[markm77]

I was just looking at the new proposal for anonymous extension declarations which looks good. However, would it be worth using the syntax extension MyType rather than extension(MyType) so that, in a future version of the extensions feature, we can add interface implementations to an existing class as shown below?

public static class MyExtensions
{
    extension MyClass : IMyNewInterface
    {
        // Add implementation of IMyNewInterface
    }
}