Language Feature: Extension Everything

[gafter]

There are existing proposals/requests #3357, #4945, #5165, #5624, #6136, but this is a slightly different take. One difference is that qualification with this would be required inside, at least for access to extension members.

Extension Everything for C#

This proposal gives a way to define new kinds of extension members. Today it is possible to define methods that act as if they are instance methods of the extended type. This proposal expands that capability, supporting both static and instance members, and supporting (or at least discussing) all of the kinds of members you might want to declare.

Here is an example that summarizes the new proposed syntax forms:

// "extension" is a new (contextual keyword) modifier, permitted before the keyword "class".
// It can be combined with "partial". It produces a static class that one would not normally use
// directly, however there will be syntax forms for directly using all of the named members
// (but not the operators) as members of this class.
public extension class ListListExtensions<T>
  : List<List<T>> // the "extends" clause names the extended type. May be an interface
  // , Interface  // we do not permit adding interfaces in the base clause
{
   // static variables are no problem, and hosted in the container ListListExtensions<T>
   private static int _flattenCount = 0;

   // static methods are simply hosted in the container, ListListExtensions<T> in this case
   public static int GetFlattenCount()
   {
     return _flattenCount;
   }

   // static properties are similarly hosted in the container.
   // They translate into a static "indexed" property, indexed with a "this" (extension) parameter.
   public static int FlattenCount
   {
       get
       {
         return _flattenCount; // extension static variable access
       }
   }

   // instance methods are compiled using the same pattern as existing instance extensions.
   // The additional hidden parameter is named @this from the SemanticModel's point of view.
   public List<T> Flatten()
   {
     // Within instance members, "this" is of type List<List<T>>
     _flattenCount++;
     ...
   }

   // Extension indexers permitted, both static and instance.
   public List<List<T>> this[int[] indices] => ...;

   // We do not support instance extension fields, but we could support them in the future
   // if the type being extended is a reference type, using ConditionalWeakTable.
   // public int InstanceMember;

   // We do not support constructors, but we could support them in the future
   // by creating an ordinary method with a special name. Some design work would
   // be required to ensure the following syntax is possible, or to design an alternative.
   // public List<List<T>>(int defaultSize) : this(...) { ... }

   // Operators are permitted. Operators are required to have a parameter (or return type,
   // for implicit conversions) of the extended type.
   // All of the lookup rules for operators will need to be amended to look in extension types
   // that are in scope.
   public static implicit operator List<T>(List<List<T>> self)
   {
     return self.Flatten();
   }

   public static implicit List<List<T>> operator +(List<List<T>> left, List<List<T>> right) => left.Concat(right);
}

Limitations:

• Instance fields not permitted (at first)
• Constructors not permitted (at first)
• Events not permitted (at first)
• The base clause cannot add interfaces
• No members may be virtual, override, abstract, sealed, etc.
• Private helper members (e.g. _flattenCount above) work the same as extension members, even though not accessible outside.
• Operators will require some language constraints on built-in types, interfaces, etc. For example, the extended type must be either the source or target of a conversion operator.
• No instance auto-properties (until we support instance fields).

Substantial LDM design work will be required

• What is the syntax?
• What is allowed and not allowed? Extending int[]?
• We will need to describe how we go from a member access e.M to the set of extensions that might be designated by the access.
• Every construct that can use one of these will have to have its spec amended
  • Member access, method overload resolution
  • Indexing
  • Every kind of operator (e.g. what are the constraints at the declaration, where do we look on the use)
  • Later, constructors
• For conversions, what limitations do we need to not run into trouble?

[alrz]

Why we still require to name the extension class while it can be self explanatory like:

public extension int { }
// instead of
public extension class IntegerExtensions : int { } 

Using class-base to specify the target type can get weird in case of generic, static or array types:

public extension class IntegerArrayExtensions : int[] { }
// for defining static extension methods ..
public extension class ConsoleExtensions : Console { } 
// would this be allowed?
public extension class GenericExtensions<T> : T /* where T : ... */ { }

Alternatively,

public extension int[] { }
public extension Console { }
public extension<T> T /* where T : ... */  { }
public extension<T> List<List<T>> { }

This would reserve class-base for when we have virtual extension methods (#258) to add interface implementations to existing types (#8127).

[svick]

I apologize if I misunderstood the proposal, but I don't think the way it treats this makes much sense.

---

return this._flattenCount;

So, static extension members are accessed from static extension members using this., which looks like instance member access? I think that's very confusing.

---

The additional hidden parameter is named @this from the SemanticModel's point of view.

So, to keep a static cache of the last instance some member was called on, I would have to write the following?

this._cache = @this;

Again, quite confusing.

[svick]

@alrz I think the name can be useful. You can use it to explicitly call the extension method as a normal method (e.g. Enumerable.ToList(query)). And it also allows you to select only some extension types from a namespace (e.g. using static System.Linq.Enumerable;).

[gafter]

@svick

I apologize if I misunderstood the proposal, but I don't think the way it treats this makes much sense.

It was messed up. Static members are accessed by qualifying with the extended type. I've fixed up the OP.

So, to keep a static cache of the last instance some member was called on, I would have to write the following?

this._cache = @this;

Again, quite confusing.

The @ is to show that it is an ordinary identifier when viewed through the symbol table (compiler API), and that you can access it that way if you want. You can still refer to it using the this keyword.

[svick]

@gafter Ok, now it looks good to me.

[alrz]

@svick Currently you can not use using static for extension methods. Also they can't be defined in generic classes. As you can see in the original post ListListExtensions<T> is a generic class (I'm not saying that this can't be done in the new syntax, though).

You can use it to explicitly call the extension method as a normal method

If that's your concern you'd better use the existing extension syntax (that'd be the benefit for the old syntax to be not totally deprecated). But I don't expect to be able to use extension methods as if they are static methods when I explicitly defined them in an extension declaration as non-static members!

And it also allows you to select only some extension types from a namespace

I didn't suggest that they be in the scope out of nowhere, just like what we have with extension methods, the containing namespace must be imported using using.

I think naming classes like extension class WhateverExtensions : Whatever is just too repetitive.

[svick]

@alrz

Currently you can not use using static for extension methods.

This compiles fine for me:

using static System.Linq.Enumerable;

class Program
{
    static void Main()
    {
        new int[0].ToList();
    }
}

[alrz]

@svick Oh right. But this doesn't ToList(new int[0]); assuming using static System.Linq.Enumerable;.

[quinisext]

While I do agree that naming an extension class is useful, I think that (formally) deriving this class from the extended class makes little sense overall.

public class ClassToExtend
{
    public ClassToExtend (int arg) { ... }
}

public extension class ExtensionClass : ClassToExtend
{
    // For when constructors are supported.
    public ExtensionClass (int arg1, int arg2) { ... }
}

...

// Unlikely the desired syntax, but should make sense if
// ExtensionClass is a descendant of ClassToExtend.
ClassToExtend a = new ExtensionClass(1, 2);

// Likely the desired syntax.
ClassToExtend b = new ClassToExtend(1, 2);

// Totally not what we need.
ExtensionClass c = ...

In short, extension class is not a descendant of the extended class, so it shouldn't look like one.
The syntax could rather go like this:

public extension class ExtensionClass<T> for ClassToExtend<T> { ... }

[alrz]

@quinisext

I do agree that naming an extension class is useful,

Can you give some examples, how is that useful? I also note that if your extension class is generic, which it very may well be, you can't use using static as @svick mentioned. This is not a problem right now because extension methods are generic, not the enclosing class, so using static is fine. Even if you could "use extension methods as if they are static methods even though you explicitly defined them as non-static members" which makes little sense, type inference won't help you and you will need to explicitly specify type arguments. So for complete compatibility with existing extension classes, generated class must not have a type parameter list and perhaps with an optional name, because it's not always useful.

[quinisext]

@alrz
Even if the name is optional, I believe the class keyword should be obligatory.

public extension class<T> for T[] { ... }

(I'm not insisting on for, of course, just not :).
It makes the code clearer to read (especially when extension is not highlighted by the editor) and more uniform, and most probably easier to parse. Without it, the extension would probably need to be promoted to a full-fledged keyword status. Naming makes the code more uniform too, though.
Actually i'd rather have as little new syntax/keywords as possible:

// Only a for-part is something new; and no new keywords, contextual or not.
public class ArrayExtension<T> for T[] { ... }

// Easier to get as a special case of the previous. Still no good looking.
public class for ExtendedClass { ... }
public class<T> for T[] { ... }

Names would allow to using static an extension type without using the rest of its namespace.
Names are useful to me personally, due to the way I use reflection (which I do extensively) (can be achieved through attributes though).
Names would allow to create two distinct extension classes in one namespace, which can be using static separately.
They're also future-proof, in case some day language gets parametrized usings.

using static SomeNamespace.ExtensionClass<System.String>;

Admittedly these all are pretty esoteric.

Still the syntax you proposed does seem appealing to me, I'm just afraid it would complicate parsing, and the word "extension" is too good a word to make it a keyword. Maybe something like

public extending<T> T[] { ... }

[omariom]

Interesting 👍

Which existing types in CoreFx or third parties could benefit from the feature?

[MgSam]

• Spec seems interesting and reminiscent of some previous community suggestions, however, I personally think you don't get much value-add over existing extension methods unless you add the ability to have data associated with an instance (extension instance properties).
• Agree with previous comments that re-using the extends operator, :, is a bad idea. It already has too many meanings, don't add another one.
• @gafter You've previously told me in another one of these threads that using ConditionalWeakTable for associating data with an instance slows down the entire program, even where it's not being used. Is this not still the case? Why would extension instance properties be limited to value types?

[paulomorgado]

@MgSam;

• Spec seems interesting and reminiscent of some previous community suggestions, however, I personally think you don't get much value-add over existing extension methods unless you add the ability to have data associated with an instance (extension instance properties).

Isn't this supposed to be static extensions? Why is "add the ability to have data associated with an instance (extension instance properties)" important?

• Agree with previous comments that re-using the extends operator, : , is a bad idea. It already has too many meanings, don't add another one.

That's when Java and Visual Basic start to look good for having extends and implements keywords. But for C#, I think it's too late to go that way. Another punctuation?

[MgSam]

@paulomorgado My point was this seems like a lot of design work for very little benefit over current state if adding state to the instance is not on the table. Having extension methods on static classes are the only truly new use cases in this spec, but even that is less useful than it once was with C# 6.0's using static feature. (You can just dump all the static classes you want into scope in your file)

Agree the ship has sailed long ago on extends and implements, but that doesn't mean we should perpetuate the mistake by reusing : yet again. Make it an English-language keyword word rather than punctuation.

[quinisext]

@MgSam
I use extension methods extensively (sorry), so I miss such feature for quite some time (it is in Delphi/Free Pascal for a long time now).
It allows adding properties using standard syntax, and that is enough benefit for me on its own.

[MgSam]

@quinisext I use extension methods extensively too. The only difference this proposal makes for properties is that instead of having extension methods SetFoo and GetFoo, you now can make those a property. But they still can only address static data. That doesn't seem like it's worth the design cost to me.

On the other hand, if there is a realistic way to associate instance data, then it becomes a big win. WPF has the whole concept of "attached properties", which are absolutely nasty to declare, yet this proposal still doesn't fill the huge niche that those attempted to address. The C# design team tried to tackle extension everything once before and rejected it because the design was incompatible with this use case.

[gafter]

I consider "extension static members" including "extension operators" the most important benefits of this.

[quinisext]

@MgSam
I'm actually more interested in readonly properties. But having properties you can always implement
your own mechanism of attaching them:

public class ExtensionClass for System.Windows.Controls.Button  // Whatever would be the syntax.
{

    // You would still need to deal with abandoned values, but there is probably no good way
    // to solve this problem in general case.
    private static Dictionary<MyClass, int> values;

    public MyIntProperty 
    {
        get { return value[this]; }
        set
        {
            values[this] = value;
        }
    }

    // Some neat tricks.
    public double CanvasLeft
    {
        get { return Canvas.GetLeft(this); }
        set { Canvas.SetLeft(this, value); }
    }

    public RoutedEventHandler ClickHandler
    {
        get { return Click;   }
        set { Click += value; }
    }

    // Extension constructor.
    public ExtensionClass (out Button variable)
    {
        var result = new Button();
        variable = result;
        return result;
    }
}

// Now also this coud work.
var button = (Button)null;
var Panel = new System.Windows.Controls.Canvas
{
    Children =
    {
        new Button (ref button)
        {
            CanvasLeft    = 20,
            MyIntProperty = 42,
            ClickHandler  = (s, e) => { DoSomething(); }
        }
    }
};
button.Click += MyOtherHandler;

You can't do these things with Set/GetSomething().

Extension constructors may work well with immutable data. And extension operators would allow you to connect mathematical types that was initially unaware of each other.

[HaloFour]

If the concept of "extended" state is to be on the table I would like to see something come out of the CLR that would enable such functionality without incurring the penalties associated with ConditionalWeakTable. Otherwise my opinion is that C# shouldn't encourage such use and that if people want the "extended" state then they can simply follow those patterns manually.

Beyond that, extension static methods, properties and operators do seem like they'd be worth it. Extension constructors are weird but I can see their value. Maybe that could lead to asynchronous constructors #6788.

[jveselka]

@quinisext That's a memory leak. Correct implementation would need something like WeakReference Dictionary. I'd rather see language support for extension fields than implementing this (I would probably get it wrong).
EDIT: I have just found out that there is something like WeakReference Dictionary in .NET already - System.Runtime.CompilerServices.ConditionalWeakTable class. It's only downside is restricting values to reference types.

[quinisext]

@zippec

That's a memory leak.

That's exactly what I meant by

You would still need to deal with abandoned values.

As I said, I don't see a good general solution, that can be used as an implementation for extension fields/auto-properties. But that was not the point of the example. The point was to show that extension classes are worth implementing even without that particular feature.
As to the feature, it would be nice to have it somehow, but is there a really good way to make it? If you do it manually, you at least see what's going on and understand the costs and limitations.

[VSadov]

Another idea for the syntax - just use the type that we are extending, possibly with additional constraints.

public extension class List<T> {..}
public extension class List<T> where T: List<int> {..}
public extension class List<T> where T: class {..}
public extension class List<T> where T: int {..}

[HaloFour]

@VSadov

What would the name be for the generated static class if the developer doesn't explicitly provide one? Whatever it is would have to be predictable in order for a recompilation to not break existing consumers, and it would have to be resilient to collisions in case you want to extend two different classes named List, or with a different arity of generic parameter types.

[quinisext]

@VSadov
Would it scale well?

public extension class int[] { ... } // Formally works but unnatural for c#.

Also, as HaloFour said, what is the "predictable" name for int[] extension class? It is just unobvious in general case.

[VSadov]

The challenge here is to have a syntax very similar to a class definition, but yet we want to sneak in a type reference to a potentially more constrained instantiation of a type - like List< int >.

My suggestion may not even be better than what @gafter proposed. Just another idea how we can abuse preexisting syntax.

Having no name is ok. Not everything in the source ends up with formal names.
As implementation, compiler can generate something unspeakable and specify applicability of the extension container via attributes.

[AustinBryan]

@quinisext

In short, extension class is not a descendant of the extended class, so it shouldn't look like one

Why though? : as been used to show things other than inheritance in C# before.

Edit:
Firstly, there's MyClass : IMyInterface. MyClass isn't inheriting IMyInterface, it's implementing it. In Java, they use extends for classes and implements for interfaces. That's Another reason why I don't want an extends , or extensionof keyword, is that it might seem like it's inheritance for Java people.

Having public extension class : MyClass shows that this is a special type of class whereas public class extensionof MyClass might be confused with inheritence and just in general feels more like Java and less like C#.

where T : class, where T : new(), where T : struct. All of those are not things that are even remotely possible to inherit from, not in the way sealed classes can't be inherited, either, but we know what it means, and it's cleaner.

And I actually like the naming of the classes better than just using the type, it feels more like what's really going on: I'm making a new class that the compiler will treat as part of the original.

[AustinBryan]

Another idea would be to add a modifier for classes that is basically the extension-everything counterpart to sealed, in that it prevents people from extending your class.

Something like public unextendable class Person would make Person not able to be extended.

[iam3yal]

@AustinBryan

Another idea would be to add a modifier for classes that is basically the extension-everything counterpart to sealed, in that it prevents people from extending your class.

Something like public unextendable class Person would make Person not able to be extended.

Why would you want to do that? this sounds like a bad idea to me because you shouldn't care whether people extending your types, it's their choice, not yours as they don't change any behaviour, they add new on top of existing one and atm the experience is unpleasant, I mean in order to "extend" types today, depends on the situation, you have to copy/paste the code, use wrappers or static classes and then at the consumer side you have to relate to some other type as opposed to the original type that was "extended" so why would you add a feature that tries to make the experience better and address this problem only to reintroduce the exact same problem?

It makes sense to seal a class or individual methods because people don't want others to modify the behaviour but it doesn't make sense to prevent others from extending it unless the person is a "control freak". 😄

[AustinBryan]

Will we still be required to use the this keyword to use extension methods, properties, etc.?

[gafter]

Almost certainly yes

[AustinBryan]

@gafter Why is that?

[paulomorgado]

What's the alternative, @AustinBryan?

[ashmind]

@paulomorgado #998 (make this optional for extensions)

[Austin-bryan]

@paulomorgado Make it so we don't have to use this... (obviously). There's literally no point. It's not like they need it, it was just a design choice, and a pretty annoying one. It just takes up more characters and I find it annoying. And the argument "it's not necessary to have" is very, very weak. So many C# features are not necessary, like expression bodied anything, properties, local functions, var, target-typed new, etc., all of these things can be done without them, it just makes life easier to have. Some features simply improve quality of life and this is one of them.

Just because you use this. all the time so you don't find it annoying, doesn't mean the rest of us do, as I only use this for when I have to.

[paulomorgado]

@willard720, I find it informative. Can you find a situation where qualifying with this gets you in any trouble?

[Sergio0694]

I'm not sure I understand the current status of this issue.
It's marked as 3 - Working and has an assignee (@MadsTorgersen), but it is also closed.
Is this still planned, either for C# 8.0 or for a future minor release? Or, is it already working in a C# 8.0 preview/internal build? 🤔
@gafter what's the official word on this?

Thanks!

[orthoxerox]

@Sergio0694 language feature design is now tracked in dotnet/csharplang, this is an obsolete issue.

[Sergio0694]

@orthoxerox whoops, my bad, I was sure I was already in the csharplang repo 😄
Thanks for pointing this out!
I see it's been added to the X.0 milestone, hopefully it'll be out soon!

[vbodurov]

@MadsTorgersen @Sergio0694 can you please point where in https://github.com/dotnet/csharplang/milestones is this #11159 feature located. I cannot find it.

[vbodurov]

OK I found it dotnet/csharplang#192

Please do this as soon as possible!

[nutix56]

Why we still require to name the extension class while it can be self explanatory like:

public extension int { }
// instead of
public extension class IntegerExtensions : int { } 

Using class-base to specify the target type can get weird in case of generic, static or array types:

public extension class IntegerArrayExtensions : int[] { }
// for defining static extension methods ..
public extension class ConsoleExtensions : Console { } 
// would this be allowed?
public extension class GenericExtensions<T> : T /* where T : ... */ { }

Alternatively,

public extension int[] { }
public extension Console { }
public extension<T> T /* where T : ... */  { }
public extension<T> List<List<T>> { }

This would reserve class-base for when we have virtual extension methods (#258) to add interface implementations to existing types (#8127).

we can no longer call our extension member if it became a official implement

[Elviszip]

all the other projects for "extension everything" have been closed.

Is it possible to revive the project?

Thanks!

[HaloFour]

@Elviszip

all the other projects for "extension everything" have been closed.

Most issues have been moved to discussions until they have more of a concrete proposal. I think this is the relevant discussion for this topic: dotnet/csharplang#5498

Is it possible to revive the project?

There is currently a working group looking into roles and extensions together as a project, so it is alive and well:

https://github.com/dotnet/csharplang/tree/main/meetings/working-groups/roles

With an evolving proposal here: https://github.com/jcouv/csharplang/blob/roles-spec/proposals/roles.md