Champion "Replace/original and code generation extensions" #107
Comments
gafter
changed the title
gafter
changed the title
|
@mattwar Are you guys still actively working on this feature or has it been tabled for now due to the implementation complexity? |
|
No news on this? |
|
I just read @MadsTorgersen 's design notes that said this feature is blocked from somewhere else not on the language end. Could you guys please elaborate? Is the VS tooling the roadblock here now? In my view, this feature is far and away the most useful one out of the entire backlog for C#. I know the brass likes to dedicate resources to issues that community puts a lot of votes on, so I've made a UserVoice issue as well. Please vote for this feature here and on UserVoice! |
|
Yes. VS tooling is the roadblock. This work will require substantial resources to get through the tooling side in any sort of sensible and efficient manner. It requires coordination between many different teams as well. As such, while still super great and valuable, it's unclear what the right timeline/delivery avenue would be for it. |
|
Sorry if this is a dumb question but what exactly does tooling mean here? Sounds like there's nothing the community can do, or is there? |
Everything about the experience in a product like VS. That includes, but is not limited to:
etc. etc. |
|
@CyrusNajmabadi Just for clarification it won't be strictly specific to Visual Studio, right? |
|
I'm not sure i understand your question @eyalsk . Can you clarify? |
|
@CyrusNajmabadi I'm asking whether these features you mentioned like IntelliSense, Navigation, Debugging are going to be available in say Visual Studio Code or any other editor? as opposed to being available in Visual Studio only. |
|
The owners of those respective products would have to make a decision on that. :) |
|
@CyrusNajmabadi Got'cha, thanks. 😉 |
|
Why couldn't you guys just add the feature to the compiler and the IDE support will come whenever it comes? This is certainly the strategy TypeScript takes, where language/compiler features sometimes don't get VS support for months/years after they're introduced. |
Because the IDE experience won't meet the user experience bar we've set for language changes. Plus, it will just be a terrible experience. Think about it this way, if there's no IDE support, and you have a code generator that generates symbols that you try to reference, then you'll get tons of error squiggles that never go away as you're trying to use the IDE. |
|
What doesn't make sense is the arbitrarily different bar for different MS products:
Basically, my point is C# language evolution shouldn't be constantly hamstrung by Microsoft's internal politics. The language team should be able to add features and let IDE support catch up (as it should be). That's how nearly every other open source language works. Put the feature behind an opt-in flag until you're comfortable the VS support is up to snuff. In the meantime, JetBrains will probably have R# and Rider fully supporting within 3 months. If the feature is forced to wait for the stars to align inside Microsoft, it'll likely be a minimum 5-10 years before it actually ships. |
It's not internal politics. We are the C# team. We decide what we feel is best for the language. And that includes insuring that it has a great development experience out of the box. And there's a good reason for that. We don't want to ship the language changes too early, only to find out later that we royally screwed things up once we try to then get proper IDE support for it. SourceGenerators are a massively complex undertaking. We could try to get something out without considering and attempting to prove it would work in the IDE. But then it's highly likely we would get it very wrong. Even our preliminary efforts here showed us immediately how our early source-generator designs simply would not be suitable to providing a good IDE experience. |
As you say this you guys have just finished shipping a feature which is functionally broken out-of-the-box - tuples. It requires you to know that you need to go to NuGet and search for a specific package in order to use it. The VS feature which suggest packages to download is turned off by default. Dustin was demoing replace/original last year, so clearly some form of it is possible to ship without the sky falling. It's also ironic that you're the one making this argument, Cyrus, when not long ago you were on the TypeScript team helping to ship new features every couple months. |
|
@MgSam I agree with you on some of your points but I think that they probably don't want to make the same mistake twice and the experience I had with T4 from what I can remember was pretty awful to the point I removed the extension I wrote from the VS gallery. :) |
Didn't they fix this for RTM? |
Yes. That was a demo. It was woefully incomplete. It worked in extraordinarily small scenarios. It had serious issue that we did not (and still do not) have solutions for real projects. We do not ship prototypes just because they're cool and they demo'ed well. A lot of real effort went into exploring this feature. Even just the language feature alone had a whole host of issues that we did not know how to resolve. And that was before the very first problem of: how will this even hook up into the IDE compilation flow model? |
No. But we're working on it. The core problem is that there have been extremely hard perf goals to meet, such that even loading a single extra dll is not allowed. That's problematic in the context of any sort of extensibility story. We made a hard decision to ship in that state, even though we would have preferred it be better.** And that goes to my point. That was something we did not want to do, and it was only because the scope of the problem was so limited. i.e. you would get an error about tuples, but at least you could solve it. That doesn't even begin to compare to the issues you'd get with source generators. Again, imagine it was like tuples, and you got lots of errors in your IDE while trying to work. Except now there wouldn't be anything you could do about that. Also consider that just typing things like "replace" and whatnot would simply throw of the IDE parser completely, leading to code that was nearly impossible to edit. Seriously, if you want to get a feel for what it would be like, go back to something like 2013 and try to type modern C# in it and see how quickly the IDE experience goes off the rails. Just basic editing will fall to the floor because we won't understand teh code you're writing on a syntactic and semantic level. And, currently, our IDE is based around the idea that we will understand at least that much. -- ** We've won back enough performance across the board though that we feel lke we can get an exception here and that this feature can be enabled soon. If/when that happens, it will mean there was a short window of time when we had a not-great experience. That's not at all comparable to the types of timeframes we're talking about for something like SourceGenerators. And again, it's not like we can "just ship source generators now and just make the IDE work later". We have to understand the needs of hte IDE so that we can properly design source generators in the first place. We know this first hand from other work we've done in the past, including work in analyzers. The original analyzer work did not think closely enough about the compiler/IDE interaction, and as such had absolutely unacceptable performance characteristics. By designing things together, we were able to come up with something that met our feature goals while also having acceptable performance. |
There's no irony here. Nor is this an accurate representation of what actually happens in TypeScript. In TypeScript features are shipped and the TS team themselves does update the LS side of things at the same time as they're doing the compiler work**. It is the case that they can release a compiler that people can use before an official VS build is made available, but that doesn't mean that they're shipping language features that they then might not have any clue how to the IDE work for. That's what's different here. IT's not like we know what source-generators are at a language level, could ship that, and could then just "do the IDE work". We do not know what source-generators are precisely because the design of the language feature itself is so informed by the tooling that will consume it. -- ** I know. I was involved with it for several years, and i still work closely with them as they're an adopter of Roslyn and an early adopter of lots of the improvements we make. I wrote a ton of both the LS features for TS as well as the majority of the LS implementation in VS. I also wrote the extensibility points in Roslyn to allow TS to become a Roslyn language. |
Totally fair. So there's a lot more work to be done. I just don't buy the argument that it's impossible to do the work unless 6 other MS teams first jump on board. Good software/API design doesn't require explicit buy in and support from every user before you proceed. If it did, no software would ever get shipped. I use PostSharp- which accomplishes many of the same things as this feature. In the first versions had nearly zero IDE support and the support still sucks pretty bad. That doesn't mean it's still not super useful. Like I said, do your best effort, put it behind a feature flag and let your partners implement IDE support themselves. I still haven't heard a reason why making an optional feature that's turned off by default is problematic. Again, you guys do this constantly in TypeScript.
Except if I'm using R#, in which case it'll probably work perfectly. Again, my point that language design shouldn't be hampered by MS politics.
This is completely off on a tangent, but non-nullable types were released in TypeScript 2.0 in September. The When TypeScript first came out, you had Mads Kristensen adding capabilities on his own into VS (like compile on save) because no one else was doing it. The TypeScript-VS team coordination has always been dysfunctional. Yet TypeScript has been a huge success nonetheless. And you guys should be applauded for pushing the language forward without waiting for VS. I don't mean to get into an extended back-and-forth over this, but it's super-frustrating to see useful features like this continually delayed indefinitely so that the team can instead turn to syntax sugar features that have little real-world impact on code quality or productivity (like the entire C# 7.0 release). And the reason is always the same- "this feature requires *** team inside Microsoft to cooperate so let's delay it indefinitely" Seriously, you guys do a better job cooperating with the Angular team on TypeScript then you do cooperating with other teams inside of Microsoft. So, in my opinion, the C# team needs to stop gating progress on other MS teams and move ahead doing your best to design features and APIs. If it ends up needing changes for native VS support to work later on so be it- that's agile software development. |
I have given the reason: Because we literally don't know how the feature should work. We don't know what the right design is for it. How do we even create it if we can't figure out these questions :)
Good thing i didn't make that argument ;-) The argument is that the C# team itself (i.e. us) are not willing to ship the feature when we can't deliver even a basic experience that is satisfactory. Nor are we going to just ship the language/compiler feature if we don't even have an idea for a design that we think could be sufficient.
And when the partners (i.e. the C# team) says to the C# team: "uh... there's no way we can get any sort of acceptable experience here with the design you shipped to customers..." then we're in a big heap of trouble. :)
Again, TS does not create language/compiler features that they don't know how they'll integrate into their LS. I'm not sure how much more plainly i can state that.
Please stop saying "*** team" :). The reason here is that the C# team itself does not know what the right design is for thsi feature. We don't know what it is and even as we experimented and prototyped many options here we ran into enormous issues (even at the language/compiler level) that we simply had no solution for. On top of that, the C# team itself does not want to ever ship language/compiler features that it doesn't know if it will be able to deliver a suitable developer experience around. That is a core goal of our language, and we believe it is very important. Right now we could come up with literally zero designs that we could feel confident that we'd be able to provide even a basic experience around, let alone a good experience. It would be enormously irresponsible of us to ship something like that at the language level, especially with the expectations people have had from us for the last decade+.
It's not other teams. We are the C# team. The C# team has not been able to come up with a satisfactory design for this feature, and also feels that there are enormous complexities and problems that would be extremely difficult to deal with for the developer experience. -- I feel like you're thinking the situation is one of: "Oh man... we have this amazing feature. And we totally know how to do it. But those pesky devs over there are preventing it from going out! :( ". That's not what is happening here. We simply do not have a design that we feel actually delivers the right feature to users (let alone an implementation to go along with that design). And we are never going to ship something that we can't even come up with a design for. --
Sorry, the rules of C# and the promises we've made over the years still apply. Backcompat is still a thing. We're not going to just throw this feature out there, then realize it is full of problems, and go "sorry, gotta break everyone, that's agile software development". We are the language and platform. We get that this would be hugely useful. But that's all the more reason to try to get something good that can last before shipping things out to people. |
|
https://github.com/dotnet/roslyn/projects/54 |
|
Wouldn't trying to prototype this indicate a serious commitment? |
|
Actually not. You may build prototype after prototype, but without serious commitment, never able to push it through RTM phase. |
|
We're not going to be able to make this happen without prototyping it. It's a challenging and complex space. I cannot be designed in absence as this is really about a tooling feature (like analyzers), and much less of a language feature. MS is currently paying people to do this and allocating the time to make this happen, ahead of a long list of other with that could be fine instead. If you don't see that as commitment, then I don't know what to tell you. |
|
This is the GitHub project used to track our upcoming source generator work. The aspiration is to ship this as a part of the C# 9.0 release.
As @CyrusNajmabadi noted this is a complex area and one where we feel strongly that we need direct customer feedback to ensure we deliver the correct solution. Delivering a regular stream of prototypes through the C# 9.0 release where we can iterate with those customers is the best way to get that feedback.
+100. |
|
Maybe you can join forces with https://github.com/AArnott/CodeGeneration.Roslyn |
|
@AArnott Is certainly welcome to weigh in |
|
I'm stoked about the C# 9 built-in feature. I'd love to try a prototype. |
|
About the source generator proposal, Can it brings https://doc.rust-lang.org/rust-by-example/macros.html Example: Implementing Ruby’s HashMap syntax in Rust
(Procedural) Macros in Rust is so powerful, We can generate source codes with custom DSLs, syntax and much more. |
|
Custom DSLs are very intentionally not in scope for this. |
|
@HamedFathi rust macros are known for poor IDE integration, especially procedural ones. I don't think it worth taking them as they are even if LDM was interested in it, which they aren't. |
|
I really like how macros work in Rust. So things like Serde (mapping to struct directly ) are possible without using reflection. This would change completely the way you can create your serializer and it would improve performance a lot. You could generate your mapping/serialization methods. Now the workarounds are - reflection caching, expression trees(but that requires compiling during runtime) or using some external tool for generation. Or using some kind of AOP like Fody , PostSharp... but it adds a dependency. @Pzixel there is always room for improvement and plugins are constantly evolving. |
|
@bugproof rust macros are bad for two reasons
I'd like to have macro system in C#, but LDM don't think so. This is why I probably will switch to scala when next major version comes. Partial evaluation, HKT, functional paradigm - all the nice stuff. MS has different priorities for C#, sadly. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
It's great that this is being worked on, but I wonder what is C#-specific about this? Isn't there a way of making this language-independent, in the sense that:
|
Isn't this just a description of IL... ? If you want a non-C#-specific, language-independent, way to pass information into a compilation... isn't that just a dll?
Isn't that IL? Basically, why would we need a special feature for this since it's already the fully supported scenario? |
@charlesroddie to be clear, this proposal is not being worked on. This proposal is |
|
I think #45648 is a good middle-way proposal between this proposal (original/replace) and the Source Generators proposal being implemented for C# 9.(https://github.com/dotnet/roslyn/issues/45648) |
|
I like source generators, but I'm not used to insert code over strings, that's ugly. Maybe some limited form of dlang templates + String to macro expansion: static foreach(String type in ["class","struct"])
static foreach(String fieldName in ["a","b"])
static foreach(String fieldType in ["Int,","Float"])
$type {
public $FieldType $fieldName;
}Generating 2^3 types. In contrast to Source generators, we only put those code fragments in strings which are changing and expand them by static foreach(BlockKwString type in ["class","struct","unknown"])
static foreach(IdentifierString fieldName in ["a","b"])
static foreach(TypeString fieldType in ["Int,","Float"])
$type {
public $FieldType $fieldName;
}Also, mixins for code insertion would also be beneficial. |
|
We are maintaining a private fork of Roslyn that allow for code rewriting. It is used in our product Metalama. We are considering making this fork open source, but given the additional work it requires, we would like to probe the level of interest first. Please vote and contribute to this GitHub discussion and detail the use cases you have in mind. The principal interface is the ISourceTransformer interface and it allows for a complete replacement of the |
and others
See also
The text was updated successfully, but these errors were encountered: