Extension everything public metadata types break .NET package validation

[stephentoub]

The .NET SDK includes tools that validate public API surface area remains consistent between releases.

Extension everything is generating unspeakable public surface area, and while that surface area is deterministic based on its inputs, it can change based on private details of the source and how it's passed to the compiler.

We need to look into ways the metadata can be made more stable, such that it's robust against things like ordering of how files are passed to the compiler. This, for example, is causing validation errors in dotnet/runtime's attempts to use extension everything in public surface area, because the extension type signatures are added to a ref assembly and separately with implementation to a src assembly, and because of differences outsides of the public surface area, the resulting IL differs between the two in the public APIs that end up getting exposed.

Regardless of whether this can be improved in Roslyn, we should also look into whether the package validation should be taught about unspeakable surface area. Or if we consider such unspeakable (in C#) surface area to still be something that would be a problem if it's not maintained, whether there's any way to avoid those public types in the first place.

cc: @jcouv, @jaredpar, @ericstj

[ericstj]

while that surface area is deterministic based on its inputs, it can change based on private details of the source and how it's passed to the compiler.

If it's not stable that doesn't only break api-compat, but also breaks binary compatibility. Folks need to be able to compile against one version of an assembly and run against a newer version.

[stephentoub]

while that surface area is deterministic based on its inputs, it can change based on private details of the source and how it's passed to the compiler.

If it's not stable that doesn't only break api-compat, but also breaks binary compatibility. Folks need to be able to compile against one version of an assembly and run against a newer version.

The surface area in question isn't directly used; it's unspeakable, doesn't end up being referenced in the consuming binary, and serves as metadata only (as @jcouv explained it to me).

[ericstj]

Oh that's interesting. I wonder why it's public then. Sounds like it's side-car info. Can you share with me a repro so I can have a look at it? It's possible our tools are just hitting a conservative default - we're pretty dumb when it comes to attribute comparisons and default to strictly equal.

[jcouv]

From offline discussion, the plan is to update tools that consume metadata to recognize the new metadata introduced by extensions.
Here's the tracking issue for GenAPI: dotnet/sdk#48983

We also made an adjustment to the numbering scheme for <>E__... type names, so that the numbering only counts extension blocks (but not other declarations). That should help in some cases.