Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the command-line compilers able to generate reference assemblies #2184

Closed
gafter opened this issue Apr 22, 2015 · 17 comments
Closed

Make the command-line compilers able to generate reference assemblies #2184

gafter opened this issue Apr 22, 2015 · 17 comments
Assignees
Labels
Area-Compilers Concept-Determinism The issue involves our ability to support determinism in binaries and PDBs created at build time. Feature Request Language-C# Language-VB Resolution-Fixed The bug has been fixed and/or the requested behavior has been implemented Tenet-Performance Regression in measured performance of the product from goals.
Milestone

Comments

@gafter
Copy link
Member

gafter commented Apr 22, 2015

We propose the following two new command-line options for the Roslyn compilers.

/refout:<file>

        Specifies the output file to contain a reference assembly.
        When this flag is used a reference assembly is generated.
/refonly
        Produce only a reference assembly and not the normal output.
        Compiler ignores (semantic errors in) method bodies, and tolerates
        missing method bodies. The /refout command line option is required
        when this is used, and the /out and /debug command line options
        are forbidden.

If any resources are included on the command line, those resources are included in the generated reference assembly only if /refonly appeared on the command line.

The generated reference assembly includes any assembly-internal symbols that appeared in the source. Private members are excluded from the reference assembly except when required to preserve semantics (i.e. certain situations involving structs). It is an open question whether there should be an additional option to exclude internal members.

A reference assembly produced by the compiler would always be marked with the bit that tells the CLR not to load it at runtime.

These new command-line options address our requirements as follows:

  1.   **Contract Assembly.**
    
    The requirement for a contract assembly is to have a way to hand-generate an assembly separate from the implementation of the supported API, containing precisely a user-selected set of API elements for clients of the API to compile against.
    The sources for the contract assembly would be written, either by hand or using separate tools, to exclude declarations of private, internal, or unsupported symbols, and excluding method bodies. Under this proposal the compiler would be invoked using /refout=<file> /refonly on hand-build contract sources without any resources specified on the command-line to produce this artifact.
  2.   **Reference Assembly.**
    
    The requirement for a reference assembly is to have a compiler-generated artifact that contains precisely the API of an assembly and none of the implementation details. Such a reference assembly can be used as an approximation to the "contract assembly" for building dependent assemblies. Under this proposal a reference assembly would be built using the same command line as the normal assembly, but using the /refout=<file> /refonly options instead of the /out option.
  3.   **Build Systems.**
    
    The requirement is to enable highly efficient build systems, which maintain separate reference assemblies from the implementation assemblies; when building dependent assemblies only the reference assembly is used. This benefits the build systems most if the generated reference assemblies are "deterministic" - that is, bit-for-bit identical when the supported API set has not changed. Under this proposal a reference assembly would be built at the same time as the normal assembly by the addition of the /refout=<file> option.
  4.   **IDE cross-language builds.**
    
    The requirement is for a way for the IDE to handle inter-project references even in the presence of errors. This requires internal API support and is not directly addressed by this proposal.
@gafter gafter added Language-C# Language-VB Area-Compilers Feature Request Concept-Determinism The issue involves our ability to support determinism in binaries and PDBs created at build time. labels Apr 22, 2015
@gafter gafter self-assigned this Apr 22, 2015
@gafter gafter modified the milestones: 1.1, 1.2 Apr 22, 2015
@weshaggard
Copy link
Member

Excellent I'm happy to see this making some forward progress, the proposal seems reasonable to me. We haven't put our reference assembly building stuff in our github project yet but once this is available we will have a closer look at trying to use it.

@VSadov
Copy link
Member

VSadov commented Apr 22, 2015

Nice.
I assume /refonly is also incompatible with /debug . There is no point in emitting debug information for ref assemblies.

@gafter gafter added the Tenet-Performance Regression in measured performance of the product from goals. label Apr 22, 2015
@nguerrera
Copy link
Contributor

👍 This would be a big help for cleaning up the corefx reference assembly processes.

I believe it would cover all of our scenarios.

@ericstj
Copy link
Member

ericstj commented Aug 25, 2015

👍 This is becoming more important now that reference assemblies are supported by NuGet.

@sharwell
Copy link
Member

👍 This would be awesome.

@weshaggard
Copy link
Member

@gafter Any ideas on the timeline for this feature?

@gafter
Copy link
Member Author

gafter commented Aug 26, 2015

@weshaggard Aiming for VS2015 update 2, but who knows. We accept pull requests.

@jcouv jcouv removed this from the 3.0 milestone Mar 7, 2017
@jcouv jcouv self-assigned this Mar 7, 2017
@jcouv jcouv modified the milestones: 15.2, 15.3 Mar 9, 2017
@jcouv
Copy link
Member

jcouv commented Apr 6, 2017

This mini-spec has been expanded into a more detailed design doc (https://github.com/dotnet/roslyn/pull/18501/files). I'll close this issue. The remaining refinement work items are tracked by #17612

@jcouv jcouv closed this as completed Apr 6, 2017
@jcouv jcouv added the Resolution-Fixed The bug has been fixed and/or the requested behavior has been implemented label Apr 6, 2017
rainersigwald added a commit to dotnet/msbuild that referenced this issue May 1, 2017
Provides an item, `ReferencePathWithInterfaceOnlyAssemblies`, that
consists of the reference (interface-only) versions of assemblies where
available, allowing a consuming task to be incrementally up-to-date
after implementation-only changes have been made to a reference.

When `%(ReferenceAssembly)` metadata is unavailable for a given
reference, the new item contains the implementation assembly directly.

Additionally creates a new output item for the current project's
reference assembly by default when `$(Deterministic)` is `true`. If this
is created, it is copied to the output directory using the new
`CopyRefAssembly` task, which copies only if the ref assembly is
different from the current file. If present, the ref asm is passed to
the project's consumes in metadata.

See #1986, dotnet/roslyn#2184.
rainersigwald added a commit to dotnet/msbuild that referenced this issue May 1, 2017
Provides an item, `ReferencePathWithInterfaceOnlyAssemblies`, that
consists of the reference (interface-only) versions of assemblies where
available, allowing a consuming task to be incrementally up-to-date
after implementation-only changes have been made to a reference.

When `%(ReferenceAssembly)` metadata is unavailable for a given
reference, the new item contains the implementation assembly directly.

Additionally creates a new output item for the current project's
reference assembly by default when `$(Deterministic)` is `true`. If this
is created, it is copied to the output directory using the new
`CopyRefAssembly` task, which copies only if the ref assembly is
different from the current file. If present, the ref asm is passed to
the project's consumes in metadata.

See #1986, dotnet/roslyn#2184.
rainersigwald added a commit to dotnet/msbuild that referenced this issue May 19, 2017
Provides an item, `ReferencePathWithInterfaceOnlyAssemblies`, that
consists of the reference (interface-only) versions of assemblies where
available, allowing a consuming task to be incrementally up-to-date
after implementation-only changes have been made to a reference.

When `%(ReferenceAssembly)` metadata is unavailable for a given
reference, the new item contains the implementation assembly directly.

Additionally creates a new output item for the current project's
reference assembly when `$(ProduceReferenceAssembly)` is true. If this
is created, it is copied to the output directory using the new
`CopyRefAssembly` task, which copies only if the ref assembly is
different from the current file. If present, the ref asm is passed to
the project's consumes in metadata.

Introduce $(CompileUsingReferenceAssemblies), which can avoid
passing reference assemblies to the compiler when set to false.
This is not expected to be commonly used.

IDE fast up-to-date checks need to be able to compare timestamps on
_both_ the reference assembly (to see if a recompile is needed) and the
implementation assembly (because it needs to be copied to an output
location). Provide OriginalPathmetadata on the adjusted references
that points back to the implementation assembly if a ref assembly was
found.

When a project both uses and produces reference assemblies, its primary
output assembly might be up to date while (transitive) references need
to be copied to its output directory. This case fooled the Visual Studio
Fast Up-To-Date check, because the transitive dependencies aren't listed
as project inputs (because design-time builds disable RAR's
FindDependencies for performance reasons).

To account for this, introduce a new file that is updated whenever
_CopyFilesMarkedCopyLocal does actual work, pass it along as part of the
project's output, and consider it a project input for the FUTD check.

See #1986, dotnet/roslyn#2184.
@jcouv
Copy link
Member

jcouv commented Dec 6, 2018

I have a fix to not emit resources into ref assemblies produced with /refout (#31244). Neal, Jared and I discussed what to do about ref assemblies produced with /refonly. For now, we will do the same as /refout, namely we will not include resources.
If this causes problems we didn't anticipate, we think there are some options to solve them when they arise (although it may require some additional flags passed into the compiler, perhaps /refonly:<option> or something).

@nguerrera
Copy link
Contributor

I don't think this feature is being used by corefx (yet?) but IIRC, there are designer or toolbox features that rely on certain resources in reference assemblies. @weshaggard might recall for sure.

It seems to me that if one calls /refonly and passes resources, they should be retained. The caller could simply not pass them if they're not wanted. This is unlike /refout where not passing them would compromise the main assembly.

Also are we not concerned about the breaking change here in both refout and refonly case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Compilers Concept-Determinism The issue involves our ability to support determinism in binaries and PDBs created at build time. Feature Request Language-C# Language-VB Resolution-Fixed The bug has been fixed and/or the requested behavior has been implemented Tenet-Performance Regression in measured performance of the product from goals.
Projects
None yet
Development

No branches or pull requests

10 participants