Skip to content

Latest commit

 

History

History
257 lines (164 loc) · 14.5 KB

README.md

File metadata and controls

257 lines (164 loc) · 14.5 KB
Raw

Source Control and Source Link Packages

MSBuild has no built-in knowledge of any source control system or provider. In order to retrieve information from the source control the project needs to include a package reference to the appropriate source control package. Microsoft provides source control packages for Git and TFVC managers:

  • Microsoft.Build.Tasks.Git (release)
  • Microsoft.Build.Tasks.Tfvc (experimental)

These packages implement a protocol defined by MSBuild that allows extraction of necessary information from the source control system during build. This protocol can be implemented by third party packages in order to support other source control systems.

Adding one of these packages to the project enables MSBuild to automatically detect the current source revision id (Git commit hash, TFS shelveset number), the repository URL, source files that are not tracked by the source control and other useful information.

Having this information available enables the following features:

  1. Including source revision id in AssemblyInformationalVersionAttribute and in NuSpec of the package produced by the project.
  2. Automatic detection and publishing of the repository URL.
  3. Embedding sources to the PDB that are not tracked by source control.
  4. Generating Source Link that enables debuggers to find sources when stepping through the DLL/EXE produced by the project.

To generate Source Link having just the source control package is not sufficient, since various source control providers (hosts) differ in the way how they expose the content of the hosted repositories. A package specific to the provider is needed. The following Source Link packages have been released by Microsoft:

  • Microsoft.SourceLink.GitHub (depends on Microsoft.Build.Tasks.Git package)
  • Microsoft.SourceLink.AzureRepos.Git (depends on Microsoft.Build.Tasks.Git package)
  • Microsoft.SourceLink.AzureDevOpsServer.Git (depends on Microsoft.Build.Tasks.Git package)
  • Microsoft.SourceLink.GitLab (depends on Microsoft.Build.Tasks.Git package)
  • Microsoft.SourceLink.Bitbucket.Git (depends on Microsoft.Build.Tasks.Git package)

In addition an experimental package is available for TFVC server:

  • Microsoft.SourceLink.AzureRepos.Tfvc (depends on experimental Microsoft.Build.Tasks.Tfvc package)

The system is extensible and custom packages that handle other source control providers can be developed and used. See Custom Source Link packages for details.

Each Source Link package depends on the corresponding source control package. Referencing a Source Link package makes the dependent source control package also referenced, thus providing the other source control features to the project.

Note that it is possible and supported to reference multiple Source Link packages in a single project provided they depend on the same source control package. This is necessary when the project sources are stored in mutliple submodules hosted by different providers (e.g. Azure Repos repository containing a GitHub submodule). See Configuring Projects with Multiple Source Link Providers for details.

Basic Settings

PublishRepositoryUrl

To avoid inadvertently publishing private repository data, the URL and branch of the repository supplied by the CI server or retrieved from the source control manager are not consumed directly. Instead, they are stored in the PrivateRepositoryUrl and SourceBranchName variables.

If PublishRepositoryUrl is set to true by the project, these properties are published into the RepositoryUrl and RepositoryBranch properties, which are used e.g. in the nuspec file generated for the NuGet package produced by the project.

EmbedAllSources

Set EmbedAllSources to true to instruct the build system to embed all project source files into the generated PDB.

EmbedUntrackedSources

Set EmbedUntrackedSources to true to instruct the build system to embed project source files that are not tracked by the source control or imported from a source package to the generated PDB.

Has no effect if EmbedAllSources is true.

If the project generates additional source files that are added to Compile item group in a custom target, this target must run before BeforeCompile target (specify BeforeTargets="BeforeCompile"). Otherwise, these additional source files will not be automatically embedded into the PDB.

ContinuousIntegrationBuild

Set ContinuousIntegrationBuild to true to indicate that the build executes on a build/CI server.

ContinuousIntegrationBuild variable is used within the build system to enable settings that only apply to official builds, as opposed to local builds on developer machine. An example of such setting is DeterministicSourcePaths.

DeterministicSourcePaths

By setting DeterministicSourcePaths to true the project opts into mapping all source paths included in the project outputs to deterministic values, i.e. values that do not depend on the exact location of the project sources on disk, or the operating system on which the project is being built.

Only set DeterministicSourcePaths to true on a build/CI server, never for local builds. In order for the debugger to find source files when debugging a locally built binary, the PDB must contain original, unmapped local paths.

Starting with .NET Core SDK 2.1.300, a fully deterministic build is turned on when both Deterministic and ContinuousIntegrationBuild properties are set to true.

Example

The following project settings result in repository URL and commit hash automatically detected and included in NuSpec, commit hash included in AssemblyInformationalVersionAttribute, all source files available on GitHub linked via Source Link (including those in submodules) and source files not available on GitHub embedded in the PDB.

Note that .NET Core SDK 2.1.300 is required.

<Project Sdk="Microsoft.NET.Sdk">
 <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <PublishRepositoryUrl>true</PublishRepositoryUrl>
    <EmbedUntrackedSources>true</EmbedUntrackedSources>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0" PrivateAssets="All"/>
  </ItemGroup>
</Project>

Configuring Projects with Multiple Source Link Providers

An additional configuration might be required when a project references multiple Source Link packages and the source control is hosted on a custom domain. By default a Source Link package infers the domain of the source control host from the origin remote URL. For example, when a GitLab repository is cloned from http://git.contoso.com and a project only references Microsoft.SourceLink.GitLab package the package infers that git.contoso.com must be a hosting GitLab service. However, when multiple packages are used in the project it's not clear which service is git.contoso.com domain hosting.

Custom Host Domains

Each package defines an msbuild item group named SourceLink{provider}Host, where {provider} is the source control provider the package is built for (e.g. SourceLinkGitHubHost, SourceLinkGitLabHost, SourceLinkVstsGitHost etc.). The item group allows the project to specify domain(s) that correspond to the source control provider of the package. For example, the following setting assigns git.contoso.com domain to GitHub package:

<ItemGroup>
  <SourceLinkGitHubHost Include="git.contoso.com"/>
</ItemGroup>

Custom Content URLs

Content URL is the URL where the raw source files can be downloaded from. Items of SourceLink*Host item group allow to specify the content URL if necessary. The content URL doesn't need to be specified in most cases as it is inferred from the domain.

The default content URLs for each package is listed below (GetSourceLinkUrlGitTask.GetDefaultContentUriFromHostUri API):

content URL
GitLab https://{domain}
GitHub https://{domain}/raw
AzureRepos.Git https://{domain}
BitBucket https://{domain}

To override the above defaults specify ContentUrl metadata on the item in build/{PackageName}.props in the Source Link package. For example, GitHub.com server provides content on a CDN domain raw.githubusercontent.com:

<ItemGroup>
  <SourceLinkGitHubHost Include="github.com" ContentUrl="https://raw.githubusercontent.com"/>
</ItemGroup>

Advanced Settings and Concepts

IncludeSourceRevisionInInformationalVersion

When true and a source control package is present the SourceRevisionId is included in the AssemblyInformationalVersionAttribute. The default value is true. Set to false to suppress publishing SourceRevisionId to the attribute.

SourceRevisionId

Set by target InitializeSourceControlInformationFromSourceControlManager and consumed by NuGet Pack target and GenerateAssemblyInfo target. May be used by custom targets that need this information.

SourceBranchName

Set by target InitializeSourceControlInformationFromSourceControlManager and consumed by NuGet Pack target. May be used by custom targets that need this information.

EnableSourceLink

This property is implicitly set to true by a Source Link package. Including a Source Link package thus enables Source Link generation unless explicitly disabled by the project by setting this property to false.

SourceRoot

Item group that lists all source roots that the project source files reside under and their mapping to source control server URLs. This includes both source files under source control as well as source files in source packages.

Source root metadata:

  • Identity: full path to the source root directory ending with a directory separator

All source control roots have the following metadata:

  • SourceControl: the name of source control system, if the directory is a source source control root (e.g. git, tfvc, etc.)
  • RevisionId: revision id (e.g. git commit hash)

Additional source-control specific metadata may be defined (depends on the source control system).

For example, for Git:

  • RepositoryUrl: e.g. http://github.com/dotnet/corefx
  • BranchName: e.g. refs/heads/main (may be null if branch is in a detached HEAD state)

For TFVC:

  • CollectionUrl
  • ProjectId
  • ServerPath

Nested source control roots have the following metadata (e.g. submodules):

  • NestedRoot: URL to the source root relative to the containing source root (e.g. src/submodules/mysubmodule)
  • ContainingRoot: the identity if the containing source root

Source roots not under source control:

  • SourceLinkUrl: URL to use in source link mapping, including * wildcard (e.g. https://raw.githubusercontent.com/dotnet/roslyn/42abf2e6642db97d2314c017eb179075d5042028/src/Dependencies/CodeAnalysis.Debugging/*)

Creating Custom Source Link Packages

Each Source Link package is expected to provide mapping of repository URLs to corresponding content URLs that provide source file content.

The content URL shall identify an end-point that responds to HTTP GET request with content of the source file identified in the query. The end-point may require authentication.

The package shall depend on an appropriate source-control package (Microsoft.Build.Tasks.Git, Microsoft.Build.Tasks.Tfvc, etc.).

The package shall include build/{PackageName}.props and build/{PackageName}.targets files that get automatically included by NuGet into the project that references the package.

build/{PackageName}.props file shall set EnableSourceLink property to true if it hasn't been set already, like so:

<PropertyGroup>
  <EnableSourceLink Condition="'$(EnableSourceLink)' == ''">true</EnableSourceLink>
</PropertyGroup>

build/{PackageName}.targets file shall add a uniquely named Source Link initialization target to SourceLinkUrlInitializerTargets property, e.g. _InitializeXyzSourceLinkUrl for source control provider called Xyz.

<PropertyGroup>
  <SourceLinkUrlInitializerTargets>$(SourceLinkUrlInitializerTargets);_InitializeXyzSourceLinkUrl</SourceLinkUrlInitializerTargets>
</PropertyGroup>

The initialization target shall update each item of the SourceRoot item group that belongs to the Xyz provider with SourceLinkUrl metadata that contains the final URL for this source root that will be stored in the Source Link file. It shall ignore any SourceRoot items whose SourceControl and RepositoryUrl metadata it does not recognize.

See the implementation of GitHub Source Link package for an example.

Minimal git repository metadata

In some scenarios it is desirable to build a repository from a directory that contains its sources but does not have .git directory containing git metadata. Source Link does not require all git metadata to be present in the .git directory, but some metadata are needed. The following list describes the minimal set of directories and files that must be present in order for Source Link to operate properly.

  • .git/HEAD

    text file containing a commit SHA (e.g. 935f4b5c55167d9e4ed99b753f7340999d66d5de)

  • .git/config

    configuration file that specifies origin remote URL (e.g. [remote "origin"] url="http://server.com/repo")

If the repository has submodules the file .gitmodules must be present in the repository root and must list the relative paths of all submodules:

[submodule "submodule-name"]
  path = submodule-path

The .git/config file must contain URLs of all initialized submodules:

[submodule "submodule-name"]
  url = https://server.com/subrepo

The following additional files and directories must be present for each submodule:

  • .git/modules/submodule-name/HEAD

    text file containing a commit SHA of the submodule

  • submodule-path/.git

    text file pointing to the submodule metadata, with the following contents: gitdir: ../.git/modules/submodule-name