Merge Portable PDB implementation into master

[tmat]

Ports Portable PDB to master. Portable PDBs are now available thru /features:pdb=portable or /features:pdb=embedded.

[tmat]

@jaredpar @agocke

[ghuntley]

Related spec - https://github.com/dotnet/roslyn/blob/master/docs/specs/PortablePdb-Metadata.md

[ctaggart]

The PDB format supports source indexing. Will the Portable PDB format support source indexing?

[tmat]

@ctaggart Yes. The details are still TBD.

[ctaggart]

If you want my feedback, please reach out. I'm a .NET MVP. I helped make SourceLink over the last two years to automate source indexing. http://ctaggart.github.io/SourceLink/

My wish list:

• only support for http/https
• authentication that would be compatible with GitHub
• be able to navigate to all source files at design time too

[tmat]

@ctaggart Your feedback would indeed be welcome. The features you mention seem out of scope of the format. They are more asks for the tools that work with pdbs. The format just needs to contain enough information to support these tools.

Right now I'm thinking about adding a compiler parameter that specifies a file with additional custom debug information data. Something like /customDebugInfo:{GUID}=<path>. The compiler would add a row to CustomDebugInformation table (with Parent = Assembly) and embed the content of the specified file on the #Blob heap of the portable PDB.

In addition, we are also thinking about the ability to mark input source files as "debuggable tool generated file" (i.e. not checked into any source control, but still something that the user would want to step thru). The content of such files would be zip compressed and stored in the Portable PDB as well.

Each kind of source server (GIT, TFS, Subver, etc) has their own format. A source server specific tool would need to run as a task before compilation and create the CDI file passed to /customDebugInfo.

Since the SHA1 hashes of all source files are already included in the Document table the only information needed to find the sources in GIT repo would be the URL of the repo. So the generated CDI would just contain that.

Sounds reasonable?

[ctaggart]

That may work. Right now, adding the source index to a pdb is done as a step after compilation. The advantage is that you can then get a list of files that the compiler has added symbols for and only index them. It would be ok to add a list of all the compiled files. After the ppdb is created, it would be useful to have another hook for validation of the source index.

Today, the source index is missing a way to specify the HTTP Authentication Scheme. It works with Basic Auth and Windows Auth. There is no way to set something so that srcsrv.dll sends the credentials (.NET .PreAuthenticate = true). This prevents GitHub compatibility with Basic Auth and is the reason why I made SourceLink-Proxy. It would be great if there was a way to specify the HTTP Auth in an agreed upon Auth workflow the tools/websites can support.

[tmat]

@ctaggart Rewriting PDB and adding more data after the fact is problematic. When the compiler builds in deterministic mode it calculates the MVID and PDB ID as a hash of the PE and PDB files, respectively. These values are then saved in the file. Any changes in the content would require rehashing of both PE and PDB.

I actually don't understand why the current source information includes a map of all source files to source control URIs. To be able to find a source file on a TFS/sd server the only information necessary is the server URL and a workspace mapping.

[tmat]

@ctaggart I have created an issue to track this: #4119

[erik-kallen]

Sorry for asking a stupid question, but why isn't all source code included in the PDB? Sure it will be bigger, but I would absolutely prefer that over needing to find the exact correct version of all source files, especially when they're third-party dependencies.

[tmat]

@erik-kallen Because it would be bigger :) We might provide an option to do so, if you want to.

[0x53A]

+1 for the suggestion of erik-kallen to (optionally!) embed the source in the pdb.
This would be especially useful for small apps, which might (heavens forbid!) not even be under source control. Just saving the .exe and .pdb would then allow source-debugging.

It might also be useful for nugget libraries. Instead of having to configure 1) nuget, 2) symbol server and 3) source server, you could skip 3. (Some nuget packages also contain the pdb, so these could be self-contained, but that practice might not be optimal, as it increases the size of the package)

Lastly, it might be useful for offline scenarios, where you have the pdb but no access to the internet to download the sources.