Support application manifests on Windows

[ColonelJ]

Currently Rust has no support for adding manifests to Windows executables (.exe and .dll) despite some previous failed attempts (see rust-lang/rust#11207).

Application manifests are essentially just an XML resource which is embedded in the executable and is read by Windows to establish how the executable should be run and which DLLs it depends on. There are also other types of manifests which follow the same basic schema but have slightly different purposes, e.g. you can publish configuration files to install on the system which specify that a particular version of a DLL installed is backwards compatible and should be used in place of a whole range of previous version numbers.

Manifests may be useful for several reasons in Rust:

• Allows easily viewable version numbers and descriptive text on an .exe or .dll
• Can specify which version of a system (or any other) .dll your application should use, thus avoiding DLL hell - notable alternative use case for this is to ensure version 6 or above of Microsoft Common Controls which will enable XP styles of window controls in 32-bit applications; this particular one probably ought to be automatically included for apps requiring COMCTL32.dll since without you'll get some ugly ancient looking GUIs.
• Ability to specify UAC privilege level required for your application and avoid Windows determining this heuristically (temporarily solved for test runs using an environment variable in Prevent UAC blocking certain test executables on 32-bit Windows. rust#21496)
• Allows use of code signing for distributed binaries (also needs a certificate). This is a must for writing Windows drivers nowadays.
• Enable use of certain features/modes such as (super) high-res scrolling, DPI awareness, printer driver isolation...

There is another significant feature which should NOT be used by default, which is that you can specify which OS version behaviours are supported by your application, which effectively requests certain APIs to behave differently. The reason this is bad is that it is impossible to know if anything that plugs in to your app (e.g. shell extensions, which come in through e.g. common file dialog) also supports these behaviours, so it is generally not safe to turn them on. Even so, for flexibility it would be good to allow users to explicitly request these behaviours.

I believe that the scope and format of these application manifests is sufficiently limited (particularly parts most relevant to Rust) that the Rust ecosystem ought to have the ability to generate these by itself with the help of Rust language attributes to specify particular items/settings to go into the manifest. Having some sane approach to version numbering (which ties into Cargo somehow) would be massively beneficial and avoid all sorts of problems with incompatible binaries. The version numbers for manifests are required to have four parts 0-65535 (without exception) which Microsoft expliclty labels as major.minor.build.revision in the documentation, and changes in the last two are not to break backwards compatibility, which would generally be done by incrementing the major version rather than minor. This I find to be a good link on the motivations behind the versioning system on Windows and there might be no reason why Rust couldn't implement such automatic revision number stepping on top of a similar version attribute.

Likely people will want to override whatever manifest Rust provides (or disable it completely) and this should be catered for. However, it could potentially be useful to have an option for some sort of hybrid approach where Rust can change certain fields such as version number from an existing manifest file. N.B. application manifests are always called filename.exe.manifest or filename.dll.manifest so they could always simply be detected from the filesystem rather than specified on the command line or in an attribute (though being able to specify an alternative manifest path via an attribute may be another very useful feature).

As a demonstration of most of the things I've said, and to demonstrate how simple the XML is, I have prepared an example, which in this case would normally be called helloworld.exe.manifest:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1"
          manifestVersion="1.0">
  <assemblyIdentity type="win32"
                    name="helloworld"
                    version="1.7.2.19"
                    processorArchitecture="x86"/>
  <description>Hello World Application</description>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity type="win32"
                        name="Acme.Soft.SayStuffLib"
                        version="3.7.2.4"
                        processorArchitecture="x86"
                        language="*"/>
    </dependentAssembly>
  </dependency>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity type="win32"
                        name="Microsoft.Windows.Common-Controls"
                        version="6.0.0.0"
                        processorArchitecture="x86"
                        publicKeyToken="6595b64144ccf1df"
                        language="*"/>
    </dependentAssembly>
  </dependency>
  <file name="hellointl.dll"
        hash="12345678123456781234567812345678abcdef00"
        hashalg="SHA1"/>
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel level="asInvoker" uiAccess="false"/>
      </requestedPrivileges>
    </security>
  </trustInfo>
  <asmv3:application xmlns:asmv3="urn:schemas-microsoft-com:asm.v3">
    <asmv3:windowsSettings
        xmlns="http://schemas.microsoft.com/SMI/2011/WindowsSettings">
      <dpiAware>true</dpiAware>
      <printerDriverIsolation>true</printerDriverIsolation>
    </asmv3:windowsSettings>
  </asmv3:application>
</assembly>

This shows providing version number and description of your app, referencing DLLs by assembly name and version which may be Rust built or on the system, referencing a private DLL which must match the expected file using SHA1 hash, activating XP styles by requesting version 6 of common controls, telling UAC that no special privileges are required (even if the app was called setup.exe) and indicating that your app is DPI aware and that the printer driver should run in a separate process (so it doesn't crash your app if it is faulty). If your app were code signed it would have a public key token as COMCTL32.dll does.

I think there's still a lot of work to do on reconciling differences with how Rust is currently and how these features could possibly be implemented, but I think once Rust (and Cargo too most likely) is capable of all the things above and can generate the manifest mostly by itself, things are looking pretty strong on Windows.

[klutzy]

Technically, manifest is just part of resource if embedded, and such resource is compiled by system linker.

link.exe (MSVC toolchain) accepts /MANIFEST. With /MANIFEST:embed flag, linker will produce final output with embedded manifest. VC++ wizard seems to set such flags automatically.

There is other way to embed manifest: register it in resource file (.rc), and pass it to resource compiler and linker. MSVC toolchain provides rc.exe for such purpose.
binutils also provides windres.exe, so you can embed manifest with gcc/binutils toolchain: windres resource.rc resource.res and rustc -C link-args "resource.res".
(rust-lang/rust#10878 didn't take this road because it requires additional dependency on windres.exe.)

However, it may not be convenient to manage manifest and resource file just to set UAC.
link.exe provides /MANIFESTUAC flag for such use.
The flag doesn't exist in gcc/binutils. Instead, GCC recently added "default manifest" routine. (patch discussion) It works like: if the toolchain has "default-manifest.o" file, it is passed to linker with the lowest precedence. (If you pass your own manifest to linker, the default one will be ignored, not merged. This is why I said "magical", but now I think it's not a big issue if the default manifest only has UAC, like /MANIFESTUAC does.)
I don't know if mingw-w64 and/or msys2 package have default-manifest.o file. If this is done, we can get UAC support for free!

Now my personal thought. I don't think rustc should handle manifest or resource in general. I think It's not job of rustc to handle version/dependency issues as well - it's cargo job. I have no idea if it's good for cargo to embed manifest or not.
For UAC, I now have new idea similar to GCC patch: prepare build our own default-manifest.o and bundle it to installer. teach rustc to pass it to linker if the default manifest is available.

cc @vadimcn @retep998: thoughts?

[klutzy]

(@vadimcn: I'm curious if the gdb crash in rust-lang/rust#11207 was specifically due to UpdateResource or it's general issue with manifest.)

[vadimcn]

My only reservation about shipping a pre-built manifest.o was that currently rustc contains no logic for searching for .o files. Adding it just for the sake of Windows manifests would be a bit of overkill, IMO, when it isn't that hard to run the windres tool after linking from your makefile or build script, if you really do care about embedding a manifest.
But if GCC is going to do searching for us, I don't see why not.

[retep998]

If we want to claim we properly support Windows then Cargo will need to be able to generate manifests and give them to rustc so it can link them in. Definitely something that should be worked on post 1.0.

As for default-manifest.o, it would be a nice way to resolve certain issues in the meantime, but it should only be a temporary measure.

[ColonelJ]

Although I agree that Cargo ought to have a lot of involvement with manifests, the problem is that, for the application manifest in particular, the version numbers of dependencies are supposed to match those of the binaries linked against at compile time. The job of compiler/linker is handled by rustc, not by Cargo. As such I think it is preferrable for rustc to handle manifest generation whilst providing support for Cargo to control the process. With some simple controls in Rust it should be easy to build an app with slightly customised manifest (e.g. specify a version for your app), without requiring you to build using Cargo. N.B. if Cargo generates the entire manifest file by itself this implies it has to handle all aspects of the manifest, not just versioning, and since the manifest may change program behaviours, compiling using Cargo becomes the only option.

Also regardless of how it is implemented users should still be able to add their own custom parts to the manifest somehow or other. User providing their own complete manifest file I think is highly undesirable when you want Cargo to control the versioning.

Regarding a default manifest, I think there should indeed be defaults for the manifest but there is not one single default manifest that can be applied to all applications. For one, you'd most likely want a different manifest for 64-bit vs 32-bit, but for a less trivial example, I believe applications should default to requiring version 6 of COMCTL32.dll, but if the application does not use COMCTL32.dll then this should clearly not be in the manifest.

I should note that the main purpose of this discussion is not to solve the UAC problem nor to provide default manifests to applications. Rather it is about how Rust might be able to provide easy access for the user to the (limited) functionality available in manifests and how Rust/Cargo can integrate with Windows for the main purpose of handling versions/dependencies. Good defaults, compatibility and removal of the UAC issue are just things that should come out of this process 'for free' and solutions that only solve these problems (especially things such as /MANIFESTUAC) really aren't relevant to this discussion.

[nabijaczleweli]

What's the status on this?

[retep998]

@nabijaczleweli Absolutely zero progress.

[DeadNumbers]

@nabijaczleweli I use mt.exe on wine.

http://stackoverflow.com/questions/1423492/how-do-i-add-a-manifest-to-an-executable-using-mt-exe

[nabijaczleweli]

@DeadNumbers That's neat, I used this technique instead, though.

[cretz]

@nabijaczleweli - That link is dead, but I assume your repo at https://github.com/nabijaczleweli/cargo-update shows the technique

[nabijaczleweli]

Yeah, Katt must've killed his repo.

cargo-update does use embed-resource, which has that method of including manifests generalised.

[isonmad]

Could this be done 'magically' by cargo when it sees a .manifest file in the right location, e.g. at src/bin/cargo.rs.manifest?

Then it could simply pass the option /MANIFESTINPUT, with that manifest file, to link.exe when building the executable.

Although it does have a version field that you might ideally want to generate at build time instead, so maybe cargo could also automatically fill that field with the current version number before giving it to the linker.

Of course if cargo could install arbitrary files besides the binaries themselves rust-lang/cargo#2729 then that might help with this too, with helloworld.exe.manifest as a separate file instead of embedding it.

[Serentty]

This is required in order to use the new (added in 2019) UTF-8 code page on Windows. While I think that it would also be nice to have a higher-level Cargo flag for that, embedding manifests is the first step.

[riverar]

One thing worth mentioning is that the manifest can be placed side-by-side with the executable as a workaround for now (e.g. ferris.exe.manifest), though not automatically rust-lang/cargo#2729. I'd also like to see the solution be generalized enough to support the embed of icons and other resources too.

[ghost]

I'd also like to see the solution be generalized enough to support the embed of icons and other resources

@riverar could you clarify the part here ?