Make build.cmd with no args build everything

[natemcmaster]

Currently, running build.cmd doesn't build everything in this repo. You have to know about special parameters to get the repo build Debian, RPM, and Windows installers.

Proposal: let's make build.cmd/.sh without arguments build everything in the repo. That includes:

• C#, C++, TypeScript, and Java projects
• On Windows, MSI installers.
• On Linux, the installer that matches the current platform (deb, rpm, tar.gz, etc.)

Tricky bits: this is not trivial because we have some tricky requirements.

• the Windows Server Hosting bundle needs to include both the x64 and x86 versions of ASP.NET Core. Should build.cmd on Windows compile both x64 and x86, or just one?
• Pre-requisites. To build everything, you need .NET 3.5 (for the Wix Toolset), NodeJS, the JDK, Visual Studio, Windows SDKs, and more. We don't have scripts to install or acquire all of these automatically. We should add scripts which check early on for these prerequisties because builds will often fail with confusing messages later if prereqs are not met.

[pakrym]

the Windows Server Hosting bundle needs to include both the x64 and x86 versions of ASP.NET Core. Should build.cmd on Windows compile both x64 and x86, or just one?

Maybe build.cmd should invoke x86/x64 and then "unifying" build when ran from the root.

[natemcmaster]

@Eilon what are your thoughts on this one? Do you want contributors working our repo to be able to run "build.cmd" and have it build everything? I can see this going two ways.

Pros:

• build.cmd is simple and you don't have to hunt for command line args

Cons:

• For this work, you must have all prerequisites installed, which include .NET 3.5 (wix toolset), NodeJS, JDK, and Visual Studio.

[Eilon]

Well, I don't speak for all contributors, so I don't know 😄

One way to go about this is that if we don't have any data on any actual problems that are being reported, to leave things as they are.

For example:

• Are people complaining? (E.g. that the build takes too long? Doesn't produce enough assets? Can't figure out how to get a particular sub-component to build?)
• Are we getting poor quality PRs because the build doesn't do what might seem reasonable?

[natemcmaster]

I heard a few of our team members say it was hard to figure out how to build installers. I made a few changes that should make it easier now. And I have a few more planned as a part of targeting pack work.

Other than that, most complaints have been about trouble with installing prerequisites like Visual Studio. Implementing this change to build.cmd would probably make that even worse by increasing the prerequisites just to get a successful build.cmd.

For what it's worth, as of Friday, PRs now build everything. Windows installers, Debian packages, all CPU architectures, etc. We have never had this kind of coverage in PR checks, so I think this will help us.

[Eilon]

Okie dokie. Another thing I'm also fine with is doing reasonable experiments and evaluating the results. So, great, now it builds everything. Does anyone start complaining or think it's unreasonable in practice?

[pakrym]

I don't think anyone runs full build except people that need to test installers/sharedfx builds. And when you need to test those it's super painful to assemble the pieces.

I'm in favour of doing a full build considering we have good support for builds in subfolders.

[Eilon]

Great, this is the type of discussion I want to have!

[Eilon]

Suggestion:

Have build.cmd/.sh show a bright warning at the top saying it will build everything. EVERYTHING. But that you can build just a sub-section of the build by going to github.com/FOO (maybe using aka.ms) and running such-and-such.

[Eilon]

See also: #8025

[Eilon]

@dougbu - can you add more thoughts to this issue about what the lower-level build.cmd's should do?

[dougbu]

This work requires two main changes:

1. change build.ps1 / build.sh files in the root folders of our repos to build the world and to require all of the prerequisites
   • but, when (say) -NoBuildJava or --no-build-java (for build.sh only) is specified, disable prerequisite check for Java installation
2. change lower-level build.ps1 / build.sh files to include whatever -no* options allow a build from scratch in that directory
   • for example, src/Mvc/build.ps1 might include -NoBuildManaged, -NoBuildNative, -NoBuildNodeJS, -NoBuildJava (assuming all of that can be excluded when building dependencies) when executing the root build.ps1

I'm not sure what to do with -Pack because some projects require that be done before they're built. One option may be for the affected lower-level scripts to add this option. -BuildInstallers / -NoBuildInstallers might be similar. @ryanbrandenburg ?

[serpent5]

I landed here because Build ASP.NET Core from Source explains how I need to run the installers and configure NuGet to look at the packages folder to "use the result of your build", but there are no runnable installers and the packages folder is almost empty. The earlier sections in the instructions don't mention -Pack or -BuildInstallers and now I'm wondering from the comments here whether those instructions on how to use the build results are still relevant.

[Pilchie]

@JunTaoLuo - this seems very related to your current PR for Components. Do we still need this separate issue?

[JunTaoLuo]

This is quite separate. Even with my change, we are still only building a subset of our artifacts. Specifically, we will be building C# projects and everything C# projects depend on. We still won't build installers, site extensions, Java artifacts, etc. This issue is tracking the proposal to build everything that's possible on the platform you're on.

[Pilchie]

Got it. I'm going to move this to backlog then, as I don't think it's something that we need to do for .NET Core 3 and we're running out of time for that.

[JunTaoLuo]

Unassigning myself since this is a backlog item.

[kant2002]

Can this discussion can be revived? I hit that when I attempt to build installer for ASP.NET.
Runtime and WindowsDesktop repository by default provide these installers.
Reason why I start building installers following: During my work on dotnet/corert#4219 I attempt to build custom CoreCLR, and I thought that simpler way to make use of existing tooling was simply build Runtime and WindowsDesktop, since I use only them. But unfortunately .Net SDK require ASP.NET to be available, and I was dragged into this. That's not quite contribution scenario, but I would still like to have default experience with -BuildInstallers

[dougbu]

@kant2002 when we build the installers locally (which is very rare), we follow along with the .azure/pipelines/ci.yml. In particular, local devs don't usually need to build both x64 and x86 assets on Windows, let alone the site extensions or installers.

[kant2002]

@dougbu Can steps for building installers to be put in docs? I can submit PR. I understand that I take unusual path. If documentation update is proper way, does https://github.com/dotnet/aspnetcore/blob/master/docs/BuildFromSource.md proper place for adding that information?

[dougbu]

@kant2002 sure, we'd accept a PR that added another section at the end of docs/BuildFromSource.md. Much appreciated❕

[dougbu]

From old DoI board:

Make it possible to build all Windows assets in one command? (Linked issue couldn't be done without providing some options to build everything in one go.)

[AraHaan]

Building installers for me locally now fails due to the Unzip parts in the TargetingPack.wixproj and the SharedFramework.wixproj wix project files (due to it seeming that the zip files are never made).

[AraHaan]

It seems the reason why the installers refuse to build could be because I was not passing in -sign that the BuildFromSource docs forgot to mention.