Do not set /m parameter for MSBuild by default #1104

ravensorb · 2016-07-23T16:00:05Z

What You Are Seeing?

Xamarin based projects are not compiling correctly. Either SDK files are not found or Android packages are not downloading correctly on a clean build server with minimal build tools installed.

Note: This works perfectly if I execute msbuild manually on exact same system

and

C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5207: Please install package: 'Xamarin.Android.Support.v4' available in SDK installer. Java library file C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v4\23.3.0.0\embedded\classes.jar doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]
C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5207: Please install package: 'Xamarin.Android.Support.v4' available in SDK installer. Java library file C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v4\23.3.0.0\embedded\libs/internal_impl-23.3.0.jar doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]
C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5206: Please install package: 'Xamarin.Android.Support.v4' available in SDK installer. Android resource directory C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v4\23.3.0.0\embedded\./ doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]
C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5207: Please install package: 'Xamarin.Android.Support.v7.CardView' available in SDK installer. Java library file C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v7.CardView\23.3.0.0\embedded\classes.jar doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]
C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5206: Please install package: 'Xamarin.Android.Support.v7.CardView' available in SDK installer. Android resource directory C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v7.CardView\23.3.0.0\embedded\./ doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]
C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5207: Please install package: 'Xamarin.Android.Support.v7.RecyclerView' available in SDK installer. Java library file C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v7.RecyclerView\23.3.0.0\embedded\classes.jar doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]
C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(387,2): error XA5206: Please install package: 'Xamarin.Android.Support.v7.RecyclerView' available in SDK installer. Android resource directory C:\Users\build\AppData\Local\Xamarin\Xamarin.Android.Support.v7.RecyclerView\23.3.0.0\embedded\./ doesn't exist. [E:\Builds\GitHub\Xamarin-Forms-Labs\src\Forms\XLabs.Forms.Droid\XLabs.Forms.Droid.csproj]

I have also replicated this on both a Windows 10 (Build 1511 base) and a Windows 2016 Tap 5 build server.

It seems that there is something that cake is doing when it executes msbuild that is resulting in the error.

What is Expected?

Build should work as expected. if I run msbuild manually it works 100% of the time on a clean system

What version of Cake are you using?

Tried

Version 0.13.0+Branch.main.Sha.f7346f2a5fb4e19a24cf15780ce635529f921be4
Are you running on a 32 or 64 bit system?
64

What environment are you running on? Windows? Linux? Mac?

Windows 2012R2
Windows 10 (based on 1511)
Windows 2015 Tap 5

Are you running on a CI Server? If so, which one?

Standalone and appveyor (both have same issue)

https://ci.appveyor.com/project/xlabs/xamarin-forms-labs-hva8p/build/2.3.0.3.master

How Did You Get This To Happen? (Steps to Reproduce)

Test 1
Windows 2012 R2

Setup
- Windows Updates as of 7/21/2016
- Install Chocolatey
- Install 7zip
- Install Visual Studio 2015 Enterprise w/Update 3 (full install all components except emulators and lightspeed)
Build Process
- Clone repository
- Execute .\build.ps1 -Target Package
- Result Fails
- Immediately execute .\build.ps1 -Target Package (which does a full clean)
- Result Succeeds

Test 2
Windows 2012 R2

Setup
- Windows Updates as of 7/21/2016
- Install Chocolatey
- Install 7zip
- Install Visual Studio 2015 Enterprise w/Update 3 (full install all components except emulators and lightspeed)
- Install Xamarin Tools (from Xamarin.com) with defaults (see screenshots below)
Build Process
- Clone repository
- Execute .\build.ps1 -Target Package
- Result Fails

!(AndroidSDKInstall)[https://i.imgur.com/T76dTVv.png]
!(MissingComponents)[https://i.imgur.com/fkWOYAx.png]

Test 3
Windows 2012 R2

Setup
- Windows Updates as of 7/21/2016
- Install Chocolatey
- Install 7zip
- Install Visual Studio 2015 Enterprise w/Update 3 (full install all components except emulators and lightspeed)
- Install Xamarin Tools (from Xamarin.com) with defaults (see screenshots below)
Build Process
- Clone repository
- Execute ..nuget\nuget restore .\src\XLabs.sln -ConfigFile ..nuget\nuget.config
- Execute msbuild /m /v:normal /p:Configuration="Release" /p:Platform="Any CPU" /p:TreatWarningsAsErrors=False /target:Build "E:/Builds/GitHub/Xamarin-Forms-Labs/src/XLabs.sln
- Result Fails

Test 4
Windows 2012 R2

Setup
- Windows Updates as of 7/21/2016
- Install Chocolatey
- Install 7zip
- Install Visual Studio 2015 Enterprise w/Update 3 (full install all components except emulators and lightspeed)
- Install Xamarin Tools (from Xamarin.com) with defaults (see screenshots below)
Build Process
- Clone repository
- Execute ..nuget\nuget restore .\src\XLabs.sln -ConfigFile ..nuget\nuget.config
- Execute msbuild /v:normal /p:Configuration="Release" /p:Platform="Any CPU" /p:TreatWarningsAsErrors=False /target:Build "E:/Builds/GitHub/Xamarin-Forms-Labs/src/XLabs.sln
- Result Sucess

The issue seems to be related to the /m property

Link to GitHub Repository

Output Log

cakebuild-20160723-0706.txt

The text was updated successfully, but these errors were encountered:

gep13 · 2016-07-23T16:06:30Z

@ravensorb Have you ran Cake with diagnostic verbosity, found the msbuild command it is using, and then tried running that directly?

gep13 · 2016-07-23T16:07:30Z

@ravensorb does that work? If so, what is different between you running manually?

ravensorb · 2016-07-23T16:08:10Z

yes -- the log file attached shows a failed build. I have tracked it down the the "/m" as I mentioned in the write up. It is possible that this is actually an msbuild issue. I am going to try to set MaxCpu to 0 in my MSBuildSettings next

devlead · 2016-07-23T16:09:47Z

@ravensorb make sure you're running same version of MSBuild too, you can set toolversion via the settings.

gep13 · 2016-07-23T16:09:48Z

Off the top of my head, I am not familiar with the "/m" flag. What is it? And how is it getting set?

ravensorb · 2016-07-23T16:10:41Z

@devlead yes, I am 100% sure. I took the msbuild command from the detailed log, removed the /m and ran it -- it worked

@gep13 the /m tells msbuild to do a multi-threaded build

gep13 · 2016-07-23T16:11:37Z

So are you setting that in your cake file? On phone just now, so hard to check.

ravensorb · 2016-07-23T16:15:47Z

@gep13 seems like default behavior of cake is to set the /m option. I am going to override it and see if that helps right now

gep13 · 2016-07-23T16:28:49Z

@ravensorb so, looks like this value is set by default: https://github.com/cake-build/cake/blob/develop/src/Cake.Common/Tools/MSBuild/MSBuildRunner.cs#L55 either to just /m or /m:{x} where x is the number of CPU's on the machine.

gep13 · 2016-07-23T16:32:32Z

@cake-build/team should the following:

builder.Append(settings.MaxCpuCount > 0 ? string.Concat("/m:", settings.MaxCpuCount) : "/m");

be:

builder.Append(settings.MaxCpuCount > 0 ? string.Concat("/m:", settings.MaxCpuCount) : "/m:1");

gep13 · 2016-07-23T16:34:02Z

Above suggestion is based on this:

ravensorb · 2016-07-23T16:37:21Z

@gep13 I believe a /m on its own will use all possible CPUs. There should be a way to "remove" the /m if needed (maybe set MaxCpuCount = -1 or make it a nullable type (null don't include, 0 include /m on its own, anything else try as an actual number)?

gep13 · 2016-07-23T16:49:02Z

@ravensorb yes, however, surely \m:1 would be the same as not including it, right? As you are saying only use one CPU.

patriksvensson · 2016-07-23T16:57:47Z

We can probably make it nullable (don't set if null) and make zero mean /m. The default value should probably be null to keep as close to just running msbuild.exe myproject.sln.

What do you think?

ravensorb · 2016-07-23T16:57:48Z

@gep13 not sure yet -- still testing. That will be one of the tests I run and I'll let you know what the result it.

ravensorb · 2016-07-23T16:58:08Z

@patriksvensson I would agree

gep13 · 2016-07-23T16:59:01Z

Me too.

gep13 · 2016-07-23T17:00:06Z

@ravensorb would you be in a position to submit a PR?

@patriksvensson should this be a hotfix?

patriksvensson · 2016-07-23T17:03:06Z

@gep13 Not against it being a hotfix, but since it wasn't introduced in 0.14.0, perhaps it's better to include it in 0.15.0. That is if there's a workaround that solves the issue.

What do you think?

gep13 · 2016-07-23T17:08:51Z

Makes sense to me :-)

ravensorb · 2016-07-23T18:31:59Z

Using the following

MSBuild(solution, settings =>
    settings.SetPlatformTarget(PlatformTarget.MSIL)
                .SetMaxCpuCount(1)
                .WithProperty("TreatWarningsAsErrors",buildSettings.Build.TreatWarningsAsErrors.ToString())
                .WithTarget("Build")
                .SetConfiguration(configuration));

Seems to work -- everything compiles now.

I have also submitted an issue on the msbuild git repository

gep13 added this to the v0.15.0 milestone Jul 24, 2016

gep13 added the Bug label Jul 24, 2016

gep13 changed the title ~~Trouble building solution on a clean build server~~ Corrected setting /m parameter to MSBuild when not requested Jul 24, 2016

patriksvensson mentioned this issue Jul 24, 2016

GH1104: Made MSBuildSettings.MaxCpuCount nullable #1105

Merged

gep13 closed this as completed in #1105 Jul 25, 2016

patriksvensson changed the title ~~Corrected setting /m parameter to MSBuild when not requested~~ Do not set /m parameter for MSBuild by default Jul 25, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not set /m parameter for MSBuild by default #1104

Do not set /m parameter for MSBuild by default #1104

ravensorb commented Jul 23, 2016 •

edited

Loading

Are you running on a 32 or 64 bit system?

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

devlead commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

patriksvensson commented Jul 23, 2016 •

edited

Loading

ravensorb commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

patriksvensson commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

Do not set /m parameter for MSBuild by default #1104

Do not set /m parameter for MSBuild by default #1104

Comments

ravensorb commented Jul 23, 2016 • edited Loading

What You Are Seeing?

What is Expected?

What version of Cake are you using?

Are you running on a 32 or 64 bit system?

What environment are you running on? Windows? Linux? Mac?

Are you running on a CI Server? If so, which one?

How Did You Get This To Happen? (Steps to Reproduce)

Output Log

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

devlead commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

patriksvensson commented Jul 23, 2016 • edited Loading

ravensorb commented Jul 23, 2016

ravensorb commented Jul 23, 2016

gep13 commented Jul 23, 2016

gep13 commented Jul 23, 2016

patriksvensson commented Jul 23, 2016

gep13 commented Jul 23, 2016

ravensorb commented Jul 23, 2016

ravensorb commented Jul 23, 2016 •

edited

Loading

patriksvensson commented Jul 23, 2016 •

edited

Loading