Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supply official build id to inner-source build command #75293

Merged
merged 13 commits into from
Sep 13, 2022

Conversation

mmitche
Copy link
Member

@mmitche mmitche commented Sep 8, 2022

Fixes #75290

@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost assigned mmitche Sep 8, 2022
@mmitche mmitche requested a review from crummel September 8, 2022 18:38
@mmitche
Copy link
Member Author

mmitche commented Sep 8, 2022

@mmitche
Copy link
Member Author

mmitche commented Sep 8, 2022

Included a fix to avoid using the ESRP connection that only some branches can use.

@mmitche
Copy link
Member Author

mmitche commented Sep 9, 2022

This didn't quite work. The packages got the correct build number, but they got dev instead of alpha.1

@mmitche
Copy link
Member Author

mmitche commented Sep 9, 2022

@carlossanlop @hoyosjs Maybe some more eyes on this now...

  • I changed the usages of DisableSourceLink to instead check for EnableSourceLink. https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Arcade.Sdk/tools/RepositoryInfo.targets#L13-L17. It appears that arcade will use the DisableSourceLink switch to properly set EnableSourceLink, and that repos should only ever test against EnableSourceLink, not DisableSourceLink. I'm not sure what the history around this is, as it seems like a very bad idea to have two opposite switches for the same feature, since essentially you have 6 possible states which can conflict. @tmat?
    • Enable (not set/true/false)
    • Disable (not set/true/false)
  • Don't assume EnableSourceLink is going to be true for a CI build (it isn't for source build legs)
  • Pass the ContinuousIntegrationBuild and OfficialBuildId switches to the inner source build command. This would usually be part of the base command, but runtime overrides that completely.

@tmat
Copy link
Member

tmat commented Sep 9, 2022

Re Source Link: EnableSourceLink is the official property owned by Source Link package.

If false the Source Link JSON file is not gonna be generated. However, even with that disabled other parts of the build might still query GIT for information (e.g. commit SHA included in version numbers). This is disabled by EnableSourceControlManagerQueries.

In Arcade DisableSourceLink is a big hammer that disables everything, including DeterministicSourcePaths.
Perhaps it could be renamed to something better. I think it was added to enable build from a .zip file that contains source of the repo, but no git info. Don't recall details though, e.g. why disabling DeterministicSourcePaths is needed.

@mmitche
Copy link
Member Author

mmitche commented Sep 9, 2022

Alright, I think this test build is going to work...builds locally as expected: https://dev.azure.com/dnceng/internal/_build/results?buildId=1988755&view=results

@mmitche
Copy link
Member Author

mmitche commented Sep 9, 2022

Artifacts look gooooood:

image

@hoyosjs
Copy link
Member

hoyosjs commented Sep 10, 2022

The inner loop is a product issue that's unrelated to this change, fixed already #75041:

2022-09-09T19:32:05.2818961Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error : Fatal error. Internal CLR error. (0x80131506) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2826039Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.SZArrayHelper.GetEnumerator[[Internal.JitInterface.CorJitFlag, ILCompiler.ReadyToRun, Version=7.0.0.0, Culture=neutral, PublicKeyToken=null]]() [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2834377Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at Internal.JitInterface.CorInfoImpl.getJitFlags(Internal.JitInterface.CORJIT_FLAGS ByRef, UInt32) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2846510Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at Internal.JitInterface.CorInfoImpl._getJitFlags(IntPtr, IntPtr*, Internal.JitInterface.CORJIT_FLAGS*, UInt32) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2850741Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr ByRef, IntPtr, IntPtr, IntPtr, Internal.JitInterface.CORINFO_METHOD_INFO ByRef, UInt32, IntPtr ByRef, UInt32 ByRef) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2870455Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at Internal.JitInterface.CorInfoImpl.JitCompileMethod(IntPtr ByRef, IntPtr, IntPtr, IntPtr, Internal.JitInterface.CORINFO_METHOD_INFO ByRef, UInt32, IntPtr ByRef, UInt32 ByRef) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2879277Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(ILCompiler.DependencyAnalysis.IMethodNode, Internal.IL.MethodIL) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2889940Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at Internal.JitInterface.CorInfoImpl.CompileMethod(ILCompiler.DependencyAnalysis.ReadyToRun.MethodWithGCInfo, ILCompiler.Logger) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2900358Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<ComputeDependencyNodeDependencies>g__CompileOneMethod|38_3(ILCompiler.DependencyAnalysisFramework.DependencyNodeCore`1<ILCompiler.DependencyAnalysis.NodeFactory>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2912736Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.Parallel+<>c__DisplayClass19_0`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<ForWorker>b__1(System.Threading.Tasks.RangeWorker ByRef, Int32, Boolean ByRef) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2922837Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.TaskReplicator+Replica.Execute() [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2930177Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2939939Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2949489Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(System.Threading.Tasks.Task, Boolean) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2958655Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.TaskScheduler.TryRunInline(System.Threading.Tasks.Task, Boolean) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2965741Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.Task.InternalRunSynchronously(System.Threading.Tasks.TaskScheduler, Boolean) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2969513Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.TaskReplicator.Run[[System.Threading.Tasks.RangeWorker, System.Threading.Tasks.Parallel, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]](ReplicatableUserAction`1<System.Threading.Tasks.RangeWorker>, System.Threading.Tasks.ParallelOptions, Boolean) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2975196Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.Parallel.ForWorker[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Int32, Int32, System.Threading.Tasks.ParallelOptions, System.Action`1<Int32>, System.Action`2<Int32,System.Threading.Tasks.ParallelLoopState>, System.Func`4<Int32,System.Threading.Tasks.ParallelLoopState,System.__Canon,System.__Canon>, System.Func`1<System.__Canon>, System.Action`1<System.__Canon>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2983142Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.Parallel.ForEachWorker[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Threading.Tasks.ParallelOptions, System.Action`1<System.__Canon>, System.Action`2<System.__Canon,System.Threading.Tasks.ParallelLoopState>, System.Action`3<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64>, System.Func`4<System.__Canon,System.Threading.Tasks.ParallelLoopState,System.__Canon,System.__Canon>, System.Func`5<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64,System.__Canon,System.__Canon>, System.Func`1<System.__Canon>, System.Action`1<System.__Canon>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.2996591Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at System.Threading.Tasks.Parallel.ForEach[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Threading.Tasks.ParallelOptions, System.Action`1<System.__Canon>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3000898Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.<ComputeDependencyNodeDependencies>g__CompileMethodList|38_2(System.Collections.Generic.IEnumerable`1<ILCompiler.DependencyAnalysisFramework.DependencyNodeCore`1<ILCompiler.DependencyAnalysis.NodeFactory>>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3006343Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysisFramework.DependencyNodeCore`1<ILCompiler.DependencyAnalysis.NodeFactory>>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3011732Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2[[ILCompiler.DependencyAnalysisFramework.NoLogStrategy`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ILCompiler.DependencyAnalysisFramework, Version=7.0.0.0, Culture=neutral, PublicKeyToken=null],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ComputeDependencies(System.Collections.Generic.List`1<ILCompiler.DependencyAnalysisFramework.DependencyNodeCore`1<System.__Canon>>) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3023280Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2[[ILCompiler.DependencyAnalysisFramework.NoLogStrategy`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], ILCompiler.DependencyAnalysisFramework, Version=7.0.0.0, Culture=neutral, PublicKeyToken=null],[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ComputeMarkedNodes() [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3037817Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.ReadyToRunCodegenCompilation.Compile(System.String) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3050126Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.Program.RunSingleCompilation(System.Collections.Generic.Dictionary`2<System.String,System.String>, ILCompiler.InstructionSetSupport, System.String, System.Collections.Generic.Dictionary`2<System.String,System.String>, System.Collections.Generic.HashSet`1<Internal.TypeSystem.ModuleDesc>, ILCompiler.CompilerTypeSystemContext) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3059760Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.Program.Run(System.String[]) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3069399Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(463,5): error :    at ILCompiler.Program.Main(System.String[]) [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]
2022-09-09T19:32:05.3078397Z D:\a\_work\1\s\artifacts\bin\Crossgen2Tasks\Debug\net7.0\Microsoft.NET.CrossGen.targets(351,5): error NETSDK1096: Optimizing assemblies for performance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false. [D:\a\_work\1\s\src\installer\pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Runtime.sfxproj]

eng/versioning.targets Outdated Show resolved Hide resolved
src/coreclr/runtime-prereqs.proj Outdated Show resolved Hide resolved
src/native/corehost/corehost.proj Outdated Show resolved Hide resolved
mmitche and others added 4 commits September 12, 2022 07:49
@mmitche
Copy link
Member Author

mmitche commented Sep 12, 2022

Feedback applied. Thanks @hoyosjs

eng/SourceBuild.props Outdated Show resolved Hide resolved
eng/SourceBuild.props Outdated Show resolved Hide resolved
mmitche and others added 3 commits September 12, 2022 09:27
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
@mmitche
Copy link
Member Author

mmitche commented Sep 12, 2022

@mmitche
Copy link
Member Author

mmitche commented Sep 13, 2022

Alright, looking solid now i official build.

@mmitche mmitche merged commit b1edd42 into dotnet:main Sep 13, 2022
@mmitche mmitche deleted the suppply-obid branch September 13, 2022 15:55
@ghost ghost locked as resolved and limited conversation to collaborators Oct 13, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Packages inside source build intermediate packages have improper nupkg branding
7 participants