Support building Microsoft and Source-Build from a single branch

[MichaelSimons]

There is a shared desire to build both the Microsoft flavor of .NET as well as all of the source-build distro versions from the same source (one repo/commit/tag). There will be different build configurations/variations but the builds should be capable of running on the same source.

Related to:
#4045
#4010

Proposal:

1. Build/ship the MSFT and SB products from a single branch/source
2. Allow the current set of OSS licensed binaries within the product but don't allow additional binaries by default. An explicit action would be needed to get new binaries into the VMR. A script will be added to the VMR to strip binaries for source-only/source-build scenarios.
3. Non-OSS licensed code and binaries will not be permitted within the VMR.

Work Breakdown:

1. Capture the VMR's permissible source policy in documentation #4086
2. Remove binary cloaks from the VMR's main branch #4087
3. Provide a script to remove all checked binaries from your local environment #4089
4. Remove/address all non-OSS licensed code and binaries within the VMR
   1. winforms - Winforms repo bringing in a non-open source license winforms#10779
   2. Application-Insights - Update Application-Insights dependency to 2.22 sdk#38513
5. Delete the main-ub branch #4090
6. Promote the source-build's license scan pipeline to be a general VMR pipeline #4091
7. Rethink Binary Scanner Implementation #4108
8. Remove binary scanner in arcade-services arcade-services#3453

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ellahathaway]

Allow the current set of OSS licensed binaries within the product but don't allow additional binaries by default.

@MichaelSimons Can you clarify this? Initially, I took this to mean that we have a current set of OSS licensed binaries (assuming that this exists as an explicit list somewhere... is it used with the license scanning?), and that any additional binaries, including new OSS licensed binaries, will be disallowed. However, in #4086, you say "OSS licensed binaries are allowed but require an explicit action to get new binaries into the VMR". I take this to mean that the current list of OSS licensed binaries are allowed, and any new OSS licensed binaries will only be allowed upon some action/verification that is added as part of the changes for this issue.

What is the intended behavior?

[MichaelSimons]

What is the intended behavior?

Sure, binaries are excluded currently via the source-mapping exclude/cloaking mechanism. See https://github.com/dotnet/installer/blob/main/src/VirtualMonoRepo/source-mappings.json#L38. We want to allow/add the current binaries. Future additions require an explicit action.

There are two mechanism in play here

1. The source-mappings controls what is sync'd into the VMR
2. The VMR scanner flags disallowed binaries as defined in https://github.com/dotnet/installer/blob/main/src/VirtualMonoRepo/allowed-binaries.txt.

There are a few ways you could approach this. For example, you could modify the source-mappings to always allow binaries and then modify the binary scanner to only allow the current set. When a new binary is added the binary scanner would fail and the user would have to decide to allow it or cloak it. If you go with this approach, the binary scanner would have to be modified to fail instead of warn like it does today. Alternatively you could modify the source-mappings to only allow the current binaries and cloak any new ones. If a developer needs a new one they would have to add an include in the mapping plus add it as an allowed binary in the binary scanner. The downside of this approach is that it requires two changes by developers, to address this maybe there is a way to have a shared definition of allowed binaries.

[jkotas]

Allow the current set of OSS licensed binaries within the product but don't allow additional binaries by default.

dotnet/installer#18563 implemented this in a very strict way. I do not think that a strict enforcement of this rule makes sense.

For example, some repos have tests that use binary files. I do not think it makes sense to require explicit action in the VMR when a new test and a new binary file is added following the established structure in the repo. The whole directory these tests should get a free pass.

[MichaelSimons]

All planned work was completed. Closing.