Add support for multiple MSVC platform toolsets for a single Visual Studio installation

[Pieter-Dewachter]

Visual Studio versions starting from 2017 support multiple "platform toolsets" for a single Visual Studio installation. SCons seems to only support the matching toolset version for the currently installed Visual Studio version (for example vc143 when Visual Studio 2022 is installed). The vc140 toolset is a notable exception since it can be detected with the older method of reading registry entries (as used by SCons already). Basically, before this pull requests the following combinations would not work correctly (without adding MSVC_USE_SCRIPT and MSVC_USE_SCRIPT_ARGS to the environment):

• Visual Studio 2019: vc141 compiler
• Visual Studio 2022: vc141 and vc142 compilers

I have tested my (very small) changes on a clean Windows 11 installation (in a virtual machine) with all of the following configurations without any problems:

• Visual Studio 2015: vc140 compiler
• Visual Studio 2017: vc140 and vc141 compilers
• Visual Studio 2019: vc140, vc141 and vc142 compilers
• Visual Studio 2022: vc140, vc141, vc142 and vc143 compilers

A small note I have to make: the warning message differs a little bit when you don't have a Visual Studio version installed at all versus when only the specific toolset is not available, although I believe the messages make sense in a way. Two example situations with their corresponding warning that SCons throws:

• You want to compile with vc142 but you only have Visual Studio 2017 installed:
  scons: warning: VC version 14.2 not installed. C/C++ compilers are most likely not set correctly.
• You want to compile with vc142 and have Visual Studio 2022 installed, but not the platform toolset for vc142:
  scons: warning: Could not find MSVC compiler 'cl'. It may need to be installed separately with Visual Studio.

Contributor Checklist:

• I have created a new test or updated the unit tests to cover the new/changed functionality.
• I have updated CHANGES.txt (and read the README.rst)
• I have updated the appropriate documentation

[mwichmann]

Waiting for the experts to weigh in (@jcbrill ?) but it seems like looking for more-than-one-VC as those patterns do could cause confusion about what you actually get.

[jcbrill]

...but it seems like looking for more-than-one-VC as those patterns do could cause confusion about what you actually get.

Quite right.

Some (really) quick thoughts...

There is a bit of a terminology problem involved. The proposed implementation is really the "build tools" rather than a "toolset". The build tools are usually referred to as something like "v143" while a fully specified toolset is referred to as "14.32.31326".

The -vcvars_ver argument to the msvc batch files can be a full toolset version or a partial prefix (MAJOR.MINOR) with matching based on an orderted directory sort of the toolset folders. As the implementation only provides the first digit of the minor version number, only the "latest" build tools can be selected (i.e., it is not possible to specify an "older" build tools or a full toolset version).

The order of the results from a vswhere query are "unpredictable". At one point, I believed that the order returned may have been in reverse order of installation (LIFO). I'm not sure what happens when the vswhere version range is expanded as then there are multiple installations that could be the first returned. This could potentially be exacerbated by the implementation of preview versions if there is a "don't care" option (i.e., return preview or release).

There is no guarantee that the first installation returned by the vswhere query supports the desired build tools, and by extension, toolset. For example, when requesting build tools v141 with vs2022 and vs2017 installed, there is no requirement that build tools v141 are installed in vs2022. I believe it is possible that the first instance returned by vswhere could be vs2022. Without a v141 toolset, the build will likely fail due to a msvc script error. At present, this becomes difficult to diagnose as the msvc batch output, while written to stdout, is effectively unavailable via SCons.

Even if the first selected installation supports the build tools selected, a user may have a reason for wanting a specific installation of msvs due to ancillary tools that could be installed. In the example above, if the v141 build tools are installed in both vs2022 and vs2017, the product returned by vswhere may not necessarily be the product the user wants to use. VS2022 could be returned but there may be other reasons the user wants to use VS2017.

Another potential problem is that the selected product could change based on the products installed and/or removed over time. Depending on the toolsets installed this could change the toolset version. Whether this matters or not probably can't be determined a priori. Although to be fair, this happens now with msvs updates.

To date, while SCons uses MSVC_VERSION it is slightly a misnomer as it is really the visual studio product that may contain multiple build tools/toolsets. I'm not sure "overloading" the msvc product and the build tools is a good idea given the existing user base. An end-user cannot necessarily control the product/build tools combination by using a single environment variable (MSVC_VERSION).

#4106 discusses three msvc script enhancements:

• MSVC_USE_SCRIPT_ARGS which has been implemented
• MSVC_USE_ARGS not implemented yet but is one solution to the problem at hand
• MSVC_USE_SETTINGS which is complete and pending adoption

In the short term, I was planning on implementing MSVC_USE_ARGS which would not only allow specifying by build tools but a full toolset as well. This has the advantage that the user "product" request is unchanged (MSVC_VERSION='14.2') and the build tools/toolset request is a separate specification (MSVC_USE_ARGS='-vcvers_var=14.16').

For example:

env = Environment(MSVC_VERSION='14.2', MSVC_USE_ARGS='-vcvers_var=14.16')
env = Environment(MSVC_VERSION='14.3', MSVC_USE_ARGS=['-vcvers_var=14.29.30133'])

This would not change the selection behavior for existing users.

Implementation of MSVC_USE_ARGS is the last of the three "escape hatch" mechanisms which would allow all features of the msvc batch files to be available to an end user. Until first-class support for build tools and/or toolsets can be added, this provides a mechanism to achieve the desired outcome and is useful in other contexts (e.g., specification of SDK version, enable spectre libraries, etc.).

This might also close a couple of outstanding issues on the msvc bug scrub list as well.

As someone who has implemented toolset support inside SCons once and outside of SCons twice, first-class support for toolsets in SCons is possible but requires a non-trivial expansion of data structures and logic.

Feel free to point out any factual errors and errors of omission.

As always, I could be wrong...

[jcbrill]

P.S.: To date, it has been my experience that an implementation that takes a user version request (e.g., MSVC_VERSION`='14.1') and returns a different product (e.g., MSVC_VERSION='14.3') is a non-starter.

[Pieter-Dewachter]

You are definitely right about the potential problem that arises when a user has installed both Visual Studio 2017 and Visual Studio 2022 (without the build tools for v141). The naming thing is also quite strange to me, it seems like even Micrsoft doesn't really know what means what, since the Visual Studio installer clearly speaks of "platform toolsets" rather than "build tools" for these options.

The thing I wanted to accomplish was that a user with a simple installation of Visual Studio and multiple build tools, can actually compile projects made for all of the installed build tools. I was very confused when it happened to me that I couldn't compile v142 projects using SCons anymore after upgrading from Visual Studio 2019 to Visual Studio 2022, especially since I could still compile v140 projects without any issues.

Your issue is definitely something that would need fixing before this PR could ever be merged, but I also believe that perhaps the "edge-case" handling of the automatic detection shouldn't go too far. In my mind, it should "just work" and for 99% of the cases you just want either a v140, v141, v142 or v142 build without actually caring too much about the subversions being used. How about something like this:

• By default, MSVC_VERSION searches for the correct Visual Studio version first to see if it exists.
• Afterwards, it can search for higher Visual Studio versions to try and see if they have the necessary build tools.

That way, there is a system that "just works" for the regular user and does the job 99% of the time exactly right. MSVC_VERSION actually makes sense for this as well in my (humble) opinion, since it's the exact same compiler you can use (subversions exists for all Visual Studio versions, not just if installed as a different build tool version).

If for any reason a user has specific needs for special subversions or something similar, then I agree that the solution with the extra environment entries works very well. However, I believe that's not the majority of people but I could be wrong of course.

[Pieter-Dewachter]

P.S.: To date, it has been my experience that an implementation that takes a user version request (e.g., MSVC_VERSION`='14.1') and returns a different product (e.g., MSVC_VERSION='14.3') is a non-starter.

Correct me if I'm wrong, but we are actually returning the correct compiler version as requested by the user (hence MSVC and not MSVS as in earlier versions of SCons). This compiler is the exact same as the one provided with the "correct" product version, so it makes perfect sense in my mind at least.

Also, this approach already works for the vc140 build tools even in the current SCons version, so it's just an extension of existing functionality / a bugfix if we consider this vc140 thing a feature. I have been using this existing 'feature' for a few months already extensively.

[jcbrill]

The naming thing is also quite strange to me, it seems like even Micrsoft doesn't really know what means what, since the Visual Studio installer clearly speaks of "platform toolsets" rather than "build tools" for these options.

It is confusing.

A given toolset for VS2022 might be 14.31.31103. This is the 14.3/143/v143 platform toolset and/or build tools. New toolsets may increase the second minor digit in addition to the 5 digits at the end. For example, a toolset in the VS2022 Preview version is "14.32.31326". This is still the same platform toolset/build tools: 14.3/143/v143.

In the proposed implementation, only the first 3 digits are used (e.g., "14.3") due to the current restriction on MSVC_VERSION.

Users with many versions of VS installed could experience unpredictable side effects due to the vswhere selection of the first visual studio instance in the returned list.

For example, I have VS2022 Community Preview, VS2022 BuildTools Preview, VS2022 Community, VS2022 BuildTools, VS2019 Community, VS2019 BuildTools, VS2017 Community, VS2017 BuildTools, VS2017 Express, and VS2015 Community installed.

A query for '14.1' will return VS2022, VS2019, or VS2017. I have no idea what order they would be returned and whether or not installation order matters.

Correct me if I'm wrong, but we are actually returning the correct compiler version as requested by the user (hence MSVC and not MSVS as in earlier versions of SCons). This compiler is the exact same as the one provided with the "correct" product version, so it makes perfect sense in my mind at least.

I am not saying you are wrong because I have made the same argument myself :)

The problem is that it implicitly changes the definition of MSVC_VERSION from a "product" (2015 and earlier) to a "build tool/toolset" via a single variable. It is still possible to use MSVS_VERSION instead of MSVC_VERSION (a deprecated warning is issued).

Despite being deprecated, there remains a notion that MSVS_VERSION is equivalent to MSVC_VERSION. In fact, MSVS_VERSION is set to MSVC_VERSION which would be incorrect with a build tools/toolset interpretation.

It is possible to achieve the end result of what you are asking for but in a different manner that does not affect existing users at all. Unfortunately, such an implementation requires fairly non-trivial updates to the existing code base.

Also, this approach already works for the vc140 build tools even in the current SCons version, so it's just an extension of existing functionality / a bugfix if we consider this vc140 thing a feature. I have been using this existing 'feature' for a few months already extensively.

vc140 works differently than the rest of the toolsets.

It is a special case which just happens to work in SCons because of the manner in which visual studio is installed. You will notice that the 140 toolsets are not actually in the VS2017/2019/2022 folder with the rest of the toolsets. When the 140 toolset is installed, the 14.0 keys giving the installation location for vs2015 are in the registry. SCons finds the 14.0 (2015) installation via the registry as does msvc. The msvc 2017-2022 batch files find 14.0 via the registry (see MSVSFOLDER\Common7\Tools\vsdevcmd\ext\vcvars\vcvars140.bat).

How about something like this:

By default, MSVC_VERSION searches for the correct Visual Studio version first to see if it exists.
Afterwards, it can search for higher Visual Studio versions to try and see if they have the necessary build tools.

I once proposed exactly what you described. It did not go well :(

This requires building a data structure based on all of the results returned from vswhere and walking the vs installation folder to detect which toolset versions are actually installed. By its very nature, this requires implementing a completely new data structure. However, once implemented, there is an opportunity for enhancements to the existing behavior.

It is likely better to introduce one or more new variable(s) that does not carry any existing baggage by 're-interpreting" existing variables and won't change the behavior for existing users while allowing for enhanced functionality.

I have written a detection implementation outside of SCons that allows me to test ideas and see what works and what doesn't in the context of SCons.

My own work outside of SCons uses a function that returns a dictionary that is used by SCons via either MSVC_USE_SCRIPT/MSVC_USE_SCRIPT_ARGS (SCons runs the batch file) or MSVC_USE_SETTINGS (the detection script runs the batch file). MSVC_USE_SETTINGS has not been adopted yet.

Some examples taken from an older test script (some names may have changed):

env1 = Environment( **scons_msvc(MSVC_VERSION='14.1') )
env2 = Environment( **scons_msvc(MSVC_TOOLSET='14.1') )
env3 = Environment( **scons_msvc(MSVC_BUILDTOOLS='v141') )
env4 = Environment( **scons_msvc(MSVC_TOOLSET='14.31.31103') )
env5 = Environment( **scons_msvc(MSVC_VERSION='14.3', MSVC_TOOLSET='14.1') )
env6 = Environment( **scons_msvc(MSVS_PRODUCT='2022', MSVC_BUILDTOOLS='v141') )
env7 = Environment( **scons_msvc(MSVS_PRODUCT='2022') )

env8 = Environment( **scons_msvc(MSVS_PRODUCT='2022', MSVS_COMPONENT='BuildTools', MSVS_CHANNEL='Preview',  MSVC_TOOLSET='14.1', TARGET_ARCH='amd64') )
env9 = Environment( **scons_msvc(MSVC_RUNTIME='ucrt', MSVC_ORDER='Toolset', TARGET_ARCH='x64', TARGET_ARCH_CONSTRAINT=['x86', 'x64']) )

The first two examples demonstrate the difference without affecting existing users.

env1 is effectively the current scons implementation and returns VS2017.

env2 is similar in spirit to what you are proposing and returns the "newest" v141 toolset (i.e., the greatest 14.1X.NNNNN version number) across all VS implementations that have v141 installed ranked in order of toolset version and within the same toolset version by product order from newest (VS2022) to oldest (VS2015).

env2 is explicit that the product/version is not important when there is only a request for a build tools/toolset version.

Unfortunately, validation and consistency checks become much more involved as more variables are introduced.

[bdbaddog]

basically.. what @jcbrill said.

If we make changes in what MSVC_VERSION means, we need to go through a deprecation cycle (read PITA).
So given the above MSVC_TOOLSET does seem like the simplest to deploy quickly without breaking any users builds.
In addition to adding MSVC_USE_ARGS.

[Pieter-Dewachter]

It seems like MSVC_TOOLSET would indeed be a clean way of implementing what I (and probably others) want, without breaking any "exotic" configurations. The other variables are also really nice to have, although with much more limited usecases I would presume. A big notice on the docs would be needed as well of course, so users with compilation problems can easily see the solution (in my mind it still makes sense that the same compiler always works, no matter how you installed it exactly).

Some edge cases need to be defined, probably:

• When providing an older toolset, we should probably use the old methods as fallback even for MSVC_TOOLSET (anything <= 14.0, for example 9.0Exp). Or does the Exp addition not make sense here? Or do we define that it's only applicable for versions larger than 14.0 / starting with 14.1?
• There will have to be a deterministic way in which SCons detects the toolsets, meaning it will have to sort the results given by vswhere. Not only based on version number of the product, but also the edition (Community, Pro, ...). The JSON output format would probably be the easiest route to go here.
• When not providing a full version (including the last five digits), the behavior will have to be defined as well. I think there are two options that make most sense I think:
  A) Use the highest available version, not matter with which product version it's installed (if no product version is specified, that is).
  B) Use the lowest compatible product version that has a suitable compiler installed.

When implementing MSVC_TOOLSET it probably makes sense to keep the other new suggested variables already in mind so we don't push ourselves in another corner. Does there already exist a (partial) implementation for something like this? Just to make sure nobody is doing the same thing independent of eachother. This is also the first time diving into the code of SCons itself (I worked with SCons a lot as a user, just not in its own source code), perhaps I am not the best suited for something like this? I can definitely give it a try if wanted, just want to confirm with the experts here if it's feasible or not.

[jcbrill]

Some food for thought..

Terminology used later:

• MSVS_PRODUCT - Visual Studio major release (e.g., 2022)
• MSVS_CHANNEL - Release vs Preview
• MSVS_COMPONENT - Enterprise, Community, Express, etc.
• MSVC_BUILDTOOLS - A set of one or more toolsets that is introduced in each product release (e.g, v143, 14.3)
• MSVC_TOOLSET - A set of one or more tools with a specific version 14.16.27023
• msvc tool - a combination of toolset version, tool architecture, and target architecture

VS2017, and newer, are effectively containers for multiple build tools and by extension multiple toolsets. Each toolset has multiple tools.

VS2015, and earlier, effectively have one toolset with multiple tools (amd64, x86_amd64, x86_arm, etc).

It seems like MSVC_TOOLSET would indeed be a clean way of implementing what I (and probably others) want, without breaking any "exotic" configurations. The other variables are also really nice to have, although with much more limited usecases I would presume. A big notice on the docs would be needed as well of course, so users with compilation problems can easily see the solution (in my mind it still makes sense that the same compiler always works, no matter how you installed it exactly).

I'm not sure that I agree. For example, on my development box, toolset version '14.16.27023' is available in 9 installed instances:

            "MSVCToolset(v141, 14.16.27023, 0, 2022, Rel, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Pre, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Pre, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2019, Rel, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2019, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 1, 2017, Rel, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 1, 2017, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 1, 2017, Rel, Exp, 1)",

There is no guarantee that each toolset has the desired target architecture tool installed. For example, maybe the arm64 tools are only installed in VS2019 which is the middle of the list.

There will be users that want the ability to specify exactly which product/channel/component/toolset is used possibly due to other "features" that are installed and/or corporate policy.

In addition to the compiler proper, there are other things placed in the environment path based on VS installation options.

Even compiler features (e.g., MFC/ATL, spectre libraries, etc) are based on a user's install preference and may not necessary be installed in every version of Visual Studio let alone by toolset.

For those of us that have multiple versions installed, it is handy to able to target a specific product/channel/component/toolset for testing compatibility without having to setup a virtual machine and installing Visual Studio just because the desired combination is not directly selectable.

It can be demonstrated that if the internal data structures provide the ability to perform selection of an individual msvc tool then any other functionality can be provided via additional ranked search lists.

A visual studio instance can be uniquely identified by

• Product (e.g., 2022, 2019)
• Component (e.g., Enterprise, Community, BuildTools, Express)
• Channel (e.g., Release, Preview)

Aside: In my own work, there is a fourth "attribute" (Sequence Number) that applies to VS2015 and earlier and would just muddy the discussion.

An msvc tool can be uniquely identified by:

• Product
• Component
• Channel
• Toolset version number from which a build tools version can be derived.
• "Native" tool architecture (e.g., amd64, x86)
• target architecture (e.g., amd64, x86, arm64, arm)

Some edge cases need to be defined, probably:

• When providing an older toolset, we should probably use the old methods as fallback even for MSVC_TOOLSET (anything <= 14.0, for example 9.0Exp). Or does the Exp addition not make sense here? Or do we define that it's only applicable for versions larger than 14.0 / starting with 14.1?

VS2015 and earlier can be handled in the same way as modern toolsets simply by expanding the version number to a toolset version.

            "MSVCToolset(v143, 14.33.31424, 1, 2022, Pre, Com, 1)",
            "MSVCToolset(v143, 14.33.31424, 1, 2022, Pre, BT, 1)",
            "MSVCToolset(v143, 14.32.31326, 1, 2022, Rel, Com, 1)",
            "MSVCToolset(v143, 14.32.31326, 1, 2022, Rel, BT, 1)",
            "MSVCToolset(v143, 14.31.31103, 0, 2022, Pre, Com, 1)",
            "MSVCToolset(v143, 14.31.31103, 0, 2022, Pre, BT, 1)",
            "MSVCToolset(v142, 14.29.30133, 0, 2022, Rel, Com, 1)",
            "MSVCToolset(v142, 14.29.30133, 0, 2022, Rel, BT, 1)",
            "MSVCToolset(v142, 14.29.30133, 0, 2022, Pre, Com, 1)",
            "MSVCToolset(v142, 14.29.30133, 0, 2022, Pre, BT, 1)",
            "MSVCToolset(v142, 14.29.30133, 1, 2019, Rel, Com, 1)",
            "MSVCToolset(v142, 14.29.30133, 1, 2019, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Rel, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Pre, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2022, Pre, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2019, Rel, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 0, 2019, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 1, 2017, Rel, Com, 1)",
            "MSVCToolset(v141, 14.16.27023, 1, 2017, Rel, BT, 1)",
            "MSVCToolset(v141, 14.16.27023, 1, 2017, Rel, Exp, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2022, Rel, Com, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2022, Rel, BT, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2022, Pre, Com, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2022, Pre, BT, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2019, Rel, Com, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2019, Rel, BT, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2017, Rel, Com, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2017, Rel, BT, 1)",
            "MSVCToolset(v140, 14.00.00000, 0, 2017, Rel, Exp, 1)",
            "MSVCToolset(v140, 14.00.00000, 1, 2015, Rel, Dev, 1)",
            "MSVCToolset(v120, 12.00.00000, 1, 2013, Rel, Dev, 1)",
            "MSVCToolset(v110, 11.00.00000, 1, 2012, Rel, Exp, 1)",
            "MSVCToolset(v100, 10.00.00000, 1, 2010, Rel, Exp, 1)",
            "MSVCToolset(v90, 9.00.00000, 1, 2008, Rel, Dev, 1)",
            "MSVCToolset(v90, 9.00.00000, 1, 2008, Rel, Py, 1)",
            "MSVCToolset(v80, 8.00.00000, 1, 2005, Rel, Dev, 1)",
            "MSVCToolset(v71, 7.10.00000, 1, 2003, Rel, Dev, 1)",
            "MSVCToolset(v70, 7.00.00000, 1, 2002, Rel, Dev, 1)",
            "MSVCToolset(v60, 6.00.00000, 1, 1998, Rel, Dev, 1)"

• There will have to be a deterministic way in which SCons detects the toolsets, meaning it will have to sort the results given by vswhere. Not only based on version number of the product, but also the edition (Community, Pro, ...). The JSON output format would probably be the easiest route to go here.

A single vswhere query with json output is the way to go. The prototype in the branch to support preview versions does this. The visual studio workloads/components are available in the vswhere output.

A ranking hierarchy is easy enough.

Dealing with VS2015, and earlier, is a bit trickier. One approach is to reduce the components to a limited set of possibilities: 'Dev' (devenv.com exists), 'Exp' which can be determined directly from the registry, 'Cmd' command-line tools (devenv.com does not exist nor does wdexpress.com/vcexpress.com), 'Py' for Visual C++ for Python.

For example:

    for rank, kind, method, internal_id, comp_id in (

        # vswhere components
        (450, 'vswhere', 'vswhere', 'Enterprise',   'Enterprise'),
        (440, 'vswhere', 'vswhere', 'Professional', 'Professional'),
        (430, 'vswhere', 'vswhere', 'Community',    'Community'),
        (420, 'vswhere', 'vswhere', 'BuildTools',   'BuildTools'),
        (410, 'vswhere', 'vswhere', 'WDExpress',    'Express'),

        # registry components
        (350, 'registry', 'registry', 'Develop',     'Develop'),
        (340, 'registry', 'registry', 'VCForPython', 'Python'),
        (330, 'registry', 'registry', 'CommandLine', 'CmdLine'),
        (320, 'registry', 'registry', 'Express',     'Express'),
        (310, 'registry', 'registry', 'SDK',         'SDK'),

        # registry temporary vc components
        (203, 'regtemp', 'registry', 'VCExpress', 'VCExpress'),
        (202, 'regtemp', 'registry', 'VCSetup',   'VCSetup'),
        (201, 'regtemp', 'registry', 'VCSxS',     'VCSxS'),

        # registry temporary vs components
        (103, 'regtemp', 'registry', 'VSExpress', 'VSExpress'),
        (102, 'regtemp', 'registry', 'VSSetup',   'VSSetup'),
        (101, 'regtemp', 'registry', 'VSSxS',     'VSSxS'),

    ):
        ...

• When not providing a full version (including the last five digits), the behavior will have to be defined as well. I think there are two options that make most sense I think:
  A) Use the highest available version, not matter with which product version it's installed (if no product version is specified, that is).

There is no guarantee that the selected toolset supports the necessary target architectures for a build.

Without the ability to uniquely select an msvc tool. There will be users that are unhappy with the policy.

B) Use the lowest compatible product version that has a suitable compiler installed.

These options are not mutually exlusive if the underlying data structure can be queried in different ways.

When implementing MSVC_TOOLSET it probably makes sense to keep the other new suggested variables already in mind so we don't push ourselves in another corner. Does there already exist a (partial) implementation for something like this? Just to make sure nobody is doing the same thing independent of eachother. This is also the first time diving into the code of SCons itself (I worked with SCons a lot as a user, just not in its own source code), perhaps I am not the best suited for something like this? I can definitely give it a try if wanted, just want to confirm with the experts here if it's feasible or not.

Inside of SCons, there was an abandoned effort that implemented the two-stage query you suggested earlier. It is woefully out of date, as the SCons internals have been under renovation. I keep it around as there was the occasional good idea.

Repository:
https://github.com/jcbrill/scons/tree/jbrill-msvc-specific-version

Relevant code fragments:
https://github.com/jcbrill/scons/blob/jbrill-msvc-specific-version/SCons/Tool/MSCommon/vc.py#L1288-L2367

Outside of SCons, there exists a script that does everything you are asking for and more. I have finally uploaded it to github but have not made the repository public yet.

There is a non-trivial amount of code that is not enabled and a ton of comments taken directly from the msvc batch files that probably should be excised.

The script can be used with SCons by placing it in a site_scons folder. Even simpler, the functionality of everything discussed above can be tested from a pure python script (short of actually compiling anything).

I will provide a link to the implementation soon.

I am curious if it provides everything you want (with the obvious exception of being native SCons). Although, with an appropriate adapter/bridge it probably could be used in one's own hacked version of SCons.

The Sconstruct fragment below is used to exercise SCons:

• 244 toolsets are run though SCons using MSVC_USE_SCRIPT and MSVC_USE_SCRIPT_ARGS
• 244 toolsets are run through SCons using MSVC_USE_SETTINGS
• 48 toolsets are run through SCons using the SCons detection

Manual inspection of the cache file confirmed that the toolsets are being correctly individually selected.

SConstruct fragment for stress testing SCons:

# SCons compatible detection

seen_product = set()
seen_edition = set()

def is_scons_compatible(msvc_tool):

    if not msvc_tool.msvc_toolset.default_toolset:
        # ignore non-default toolsets
        return False

    if msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_channel_suffix != 'Rel':
        # ignore preview versions
        return False

    if msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_component_suffix == 'Exp':
        # accept all Express versions
        return True

    if msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_product not in seen_product:
        seen_product.add(msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_product)
        seen_edition.add(msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_edition_specific)
        return True

    if msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_edition_specific in seen_edition:
        return True

    return False

msvc_buildtools_skip = {
    ('v120', 'arm'): True, # fatal error C1189: #error :  Compiling Desktop applications for the ARM platform is not supported.
    ('v110', 'arm'): True, # fatal error C1189: #error :  Compiling Desktop applications for the ARM platform is not supported.
}

msvc_buildtools_scriptargs = {
    ('v141', 'arm64'): '10.0.20348.0', # 10.0.22000.0 arm64 library bug
}

scons_detection = True
vsdetect_active = True

vsdetect_run_script_list = [False, True]

msvs_manager = vsdetect._VSDetectInstalled.manager_all()

msvc_tools = msvs_manager.msvc_installed.msvc_tools

for msvc_tool in msvc_tools:

    key = (msvc_tool.msvc_toolset.buildtools_version, msvc_tool.target_arch)

    if key in msvc_buildtools_skip:
        print("***SKIP TOOL***", key, msvc_tool.id)
        continue

    msvc_script_args = msvc_buildtools_scriptargs.get(key, None)

    if scons_detection:

        if not is_scons_compatible(msvc_tool):

            pass

        elif msvc_script_args:

            # MSVC_SCRIPT_ARGS not implemented yet in SCons
            print("***SKIP MSVC_SCRIPT_ARGS***", msvc_script_args, key, msvc_tool.id)
        
        else:
         
            suffix = 'Exp' if msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_component_suffix == 'Exp' else ''

            make_environment(
                MSVC_VERSION = msvc_tool.msvc_toolset.vc_version + suffix,
                TARGET_ARCH = msvc_tool.target_arch,
                HOST_ARCH = msvc_tool.native_arch,
            )

    if vsdetect_active:

        if msvc_script_args:
            print("***MSVC_SCRIPT_ARGS***", msvc_script_args, key, msvc_tool.id) 

        for vsdetect_run_script in vsdetect_run_script_list:

            d = vsdetect.scons_msvc(

                # Instance: (MSVS_PRODUCT, MSVS_CHANNEL, MSVS_COMPONENT, MSVS_SEQUENCENBR)
                MSVS_PRODUCT = msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_product,
                MSVS_CHANNEL = msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_channel_id,
                MSVS_COMPONENT = msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_component_id,
                MSVS_SEQUENCENBR = msvc_tool.msvc_toolset.msvc_instance.msvs_base.vs_seqnbr,

                # Tool: (MSVC_TOOLSET, TARGET_ARCH, TOOL_ARCH)
                MSVC_TOOLSET = msvc_tool.msvc_toolset.toolset_version,
                TARGET_ARCH = msvc_tool.target_arch,
                TOOL_ARCH = msvc_tool.native_arch,

                # Additional arguments
                MSVC_SCRIPT_ARGS = msvc_script_args,

                # Run script:
                #     False - MSVC_USE_SCRIPT and MSVC_USE_SCRIPT_ARGS
                #     True  - MSVC_USE_SETTINGS
                VSDETECT_RUN_SCRIPT = vsdetect_run_script,

            )

            make_environment(**d)

I am not trying to discourage your efforts. I'm just on about the third or fourth iteration of where you are starting and just trying to hopefully point out some of the things you are bound to run into down the road.

[bdbaddog]

Closing in favor of #4174