[bug] Conan 2.0 fails to find Visual Studio Build Tools using the 'MSBuildToolchain' #13281

rainman110 · 2023-03-01T09:07:50Z

Environment details

Operating System+version: Windows 10
Compiler+version: msvc 2019 build tools
Conan version: 2.0.0
Python version: 3.9

Steps to reproduce

Install the msvc 2019 build tools (a.k.a Visual Studio 16)
In a recipe, use the generator 'MSBuildToolchain'
and build

It fails finding the Visual Studio 16 Build Tools installation with an error:

ERROR: Error in generator 'MSBuildToolchain': VS non-existing installation: Visual Studio 16

Note: I already tried to manually activate vcvars, but conan still does not find the installation.

Logs

======== Finalizing install (deploy, generators) ========
conanfile.py (qt/5.15.8): Writing generators to C:\src\qt-conan\5.x.x
conanfile.py (qt/5.15.8): Generator 'MSBuildToolchain' calling 'generate()'
conanfile.py (qt/5.15.8): MSBuildToolchain created conantoolchain_release_x64.props
conanfile.py (qt/5.15.8): MSBuildToolchain writing conantoolchain.props
conanfile.py (qt/5.15.8): ERROR: Traceback (most recent call last):
  File "c:\local\miniforge3\lib\site-packages\conans\client\generators\__init__.py", line 70, in write_generators
    generator.generate()
  File "c:\local\miniforge3\lib\site-packages\conan\tools\microsoft\toolchain.py", line 104, in generate
    VCVars(self._conanfile).generate()
  File "c:\local\miniforge3\lib\site-packages\conan\tools\microsoft\visual.py", line 139, in generate
    vcvars = vcvars_command(vs_version, architecture=vcvarsarch, platform_type=None,
  File "c:\local\miniforge3\lib\site-packages\conan\tools\microsoft\visual.py", line 216, in vcvars_command
    cmd.append('call "%s" ' % _vcvars_path(version, vs_install_path))
  File "c:\local\miniforge3\lib\site-packages\conan\tools\microsoft\visual.py", line 232, in _vcvars_path
    raise ConanException("VS non-existing installation: Visual Studio %s" % version)
conans.errors.ConanException: VS non-existing installation: Visual Studio 16

ERROR: Error in generator 'MSBuildToolchain': VS non-existing installation: Visual Studio 16

The text was updated successfully, but these errors were encountered:

rainman110 · 2023-03-01T09:10:28Z

Note, in conan 1.x, the recipe used tools.vcvars() to activate the build environment. This worked very well!

As this is not awailable anymore, I tried to use the MSBuildToolchain and NMakeToolchain, both with the error described above.

memsharded · 2023-03-01T09:31:59Z

Hi @rainman110

Thanks for reporting this, this is really unexpected, and not sure why it is not working, the underlying code that does the VS detection based on vswhere is the same as in 1.X.

Could you please try to define in your profile tools.microsoft.msbuild:installation_path to point directly to your VS 2019 installation, to see what happens? (that is an override of the automatic vswhere detection)

rainman110 · 2023-03-01T09:41:38Z

Should this be the path to vcvarsall.bat or where should it point to?

memsharded · 2023-03-01T09:51:42Z

It should be the base installation path, something like:

>>> from conans.client.conf.detect_vs import vs_installation_path as vs
>>> vs("16")
'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community'

rainman110 · 2023-03-01T09:52:45Z

@memsharded I am a bit puzzled. It does not accept this config variable when defining it in the profile.

I need to set it as a config variable.

conan config set does not exists anymore though

rainman110 · 2023-03-01T09:58:40Z

Nevermind, I just had to add it to the [conf] section, not to the [settings] section of the profile.

Now, the error is gone. I will see, wether it works now.

rainman110 · 2023-03-01T10:01:15Z

Just for future reference, the correct settings needs to be:

[conf]
tools.microsoft.msbuild:installation_path=C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools

memsharded · 2023-03-01T10:07:57Z

Do you have installed exclusively the build-tools, but not the full VS IDE?
In that case, it is possible that the auto-detection/vswhere is not prepared for that. It would be great to know a bit more about the setup, how did you installed the BuildTools only. Do you even have vswhere installed?

rainman110 · 2023-03-01T10:08:59Z

Exactly, I only used the "free" buildtools, no IDE is installed. It comes with the compiler, MsBuild, WinSDK etc.

rainman110 · 2023-03-01T10:12:11Z

Is vswhere only called when creating the profile via detect? If so, where is the installation_path actually stored in the profile after creating it?

rainman110 · 2023-03-01T10:17:13Z

@memsharded In fact, vswhere does not list my visual studio build tools installation.

I can live for now with the workaround though!

rainman110 · 2023-03-01T10:20:14Z

However, if I pass "*" to the products arguments, it will find the build tools:

(base) C:\src\qt-conan>"c:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe"  -products * -format json
[
  {
    "instanceId": "4a629174",
    "installDate": "2022-02-10T23:39:11Z",
    "installationName": "VisualStudio/16.11.10+32126.315",
    "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools",
    "installationVersion": "16.11.32126.315",
    "productId": "Microsoft.VisualStudio.Product.BuildTools",
    "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\Tools\\LaunchDevCmd.bat",
    "state": 4294967295,
    "isComplete": true,
    "isLaunchable": true,
    "isPrerelease": false,
    "isRebootRequired": false,
    "displayName": "Visual Studio Build Tools 2019",
    "description": "Die Visual Studio Build Tools ermöglichen Ihnen die Erstellung nativer und verwalteter MSBuild-basierter Anwendungen, ohne dass die Visual Studio-IDE erforderlich ist. Es stehen Optionen zur Installation von Visual C++-Compilern und -Bibliotheken, MFC, ATL sowie C++/CLI-Unterstützung zur Verfügung.",
    "channelId": "VisualStudio.16.Release",
    "channelUri": "https://aka.ms/vs/16/release/channel",
    "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
    "releaseNotes": "https://docs.microsoft.com/en-us/visualstudio/releases/2019/release-notes-v16.11#16.11.10",
    "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660909",
    "updateDate": "2022-02-10T23:39:11.2650007Z",
    "catalog": {
      "buildBranch": "d16.11",
      "buildVersion": "16.11.32126.315",
      "id": "VisualStudio/16.11.10+32126.315",
      "localBuild": "build-lab",
      "manifestName": "VisualStudio",
      "manifestType": "installer",
      "productDisplayVersion": "16.11.10",
      "productLine": "Dev16",
      "productLineVersion": "2019",
      "productMilestone": "RTW",
      "productMilestoneIsPreRelease": "False",
      "productName": "Visual Studio",
      "productPatchVersion": "10",
      "productPreReleaseMilestoneSuffix": "1.0",
      "productSemanticVersion": "16.11.10+32126.315",
      "requiredEngineVersion": "2.11.52.58712"
    },
    "properties": {
      "campaignId": "",
      "channelManifestId": "VisualStudio.16.Release/16.11.10+32126.315",
      "nickname": "",
      "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\setup.exe"
    }
  },
  {
    "instanceId": "8c55b13a",
    "installDate": "2021-10-27T07:14:48Z",
    "installationName": "VisualStudio/15.0.0+26228.98",
    "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools",
    "installationVersion": "15.0.26228.98",
    "productId": "Microsoft.VisualStudio.Product.BuildTools",
    "state": 4294967295,
    "isComplete": true,
    "isLaunchable": true,
    "isPrerelease": false,
    "isRebootRequired": false,
    "displayName": "Visual Studio Build Tools 2017",
    "description": "The Visual Studio Build Tools allows you to build native and managed MSBuild-based applications without requiring the Visual Studio IDE. There are options to install the Visual C++ compilers and libraries, MFC, ATL, and C++/CLI support.",
    "channelId": "VisualStudio.15.Release",
    "channelPath": "C:\\Users\\otte_to-a\\AppData\\Local\\Microsoft\\VisualStudio\\Packages\\_Channels\\4CB340F5\\install_catalog.json",
    "channelUri": "https://aka.ms/vs/15/release/channel",
    "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
    "installChannelUri": "https://aka.ms/vs/15/release/cf7baf3ed_-1205811054/channel",
    "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=659724",
    "updateDate": "2022-01-05T14:30:46.1331746Z",
    "catalog": {
      "buildBranch": "d15rtwsvc",
      "buildVersion": "15.0.26228.98",
      "id": "VisualStudio/15.0.0+26228.98",
      "localBuild": "build-lab",
      "manifestName": "VisualStudio",
      "productDisplayVersion": "15.0 (26228.98)",
      "productLine": "Dev15",
      "productLineVersion": "2017",
      "productMilestone": "RTW",
      "productName": "Visual Studio",
      "productPatchVersion": "0",
      "productRelease": "RTW",
      "productSemanticVersion": "15.0.0+26228.98"
    },
    "properties": {
      "campaignId": "",
      "channelManifestId": "",
      "nickname": "",
      "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
    }
  }
]

memsharded · 2023-03-01T10:22:58Z

That is interesting. We indeed have:

 try:
        legacy_products = vswhere(legacy=True)
        all_products = vswhere(products=["*"])
        products = legacy_products + all_products
    except ConanException:
        products = None

So the product=* is there, maybe the order needs to be reversed?

The vswhere is not called for the profile detect only, but in every vcvars creation.

memsharded · 2023-03-01T10:24:03Z

Could you please point to the MSBuild thing to install only the tools? not evident in the msft site.

rainman110 · 2023-03-01T10:25:52Z

Could you please point to the MSBuild thing to install only the tools? not evident in the msft site.

Sorry, I could not understand what you want me to do 😅

rainman110 · 2023-03-01T10:28:14Z

It seems to raise an exception at line

all_products = vswhere(products=["*"]

so, all_products is not set .

memsharded · 2023-03-01T10:28:28Z

A link to download the MSBuildTools-only installer 😄

rainman110 · 2023-03-01T10:28:58Z

Sure, let me see, whether I still can find it.

rainman110 · 2023-03-01T10:54:18Z

So the old 2019 build tools installer is not provided anymore.

Still, you can install only the build tools 2019 via the 2022 community installer:

rainman110 · 2023-03-01T11:00:19Z

I started debugging the code. It raises an exception at line

all_products = vswhere(products=["*"])

which is

vswhere error: 'utf-8' codec can't decode byte 0x94 in position 717: invalid start byte

Update: The exception comes from the line output = check_output_runner(cmd).strip()

rainman110 · 2023-03-01T11:11:49Z

Okay... The problem is the encoding of the vswhere output, which contains letters such as "ö". See output above.

rainman110 · 2023-03-01T11:12:38Z

On windows, the encoding needs to be "ansi"

rainman110 · 2023-03-01T11:14:24Z

If you want, I'll provide a PR, including a potential fix

The output of vswhere uses "Ansi" encoding. It was though decoded via "Utf-8", which raised an exception in certain cases (i.e. letters like 'ö'). Thus, some visual studio installation could not be found. Fixes conan-io#13281

SpaceIm · 2023-03-01T11:42:59Z

Just to confirm that I have the same problem (french output in my case), Visual Studio detection fails (I have to explicitly set tools.microsoft.msbuild:installation_path):

vswhere output:

Visual Studio Locator version 3.1.1+f4ef329670 [query version 3.5.2144.53294]
Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 811c5e54
installDate: 09/11/2021 00:00:12
installationName: VisualStudio/17.5.0+33414.496
installationPath: C:\Program Files\Microsoft Visual Studio\2022\Community
installationVersion: 17.5.33414.496
productId: Microsoft.VisualStudio.Product.Community
productPath: C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\devenv.exe
state: 4294967295
isComplete: 1
isLaunchable: 1
isPrerelease: 0
isRebootRequired: 0
displayName: Visual Studio Community 2022
description: IDE puissant, gratuit pour les étudiants, les contributeurs open source et les particuliers
channelId: VisualStudio.17.Release
channelUri: https://aka.ms/vs/17/release/channel
enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service
installedChannelId: VisualStudio.17.Release
installedChannelUri: https://aka.ms/vs/17/release/channel
releaseNotes: https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-notes-v17.5#17.5.0
thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=661288
updateDate: 2023-02-27T12:40:09.6046623Z
catalog_buildBranch: d17.5
catalog_buildVersion: 17.5.33414.496
catalog_id: VisualStudio/17.5.0+33414.496
catalog_localBuild: build-lab
catalog_manifestName: VisualStudio
catalog_manifestType: installer
catalog_productDisplayVersion: 17.5.0
catalog_productLine: Dev17
catalog_productLineVersion: 2022
catalog_productMilestone: RTW
catalog_productMilestoneIsPreRelease: False
catalog_productName: Visual Studio
catalog_productPatchVersion: 0
catalog_productPreReleaseMilestoneSuffix: 7.0
catalog_productSemanticVersion: 17.5.0+33414.496
catalog_requiredEngineVersion: 3.5.2145.59678
properties_campaignId: 5
properties_channelManifestId: VisualStudio.17.Release/17.5.0+33414.496
properties_nickname:
properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\setup.exe

The output of vswhere normally uses "Ansi" encoding. It was though decoded via "Utf-8", which raised an exception in certain cases (i.e. letters like 'ö'). This commit forces the output to be utf-8, by providing the "utf8" option to vswhere. Thus, some visual studio installation could not be found. Fixes conan-io#13281

The output of vswhere normally uses "Ansi" encoding. It was though decoded via "Utf-8", which raised an exception in certain cases (i.e. letters like 'ö'). This commit forces the output to be utf-8, by providing the "utf8" option to vswhere. Thus, some visual studio installation could not be found. Fixes #13281

memsharded · 2023-03-01T17:45:59Z

Closed by #13284, for 2.0.1

memsharded self-assigned this Mar 1, 2023

memsharded added stage: triaging bug labels Mar 1, 2023

rainman110 mentioned this issue Mar 1, 2023

Fixed parsing the output of vswhere #13283

Closed

5 tasks

rainman110 mentioned this issue Mar 1, 2023

Fixed parsing the output of vswhere #13284

Merged

5 tasks

memsharded added this to the 2.0.1 milestone Mar 1, 2023

memsharded closed this as completed Mar 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug] Conan 2.0 fails to find Visual Studio Build Tools using the 'MSBuildToolchain' #13281

[bug] Conan 2.0 fails to find Visual Studio Build Tools using the 'MSBuildToolchain' #13281

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023 •

edited

Loading

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

SpaceIm commented Mar 1, 2023

memsharded commented Mar 1, 2023

[bug] Conan 2.0 fails to find Visual Studio Build Tools using the 'MSBuildToolchain' #13281

[bug] Conan 2.0 fails to find Visual Studio Build Tools using the 'MSBuildToolchain' #13281

Comments

rainman110 commented Mar 1, 2023

Environment details

Steps to reproduce

Logs

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023 • edited Loading

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

rainman110 commented Mar 1, 2023

SpaceIm commented Mar 1, 2023

memsharded commented Mar 1, 2023

rainman110 commented Mar 1, 2023 •

edited

Loading