Package visual studio compiler for use with tool_requires

[klausholstjacobsen]

What is your question?

OS: windows 10
conan 1.56

I have packaged the "Visual Studio 2017" toolchain in a package and use it on the consumer side in my build profile [tool_requires].
My consumer I would then like my consumer recipe to detect and use the toolchain provided by conan like this:

    def generate(self):
        tc = CMakeToolchain(self)
        tc.cache_variables["CMAKE_GENERATOR_INSTANCE"] = "[path to toolchain]"
        tc.generate()
        deps = CMakeDeps(self)
        deps.generate()

[path to toolchain] points to "...2017\BuildTools" within the package as indicated by cmake.
But CMakeToolchain refuses to detect anything but the installed Visual Studio in "c:\Program Files....."
Is there a way for me to be able to force the "system" to use the visual studio compiler from my package so that other developers in my team can build without manually installing visual studio.

Regards
Klaus

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @klausholstjacobsen

This is an interesting question. Using VS packaged in a Conan package could be very useful for many users, even ourselves. So far it has been the licensing that didn't allow to do it, but it would be great to know how this can be possible.

For the CMake integration, I am not sure why the CMAKE_GENERATOR_INSTANCE wouldn't work it sounds correct.
It would be great to have the package recipe to test it, do you think that you could share it?

[klausholstjacobsen]

Of course!
I have zipped the visual studio install dir (C:\Program Files (x86)\Microsoft Visual Studio) and put it in artifactory.
The toolchain recipe below is all about fething the toolchain from artifactory and setting up proper paths and conf values.

I have also come across the conf_info parameter "tools.microsoft.msbuild:installation_path", and been trying to work with that, but still no success.

Toolcahin recipe:

from conan import ConanFile, conan_version
from conan.tools.cmake import CMake, CMakeToolchain, CMakeDeps
from conan.tools.files import get, copy, save
from conan.tools.scm import Version
import os

class x86_64_windows_msvc2017Conan(ConanFile):
    name = "x86_64-windows-msvc2017"
    settings = "os", "arch", "build_type"
    description = "Windows toolchain for buildign with Visual Studio 2017 compiler"
    url = "None"
    license = "None"
    author = "The dude"
    topics = None

    def source(self):
        get(self, **self.conan_data["sources"][self.version])

    def package(self):
        copy(self, "*", src=os.path.join(self.source_folder, "Microsoft Visual Studio"), dst=os.path.join(self.package_folder, "x86_64_windows_msvc2017"))

    def package_info(self):
        triplet = "x86_64_windows_msvc2017"
        path = os.path.join(self.package_folder, triplet)


        #generator_instance = os.path.join(path, "2017/BuildTools")
        #self.buildenv_info.define("CMAKE_GENERATOR_INSTANCE", generator_instance)

        install_path = os.path.join(path, "2017/BuildTools")
        self.conf_info.define("tools.microsoft.msbuild:installation_path", install_path)

One strange thing I have discovered is how paths are constructed. If I print install_path in the above recipe when installing the recipe on windows the path becomes a mix of forward and backward slashes. How is the proper OS independent way of constructing paths in recipes?

Regards
Klaus

[memsharded]

I have zipped the visual studio install dir (C:\Program Files (x86)\Microsoft Visual Studio) and put it in artifactory.

Interesting. I heard from some users that it is actually possible to use just the toolchain somehow, without needing the full VS installation, making the package way smaller and faster to install.

How is the proper OS independent way of constructing paths in recipes?

Probably os.path.join(path, "2017", "BuildTools") can help. But this is always a very annoying thing in Windows. We often do some install_path = install_path.replace("/", "\\") (or viceversa) if necessary.

[memsharded]

Sorry this was not followed up, we didn't have time enough to prioritize it.
I am moving this to the Conan 2.X train, to see if there it has better chances, as Conan 2.0 is already 1 year old, so the Conan 1.X is much slower.

[memsharded]

I have succeeded with:

from conan import ConanFile
from conan.tools.files import copy, save
import os

class x86_64_windows_msvc2017Conan(ConanFile):
    name = "msvc"
    version = "16"
    settings = "os", "arch"

    def package(self):
        # toolchain
        version_vs = "16.11.34601.136"  # This must be tuned, might be automatically obtained too
        toolchain = f'set(CMAKE_GENERATOR_INSTANCE "$ENV{{CONAN_MSVC_16_FOLDER}},version={version_vs}" CACHE INTERNAL "")'
        save(self, os.path.join(self.package_folder, "msvc_toolchain.cmake"), toolchain)
        # packaging
        vs_path = "C:/Program Files (x86)/Microsoft Visual Studio/2019_2/Community"
        copy(self, "*", src=vs_path, dst=self.package_folder)

    def package_info(self):
        f = os.path.join(self.package_folder, "msvc_toolchain.cmake")
        self.conf_info.append("tools.cmake.cmaketoolchain:user_toolchain", f)
        pf = self.package_folder.replace("\\", "/")
        self.buildenv_info.define("CONAN_MSVC_16_FOLDER", pf)

And just apply it with a profile to other cmake recipe:

include(default)
[settings]
compiler.version=192

[tool_requires]
msvc/16

The keys:

• I have modified the folder in the system to 2019_2 to make sure CMake or vswhere won't find it accidentally
• The CMAKE_GENERATOR_INSTANCE as environment variable only works for the CMake.configure step, but not for CMake.build step, because it doesn't remain as cache variable
• Then, I have had to define a toolchain to be able to force it as cache variable
• The toolchain is made agnostic of the package location via an environment variable pointing to the package folder, which might be variable

I have packaged the whole directory, I am pretty sure it is possible to discard many of those parts and make the package smaller.

Does this help? We might want to document this, write a short blog post or something too.

[memsharded]

I am submitting conan-io/docs#3666 to the docs, it will close this ticket as resolved, thanks!

[memsharded]

Colleagues from the team have reported:

• Not working when using in a blank machine, CMake errors (there might be too many registry checks that CMake does)
• No working when trying to use it in an ARM64 machine, even if the VS packaged contained tools for ARM64

I guess this is more complicated that it seems on the surface. Not sure it is worth the effort, and this is not part of Conan client anyway, but more like a recipe/community thing.

Any feedback so far @klausholstjacobsen ? Thanks!

[memsharded]

No further feedback here, and this is not a Conan client issue, but more like very specific about how VS could be made relocatable, and this would be outside of our scope, as much as we would like to help, we don't have enough resources to keep investigating this.

I think this is definitely possible, as I have heard from some users that they managed to do it, so I am not closing this, and instead moving it to the "contrib" repo, where in the future this would make sense to be contributed.