Compiling Basilisk on Windows and protobuf doesn't compile

[JeanLoupCaillet]

Hello,

I'm currently trying to compile basilisk sources within Windows operating system.
After following the tutorial ( https://hanspeterschaub.info/basilisk/Install/installOnWindows.html#), i'm faced with errors when running _python conanfile.p_y

It fails with compiling protobuf/3.5.2@bincrafters/stable.

-- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.22621.
-- The C compiler identification is MSVC 19.34.31937.0
-- The CXX compiler identification is MSVC 19.34.31937.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Conan: called by CMake conan helper
-- Conan: called inside local cache
-- Conan: Adjusting output directories
-- Conan: Using cmake global configuration
-- Conan: Adjusting default RPATHs Conan policies
-- Conan: Adjusting language standard
-- Conan CMake Wrapper
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_NO_PACKAGE_REGISTRY


-- Build files have been written to: C:/.conan/3fbd77/1/build_subfolder
MSBuild version 17.4.1+9a89d02ff for .NET Framework
  Checking Build System
  Building Custom Rule C:/.conan/3fbd77/1/source_subfolder/cmake/CMakeLists.txt
  arena.cc
  arenastring.cc
  extension_set.cc
  generated_message_table_driven_lite.cc
  generated_message_util.cc
  coded_stream.cc
  zero_copy_stream.cc
  zero_copy_stream_impl_lite.cc
  message_lite.cc
  repeated_field.cc
  atomicops_internals_x86_gcc.cc
  atomicops_internals_x86_msvc.cc
C:\.conan\3fbd77\1\source_subfolder\src\google/protobuf/stubs/hash.h(241,50): error C2143: syntax error: missing ',' before '<' (compiling source file C:\.conan\3fbd77\1\source_subfolder\src\google\protobuf\extension_set.cc) [C:\.conan\
3fbd77\1\build_subfolder\source_subfolder\cmake\libprotobuf.vcxproj]
C:\.conan\3fbd77\1\source_subfolder\src\google/protobuf/stubs/hash.h(242,2): message : see reference to class template instantiation 'google::protobuf::hash<Key>' being compiled (compiling source file C:\.conan\3fbd77\1\source_subfolder
\src\google\protobuf\extension_set.cc) [C:\.conan\3fbd77\1\build_subfolder\source_subfolder\cmake\libprotobuf.vcxproj]
C:\.conan\3fbd77\1\source_subfolder\src\google/protobuf/stubs/hash.h(255,14): error C2039: 'hash_compare': is not a member of 'std' (compiling source file C:\.conan\3fbd77\1\source_subfolder\src\google\protobuf\extension_set.cc) [C:\.co
nan\3fbd77\1\build_subfolder\source_subfolder\cmake\libprotobuf.vcxproj]
...
protobuf/3.5.2@bincrafters/stable:
protobuf/3.5.2@bincrafters/stable: ERROR: Package 'e9a552ebe8f994398de9ceee972f0ad207df0658' build failed
protobuf/3.5.2@bincrafters/stable: WARN: Build folder C:\.conan\3fbd77\1
ERROR: protobuf/3.5.2@bincrafters/stable: Error in build() method, line 63
        cmake.build()
        ConanException: Error 1 while executing cmake --build C:\.conan\3fbd77\1\build_subfolder --config Release -- /m:12 /verbosity:minimal
Running cmake:
python -m conans.conan build . -if dist3/conan
Updating pip:
python -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\jean-loup\anaconda3\lib\site-packages (22.3.1)
Checking conan configuration: Done
cmake generator set to: Visual Studio 16 2019
ERROR: conanbuildinfo.txt file not found in C:\Users\Jean-Loup\Documents\Git\basilisk\dist3\conan
It is required for this command
You can generate it using 'conan install' 

By default my conan profile is :

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
compiler=Visual Studio
compiler.version=17
build_type=Release
[options]
[build_requires]
[env]

When I try to change compiler.

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
compiler=clang
compiler.version=14
compiler.libcxx=libstdc++11
build_type=Release
[options]
[build_requires]
[env]

I have this error with protobuf.

protobuf/3.5.2@bincrafters/stable: Calling build()
CMake Error: CMake was unable to find a build program corresponding to "MinGW Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
See also "C:/.conan/080bd1/1/build_subfolder/CMakeFiles/CMakeOutput.log".
protobuf/3.5.2@bincrafters/stable:
protobuf/3.5.2@bincrafters/stable: ERROR: Package 'f67ada7fd8cb6067d34a7b4246bd05be1dd86e18' build failed
protobuf/3.5.2@bincrafters/stable: WARN: Build folder C:\.conan\080bd1\1
ERROR: protobuf/3.5.2@bincrafters/stable: Error in build() method, line 62
        cmake = self._configure_cmake()
while calling '_configure_cmake', line 57
        cmake.configure(build_folder=self._build_subfolder)
        ConanException: Error 1 while executing cd C:\.conan\080bd1\1\build_subfolder && cmake -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPILER="clang" -DCONAN_COMPILER_VERSION="14" -DCONAN_CXX_FLAGS="-m64" -DCONAN_SHARED_LINKER_FLAGS="-m64" -DCONAN_C_FLAGS="-m64" -DCMAKE_CXX_FLAGS="-m64" -DCMAKE_SHARED_LINKER_FLAGS="-m64" -DCMAKE_C_FLAGS="-m64" -DBUILD_SHARED_LIBS="ON" -DCMAKE_INSTALL_PREFIX="C:\.conan\3e4f77\1" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -DCONAN_EXPORTED="1" -Dprotobuf_BUILD_TESTS="False" -Dprotobuf_WITH_ZLIB="False" -Dprotobuf_BUILD_PROTOBUF_LITE="False" -Wno-dev C:\.conan\080bd1\1

I don't want to use MinGW but LLVM, since Clang is installed with it.

Do you have any ideas of how I can manage to pass this step ?

I also tried to do:

conan remove protobuf
pip install protobuf

But still, conanfile download again.
By the way with Linux I succeed to compile. (With Clang too). But in order to have better performance with Vizard I need Windows.

Sorry for huge copy/paste.
Thanks in advance.
Jean-Loup

[davidtenone]

Hello,

I'm currently trying to compile basilisk sources within Windows operating system. After following the tutorial ( https://hanspeterschaub.info/basilisk/Install/installOnWindows.html#), i'm faced with errors when running _python conanfile.p_y

It fails with compiling protobuf/3.5.2@bincrafters/stable.

C:.conan\3fbd77\1\source_subfolder\src\google/protobuf/stubs/hash.h(241,50): error C2143: syntax error: missing ',' before '<' (compiling source file C:.conan\3fbd77\1\source_subfolder\src\google\protobuf\extension_set.cc) [C:.conan
3fbd77\1\build_subfolder\source_subfolder\cmake\libprotobuf.vcxproj]
C:.conan\3fbd77\1\source_subfolder\src\google/protobuf/stubs/hash.h(242,2): message : see reference to class template instantiation 'google::protobuf::hash' being compiled (compiling source file C:.conan\3fbd77\1\source_subfolder
\src\google\protobuf\extension_set.cc) [C:.conan\3fbd77\1\build_subfolder\source_subfolder\cmake\libprotobuf.vcxproj]
C:.conan\3fbd77\1\source_subfolder\src\google/protobuf/stubs/hash.h(255,14): error C2039: 'hash_compare': is not a member of 'std' (compiling source file C:.conan\3fbd77\1\source_subfolder\src\google\protobuf\extension_set.cc) [C:.co
nan\3fbd77\1\build_subfolder\source_subfolder\cmake\libprotobuf.vcxproj]

Have you solved this issue? I'm getting the same exact build errors as you.

[JeanLoupCaillet]

Hello,

No I haven' t solve the problem.
I tried many things.

Not having SDK (Visual Studio).
But in the conanfile if no compiler found it always take by default Visual Studio. So maybe change conanfile.py (I haven't tried).

Trying MinGW/Clang (LLVM).. (adding to path (env and system)). But still.
Trying different version of python (3.9/3.10).

Trying different version of protobuf (manually changed version in the conanfile.py).

There are a lot of combinaison.

But I can tell you this. The issue is the version of C++ compiler of visual studio. I guess above C++20 it doesn't work well for specific versions of protobuf.

That's why I tried to remove VS from laptop and trying another solutions like msys2/LLVM(clang)/MinGW. But the compiler is not detected by conanfile.py. (and PATH)
Even after changing the .conan/profile/default

I'll try later, but feel free to tell me if you succeeded by changing the version of the compiler :)

Best,

[davidtenone]

I was able to bypass the protobuf solution by excluding the vizInterface from the build using "--vizInterface False" in the build command. It's a temp workaround but I can build again. I agree with you there is something about VS 17.4 and building protobuf.

I did try using a newer version of protobuf, you can specify the version in the conanfile.py (self.requires.add("protobuf/3.5.2@bincrafters/stable"). I was able to build protobuf using this new version 3.9.1 but there is still a reference to protobuf version 3.5.2 somewhere that I haven't found and the build still fails when it tries to pull and build version 3.5.2.

I think the long term, clean solution to this problem is the newer version (3.9.1) of protobuf but I just don't know why the build is still trying to pull 3.5.2 as well.

[schaubh]

protobuffer 3.5.2 is specified on line 200 of conanfile.py, but also on line 202 where we need to install the binary to build the protobuffer files. Did you try to upgrade both to 3.9.1?

[davidtenone]

Thank you, I totally missed line 202. I have changed both now, cleaned and rebuilt. Protobuf builds now, lots of warnings but no complie errors like before. However, now I get the error below....

Running cmake:
python -m conans.conan build . -if dist3/conan
Using lockfile: 'C:\data\projects\basilisk\dist3\conan/conan.lock'
Using cached profile from lockfile
Updating pip:
python -m pip install --upgrade pip
Requirement already satisfied: pip in c:\data\projects\basilisk.venv\lib\site-packages (22.3.1)
Checking conan configuration: Done
cmake generator set to: Visual Studio 17 2022
conanfile.py (Basilisk/2.1.6b0): Calling build()
../../../../dist3/autoSource/cMsgCInterface/
Auto-generating message definitions: Done
ERROR: conanfile.py (Basilisk/2.1.6b0): Error in build() method, line 272
subprocess.check_call(cmdString)
CalledProcessError: Command '['protoc', '--cpp_out=./', 'vizMessage.proto']' returned non-zero exit status 3221225781.

[schaubh]

This means the system is not finding the protoc binary that should have been installed with line 202 in conanfile.py.

[JeanLoupCaillet]

Hi there,

Indeed, protobuf version is declared 2 times, I already tried to change them both to 3.9.1, no success.

Deactivate vizInterface is not a solution for me because, I want to use Vizard with better performance with Windows drivers (better GPU).

I'll try to figure something out. But thanks anyway @davidtenone.

Indeed, for me it's just the version of the compiler used with VS, maybe I'll try to install older version.
But for me modify env path to force to use CLANG or MinGW should work too.
@schaubh , is there something I'm missing ?

Best,

[schaubh]

I rarely compile on Windows, so I'm not much more help with this configuration. I am not aware of any other location in BSK that would require protobuffer 3.5.2 than in conanfile.py. Have you tried deleting your .conan folder and forcing a clean Conan install with just 3.9.1?

[JeanLoupCaillet]

I tried so many things, that I forget what I tested or not !!
I'll came back if I don't succeed :)

[schaubh]

I was able to get BSk to build with protobuffer 3.9.1 if I didn't include the opNav modules. All unit tests pass. If I include the opNav modules I get a compile error. Something is not compatible with opNav modules and this newer version of protobuffer. Thus, I'm not going to update the conanfile.py file until this issue is resolved. I expect this will be a while before we are able to look into this.

[dotower]

Hi, everyone!
I'm also trying to build BSK on Windows 11 via VSCode and I'm getting this error:

Any idea of what might be wrong?

[JeanLoupCaillet]

Hi Dotower,

Like that, I can't really help you.
Perhaps, you don't have CMAKE installed, or not in the PATH.

I need to know what are your steps before.

Sometimes there is a warning/error written before, so I'll need more information.

You can try conan remove protocol_installer and then retry the python conanfile.py command. (Maybe a corrupted package)
Don't hesitate,

Best,

[schaubh]

We are working on a fix where bincrafters is not required anymore for conan pulls. The branch feature/bsk_56_build_updates is being tested right now. See BSK issue #56.