Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qt problems with latest Qt from vcpkg #134

Closed
traversaro opened this issue Nov 12, 2018 · 33 comments
Closed

Qt problems with latest Qt from vcpkg #134

traversaro opened this issue Nov 12, 2018 · 33 comments

Comments

@traversaro
Copy link
Member

traversaro commented Nov 12, 2018

@InesSorrentino experienced several problems related to running the installed version of YARP and ICUB GUIs that use QMLs when using the latest version of Qt installed via vcpkg .

For some reason, the version of the executable contained in the build directory seem to be working fine. As there were recently some big changes in how Qt is installed by vcpkg (microsoft/vcpkg#4164 and microsoft/vcpkg#4629), it would be worth to check it this problems are actually reproducible.

@claudiofantacci
Copy link
Collaborator

I guess they recompiled everything a was still not working. Am I right?
Speaking about QML and vcpkg, there is a specific folder in the vcpkg root folder that is specifically for QML things (not an expert of it, so I'm just throwing whatever comes into my mind here and there). Maybe we should have a look at it and understand whether we have to add folders in PATH or something similar.
Anyway, I'm still not 100% sure that the problem is related to vcpkg specifically.

@drdanz
Copy link
Member

drdanz commented Nov 12, 2018

If you are using the vcpkg toolchain, did you disable VCPKG_APPLOCAL_DEPS? This disables copying the dlls in the executable directory

@isorrentino
Copy link

Yes, I compiled again all robotology-superbuild from scratch. The iCub Gui works if I run it from Visual Studio, but not if I launch it from command line.

@isorrentino
Copy link

No, I am not using the vcpkg toolchain.

@traversaro
Copy link
Member Author

I guess they recompiled everything a was still not working. Am I right?

Something changed: the executables in the build directory were working correct, but the installed were not working properly: in particular after the line engine.load(QUrl(QStringLiteral("qrc:/main.qml"))) in https://github.com/robotology/icub-main/blob/4f5881acafab693f763dd277f87a8996fdea01a3/src/tools/iCubSkinGui-qt/src/main.cpp#L27 the engine.rootObjects() data structure had size 0, while it typically had size 1 in correctly working applications. Interesting, other applications like yarpmotorgui and yarplogger were instead working fine also from when launched from the installation directory. Even more interestingly, the yarpamanager application was not correctly loading the icons when launched from the installation directory, while everything was working fine (icon were visualized) when launched from the build directory.

Anyway, I'm still not 100% sure that the problem is related to vcpkg specifically.

I am trying to reproduce it on a second setup to be sure about it.

If you are using the vcpkg toolchain, did you disable VCPKG_APPLOCAL_DEPS? This disables copying the dlls in the executable directory

The documented procedure to use vcpkg with the robotology-superbuild is https://github.com/robotology/robotology-superbuild#vcpkg (TL;DR: no toolchain for us) . The use of the toolchain is not convenient because it is propagated to subprojects unless you modify the Build***.cmake files (that is what we do in some private superbuilds in which we want to do cross-compilation).

@traversaro
Copy link
Member Author

I reproduced the issue experienced by @InesSorrentino (and by @claudia-lat , even if it was not reported here) with vcpkg commit microsoft/vcpkg@8dc8d0e .

To make sure that it was actually a vcpkg issue, I tried also to download the latest Qt 5.11.2 binaries from https://www.qt.io/ , and with those everything (yarpmanager with icons, yarpview) is working fine, so the issue seems to be related to vcpkg indeed.

In particular, the issue seems to be related to the Qt Resource System. yarpmanager is not able to load the icons because they are embedded in the binary using the Qt Resource System, and the QML-based application such as yarpview do not start because they are unable to load the "qrc:/main.qml" file. Ideally, we should try to reproduce the issue in a small example (so-called Minimal Working Example ) and report it to vcpkg. In the meanwhile, it may be worth to modify the superbuild readme to mention this problem, and discourage users to use vcpkg (we could also just remove the vcpkg section from the README).

@traversaro
Copy link
Member Author

Another test that it may be worth doing would be to compile YARP using the vcpkg's toolchain, instead of manually setting the enviroment variables. I do not expect the result to change, but it may be worth doing this as an additional check.

@claudiofantacci
Copy link
Collaborator

Another test that it may be worth doing would be to compile YARP using the vcpkg's toolchain, instead of manually setting the enviroment variables. I do not expect the result to change, but it may be worth doing this as an additional check.

I'll try to do it in the next days.

@DanielePucci
Copy link
Contributor

@claudiofantacci any news by chance on the above test?

@claudiofantacci
Copy link
Collaborator

Sorry guys, I froze my vcpkg builds to polish up some code.
I'll start the test now and hope to be back to you as soon as possible.

lrapetti pushed a commit to lrapetti/robotology-superbuild that referenced this issue Dec 19, 2018
Given robotology#134 , I think it make sense to do this, at least until robotology#134 is fixed.
@claudiofantacci
Copy link
Collaborator

I upgraded vcpkgs and ports to the latest version. I have the same reported issues:

  • yarpmanager opens without icons (functionalities seem ok)
  • yarpview fails silently all the times

@traversaro
Copy link
Member Author

As mentioned in #134 (comment) , I suspect the issue is related to something that is not working in the Qt Resource System.

@claudiofantacci
Copy link
Collaborator

claudiofantacci commented Dec 19, 2018

I reproduced the issue experienced by @InesSorrentino (and by @claudia-lat , even if it was not reported here) with vcpkg commit Microsoft/vcpkg@8dc8d0e .

To make sure that it was actually a vcpkg issue, I tried also to download the latest Qt 5.11.2 binaries from https://www.qt.io/ , and with those everything (yarpmanager with icons, yarpview) is working fine, so the issue seems to be related to vcpkg indeed.

In particular, the issue seems to be related to the Qt Resource System. yarpmanager is not able to load the icons because they are embedded in the binary using the Qt Resource System, and the QML-based application such as yarpview do not start because they are unable to load the "qrc:/main.qml" file. Ideally, we should try to reproduce the issue in a small example (so-called Minimal Working Example ) and report it to vcpkg. In the meanwhile, it may be worth to modify the superbuild readme to mention this problem, and discourage users to use vcpkg (we could also just remove the vcpkg section from the README).

I was able to fix everything, yet again, by adding the following qt.conf in the YARP bin installation folder:

[Paths]
Prefix = C:/Users/cfantacci/vcpkg/installed/x64-windows
Documentation = share/qt5/doc
Headers = include
Libraries = lib
Binaries = tools/qt5
LibraryExecutables = tools/qt5
Plugins = plugins
Qml2Imports = qml
Data = share/qt5
ArchData = share/qt5
HostData = share/qt5
HostBinaries = tools/qt5

Now everything works as expected.

Unfortunately, I tried to fix this issue on vcpkg in the past with microsoft/vcpkg#3280 that was partially reverted microsoft/vcpkg@473d63c.
This is very unofrtunate. We should really try to find a definitive solution to this.

cc @pattacini @randaz81

@traversaro
Copy link
Member Author

This is very unofrtunate. We should really try to find a definitive solution to this.

I guess the only permanent solution is to have some tests continuously running, either in vcpkg itself or in an external repo: building Qt is extremely slow, but perhaps with the new Azure CI system it should be doable. Related vcpkg issue: microsoft/vcpkg#72, in particular microsoft/vcpkg#72 (comment) for the "external" test solution.

@claudiofantacci
Copy link
Collaborator

The problem is that this scenario si very difficult to test, especially if it involves GUIs. You may have a program silently failing, or just not showing up the icons and you would notice that (unless I'm missing something test-wise). This is clearly an issue that must come from users, and possiby a patch from users themselves. I don't find a way out other than discussing with vcpkg people about this kind of issues and to understand whether we are misusing something or not.

@traversaro
Copy link
Member Author

The problem is that this scenario si very difficult to test, especially if it involves GUIs. You may have a program silently failing, or just not showing up the icons and you would notice that (unless I'm missing something test-wise).

Are you sure? If it is just a qrc problem, we just need to write a test that tries to access a resource using qrc, and fails if it is not able to find it.

@claudiofantacci
Copy link
Collaborator

Are you sure? If it is just a qrc problem, we just need to write a test that tries to access a resource using qrc, and fails if it is not able to find it.

You are right. I was focusing my thoughts on the GUI, and not on the Qt Resource System. 👍

Apart from this, are we going to try discussing with vcpkg people that this issue is still inplace? Should we ping about this in the other issue (or pulling people in here) or open an issue in vcpkg?

@traversaro
Copy link
Member Author

If you have a clear understanding of what the issue is (apart from the fact that qt.conf fixes it, I personally still do not understand the actual issue), I think we can open an issue vcpkg even without a MWE.

@claudiofantacci
Copy link
Collaborator

If you have a clear understanding of what the issue is (apart from the fact that qt.conf fixes it, I personally still do not understand the actual issue), I think we can open an issue vcpkg even without a MWE.

I think I found the problem! Let me double check it and report it here.

@claudiofantacci
Copy link
Collaborator

If you open the Qt5Core.dll with an hex-editor you will get something like the following list somewhere in the file:

qt_prfxpath=C:/Users/cfantacci/vcpkg/installed/x64-windows

...

C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/doc
include
lib
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/bin
bin
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/plugins
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/imports
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/qml
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/translations
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/examples
tests 

Now, by my understanding, every paths that is used by Qt start with qt_prfxpath and is expanded with the other entries of the list. Not sure if the expantion happens only to relative paths, but I would say so.
If I look at the directory C:\Users\cfantacci\vcpkg\packages\qt5-base_x64-windows I have

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        18-Dec-18     12:49                bin
d-----        18-Dec-18     12:49                debug
d-----        18-Dec-18     12:33                include
d-----        18-Dec-18     12:50                lib
d-----        18-Dec-18     12:50                plugins
d-----        18-Dec-18     12:50                share
d-----        18-Dec-18     12:49                tools
-a----        18-Dec-18     12:50             46 BUILD_INFO
-a----        18-Dec-18     12:50            331 CONTROL

for the directory C:\Users\cfantacci\vcpkg\packages\qt5-base_x64-windows\share

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        18-Dec-18     12:49                cmake
d-----        18-Dec-18     12:49                qt5
d-----        18-Dec-18     12:50                qt5-base

for the directory C:\Users\cfantacci\vcpkg\packages\qt5-base_x64-windows\share\qt5

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        18-Dec-18     12:32                debug
d-----        18-Dec-18     12:49                doc
d-----        18-Dec-18     12:49                mkspecs

and finally for the directory C:\Users\cfantacci\vcpkg\packages\qt5-base_x64-windows\share\qt5-base

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        13-Sep-18      6:25           8173 copyright

Looking back at our list, the following paths are wrong:

C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/imports
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/qml
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/translations
C:/Users/cfantacci/vcpkg/packages/qt5-base_x64-windows/share/qt5/examples

This paths are overridden by qt.conf and the file that I syntesized and reported in the previous points to the correct folder, in particular, I guess, for the qml folder. I'm not 100% sure that this is the real cause of the failure, but for sure there are some mistakes that are worth reporting upstream with an issue. What do you think?

@traversaro
Copy link
Member Author

What do you think?

I agree.

@claudiofantacci
Copy link
Collaborator

claudiofantacci commented Feb 27, 2019

I think we can finally close this issue as the PR microsoft/vcpkg#5456 in vcpkg fixed it.

@traversaro
Copy link
Member Author

I would prefer if someone successfully tested a complete robotology-superbuild install (at least of the CORE profile) before closing this issue.

@claudiofantacci
Copy link
Collaborator

Yes, also there has been another PR that corrected the tools locations and hopefully did not regress to the other full path issue in the Qt5Core.

I really fear that we are at an empasse with Qt under vcpkg.

@traversaro
Copy link
Member Author

traversaro commented Jul 29, 2019

Possibly this could be a related issue: microsoft/vcpkg#7461 .

@traversaro
Copy link
Member Author

@pattacini you recently updated vcpkg, did you experienced any problem such as the one described in this issue?

@pattacini
Copy link
Member

@traversaro yes the latest qt makes our GUI quite screwed up as in the screenshot below, where SVG icons are not rendered properly:

Annotation 2019-09-10 213652

@traversaro
Copy link
Member Author

yarpmanager icons indeed look bad, but yarpview works fine?

@pattacini
Copy link
Member

@traversaro that's correct; yarpview is rendered nicely.

@traversaro
Copy link
Member Author

traversaro commented Sep 11, 2019

Great, so the problem is much less impacting w.r.t. to the original problem for which this issue was opened. In that case, it probably makes sense to reintroduce vcpkg docs in the README.
cc @Nicogene @aerydna

@pattacini
Copy link
Member

@traversaro I can confirm that the latest Qt version 5.12.5 built via vcpkg solves the SVG icons problem 🎉

@traversaro
Copy link
Member Author

@traversaro I can confirm that the latest Qt version 5.12.5 built via vcpkg solves the SVG icons problem tada

For reference: @pattacini in his tests only tested executable in the build directory of YARP, without installing them.

@traversaro
Copy link
Member Author

This has been tested by several people, so we can finally close it and return to document the use of vcpkg for dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants