Qt plugin paths handling in nixpkgs causes problems, software picking wrong plugins

[shatsky]

1. Nix patches Qt to implicitly, unconditionally add plugin search paths based on PATH (replacing '/bin' with '/lib/qt-${ver}/plugins' in each one) to the beginning of the list. This causes Qt apps to pick plugins which are collected in /run/current-system/sw/lib/qt-${ver}/plugins/ which can (and does) break things. E. g. I've built custom kwin but it doesn't work as expected because it picks plugins from other instance of kwin which got collected there. I see this was added to pkgs/development/libraries/qt-5/5.12/qtbase.patch.d/0009-qtbase-qtpluginpath.patch by commit ec9e7e4 , merged in pull request Qt 5.12.4 #69086 by @ttuegel , but I don't see any explanation about the need for this in the commit msg nor PR text nor in source comments. Why at all allow Qt apps to discover plugins which don't belong to its deps? If it's because of theme engines I think this should be handled separately.
2. Maybe not an issue, but I see that wrappers created for qt binaries build enormous QT_PLUGIN_PATH vars, causing a lot of directory scanning on each plugin loading. Why don't we just collect for each Qt app package symlinks to all plugins of its qt deps in its /nix/store/${out-path}/lib/qt-${ver}/plugins?

Another example: I can't run pcmanfm-qt from user's outdated profile within KDE/Plasma session unless I unset QT_PLUGIN_PATH because otherwise pcmanfm-qt picks plugins both from paths derived from PATH (in qt-5.12/ subdirs, set at buildtime, as it was built against qt-5.12) and paths of Qt deps of parent KDE/Plasma process added to env by its wrapper (in qt-5.12.7/ subdirs) and crashes with Cannot mix incompatible Qt library (version 0x50c07) with this library (version 0x50c00) error. I had also seen whole KDE/Plasma breaking for user before, though I didn't know what to check at that time unfortunately, now I'm almost sure it was because of its processes similarly picking some plugins from user's outdated profile dir.

My current idea is that for every Qt app we should collect all plugins of its dep closure in lib/ subdir of its nix store dir so that it can use it as QTDIR, as described in https://doc.qt.io/qt-5/deployment-plugins.html . Paths to global plugins like styles and input methods may be managed by wrappers which can both add and remove QT_PLUGIN_PATH components, but should print warnings about these changes.

[ttuegel]

Why at all allow Qt apps to discover plugins which don't belong to its deps? If it's because of theme engines I think this should be handled separately.

Qt plugins are used for themes, input methods, image formats, integration between applications, and probably other things I'm forgetting at the moment. The reason that Qt applications are allowed to discover plugins that don't belong to their dependencies is: that is what plugins are used for in practice, and it's the behavior that Qt upstream and Qt application developers expect. The other options I am aware of are: 1. break all those things in subtle ways, or 2. require the user to rebuild all Qt packages on their system whenever they install a new Qt package. Both choices are unacceptable.

Note that item 2 above would also prevent using nix-env, but in any case I recommend that you do not use nix-env. Avoiding it will stop the "incompatible Qt library" error. If you choose to use nix-env, I recommend strongly that it is always kept up-to-date with the system, or at least consistent with itself. One way to do that is to write a Nix expression for the user environment and to always re-install the entire environment using nix-env -ir.

My current idea is that for every Qt app we should collect all plugins of its dep closure in lib/ subdir of its nix store dir so that it can use it as QTDIR, as described in https://doc.qt.io/qt-5/deployment-plugins.html . Paths to global plugins like styles and input methods may be managed by wrappers which can both add and remove QT_PLUGIN_PATH components, but should print warnings about these changes.

There are two issues here that I want to discuss separately. First is managing the size of QT_PLUGIN_PATH. Second is having wrappers manage global plugins.

In the past, we have discussed controlling the size of QT_PLUGIN_PATH by building a symlink farm so that there is only one entry in QT_PLUGIN_PATH. Actually, we have discussed doing that in several ways, besides just a symlink farm. This is slightly different than what you proposed here:

collect all plugins of its dep closure in lib/ subdir of its nix store dir so that it can use it as QTDIR

We cannot simply set the Nix store path as QTDIR because some (most?) applications have multiple output paths. We also want to avoid linking dependencies into the output path because it will cause huge conflicts when two applications with the same dependencies are installed into the user or system profile; the conflicts are resolvable, but it's not nice to create avoidable conflicts.

If you want to work on this, there is relevant prior art in #44047 and #85103.

The second issue is having the wrappers manage paths for global plugins. It seems to me that this will, in practice, amount to adding /run/current-system/sw/lib/qt-x.y.z/plugins to QT_PLUGIN_PATH.

Note that Qt plugins in Nixpkgs are slightly broken by patchelf. They should not have any Qt dependencies in DT_RUNPATH. Removing these entries would stop plugins from causing the "incompatible Qt library" error. Qt guarantees binary compatibility of plugins between library versions. The way dynamic linking works, each plugin would get the Qt library from the application that loads it. That's how the plugin loading works on other distributions. However, it requires all Qt libraries to be installed in the same prefix; in our case, that means we would need to abandon the multi-module build, and I think that would increase closure sizes significantly.

[stale]

I marked this as stale due to inactivity. → More info

[pasqui23]

Still interested

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/announcing-nomia-a-general-resource-manager-inspired-by-nix/12591/26

[stale]

I marked this as stale due to inactivity. → More info

[squalus]

Still an issue

[miniBill]

This also breaks nix-shell -p tiled for me, and unsetting QT_PLUGIN_PATH makes it run.

[hakan-demirli]

This breaks nix-shell -p k4dirstat too. Unsetting QT_PLUGIN_PATH indeed solves the problem.