Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update plugins, development and distributions requirements, and the distributions' Python version #1023

Merged

Conversation

benoit-pierre
Copy link
Member

Initially, the idea was to stay on PyQt5 5.9.x (since Qt 5.9 is supposed to be a LTS version), and only update to new point releases, but PyQt5 does not follow that, so we might as well update to the latest version.

Note: there's no QtWebEngine support with recent win32 builds of PyQt5, hence the switch to generating a 64bits distribution.

I've tested locally the new AppImage on Arch Linux and Ubuntu Trusty, and the new Windows installer/distribution on a Windows 10 VM, someone need to test the macOS image.

@morinted
Copy link
Member

Running the CircleCI build gives this error (when running the Contents/Plover script)

qt.qpa.plugin: Could not load the Qt platform plugin "cocoa" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: cocoa, minimal, offscreen, webgl.

Abort trap: 6

I've yet to get the requirements installed properly on my machine to run from source, I think I need to fixup my python installation.

@morinted
Copy link
Member

From source, the plugins manager fails to load:

running build_ext
2018-11-12 22:43:17,233 [MainThread] ERROR: error loading gui.qt.tool plugin: plugins_manager (from plover_plugins_manager.gui_qt.manager)
Traceback (most recent call last):
  File "/Users/ted/git/plover/plover/registry.py", line 56, in register_plugin_from_entrypoint
    obj = entrypoint.load()
  File "/Users/ted/Library/Python/3.6/lib/python/site-packages/pkg_resources/__init__.py", line 2343, in load
    return self.resolve()
  File "/Users/ted/Library/Python/3.6/lib/python/site-packages/pkg_resources/__init__.py", line 2349, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/plover_plugins_manager/gui_qt/manager.py", line 19, in <module>
    from plover_plugins_manager.registry import Registry
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/plover_plugins_manager/registry.py", line 4, in <module>
    from plover_plugins_manager import global_registry, local_registry
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/plover_plugins_manager/global_registry.py", line 9, in <module>
    from pip.download import PipSession, PipXmlrpcTransport
ModuleNotFoundError: No module named 'pip.download'

which I guess is intentional?

@benoit-pierre
Copy link
Member Author

Did you update the plugins manager to the latest version?

@morinted
Copy link
Member

Oh, yep, after installing requirements_plugins.txt, the issue's gone.

I'll still have to investigate what's going on with the .app and QT.

@morinted
Copy link
Member

Trying to build the .app locally,

appdir_python()
{
  env     PYTHONNOUSERSITE=1     /Users/ted/git/plover/./build/plover-4.0.0.dev8.app/Contents/Frameworks/python3.6 "$@"
}

appdir_python -m plover_build_utils.get_pip
error <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)>
error <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)>
Traceback (most recent call last):
  File "/Users/ted/git/plover/build/plover-4.0.0.dev8.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/ted/git/plover/build/plover-4.0.0.dev8.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/ted/git/plover/plover_build_utils/get_pip.py", line 33, in <module>
    get_pip(sys.argv[1:])
  File "/Users/ted/git/plover/plover_build_utils/get_pip.py", line 14, in get_pip
    'a883be89e37dbaaabe6a5d8dfa871d3dfa8b2f9e')
  File "/Users/ted/git/plover/plover_build_utils/download.py", line 50, in download
    assert os.path.exists(dst), 'could not successfully retrieve %s' % url
AssertionError: could not successfully retrieve https://bootstrap.pypa.io/get-pip.py

If I do ./build/plover-4.0.0.dev8.app/Contents/Frameworks/python3.6 -m pip list, certifi 2018.10.15 is shown which I had assumed would prevent such an error from occurring.

I saw you commented on a similar issue, did you find a solution?

@benoit-pierre
Copy link
Member Author

Yeah, but isn't picking up certifi from your user site? I think this work fine if get-pip.py is already in the download cache (which is why it works with the CI), can you with this patch:

 osx/make_app.sh               | 3 +++
 plover_build_utils/get_pip.py | 2 ++
 2 files changed, 5 insertions(+)

diff --git i/osx/make_app.sh w/osx/make_app.sh
index 1b741f88..5bb2a4bf 100644
--- i/osx/make_app.sh
+++ w/osx/make_app.sh
@@ -56,6 +56,9 @@ mkdir "$target_libs/site-packages"
 # Add sitecustomize.py -- adds the above site-packages to our Python's sys.path
 cp "$plover_dir/osx/app_resources/sitecustomize.py" "$target_libs/sitecustomize.py"
 
+# Prefetch get-pip using the system Python (to avoid SSL errors).
+get_pip --no-download
+
 # Switch to target Python.
 python="$PWD/$target_dir/$target_python"
 unset __PYVENV_LAUNCHER__
diff --git i/plover_build_utils/get_pip.py w/plover_build_utils/get_pip.py
index 81d1cc88..09115078 100644
--- i/plover_build_utils/get_pip.py
+++ w/plover_build_utils/get_pip.py
@@ -12,6 +12,8 @@ def get_pip(args=None):
     # Download `get-pip.py`.
     script = download('https://bootstrap.pypa.io/get-pip.py',
                       'a883be89e37dbaaabe6a5d8dfa871d3dfa8b2f9e')
+    if args == ['--no-install']:
+        return
     # Make sure wheels cache directory exists to avoid warning.
     if not os.path.exists(WHEELS_CACHE):
         os.makedirs(WHEELS_CACHE)

@morinted
Copy link
Member

sigh pebcak: "/Applications/Python 3.6/Install Certificates.command" wasn't run after installing python meaning the certs weren't linked.

Okay--patch not needed after that. I can recreate the qt.qpa.plugin issue locally so that means I can try and fix it now 😛

@morinted
Copy link
Member

QT_DEBUG_PLUGINS=1 ./dist/Plover.app/Contents/MacOS/Plover
QFactoryLoader::QFactoryLoader() checking directory path "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms" ...
QFactoryLoader::QFactoryLoader() looking at "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib"
Found metadata in lib /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "cocoa"
        ]
    },
    "className": "QCocoaIntegrationPlugin",
    "debug": false,
    "version": 330498
}


Got keys from plugin meta data ("cocoa")
QFactoryLoader::QFactoryLoader() looking at "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqminimal.dylib"
Found metadata in lib /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqminimal.dylib, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "minimal"
        ]
    },
    "className": "QMinimalIntegrationPlugin",
    "debug": false,
    "version": 330498
}


Got keys from plugin meta data ("minimal")
QFactoryLoader::QFactoryLoader() looking at "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqoffscreen.dylib"
Found metadata in lib /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqoffscreen.dylib, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "offscreen"
        ]
    },
    "className": "QOffscreenIntegrationPlugin",
    "debug": false,
    "version": 330498
}


Got keys from plugin meta data ("offscreen")
QFactoryLoader::QFactoryLoader() looking at "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwebgl.dylib"
Found metadata in lib /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqwebgl.dylib, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "webgl"
        ]
    },
    "className": "QWebGLIntegrationPlugin",
    "debug": false,
    "version": 330498
}


Got keys from plugin meta data ("webgl")
QFactoryLoader::QFactoryLoader() checking directory path "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/platforms" ...
Cannot load library /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib: (dlopen(/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib, 133): Library not loaded: @rpath/QtDBus.framework/Versions/5/QtDBus
  Referenced from: /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib
  Reason: Incompatible library version: libqcocoa.dylib requires version 5.11.0 or later, but QtDBus provides version 5.9.0)
QLibraryPrivate::loadPlugin failed on "/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib" : "Cannot load library /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib: (dlopen(/Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib, 133): Library not loaded: @rpath/QtDBus.framework/Versions/5/QtDBus\n  Referenced from: /Users/ted/git/plover/dist/Plover.app/Contents/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqcocoa.dylib\n  Reason: Incompatible library version: libqcocoa.dylib requires version 5.11.0 or later, but QtDBus provides version 5.9.0)"
qt.qpa.plugin: Could not load the Qt platform plugin "cocoa" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: cocoa, minimal, offscreen, webgl.

Abort trap: 6

So, sure enough QtDBus.framework is present in my system install but not included in the

5.11.2 is working so it seems like 5.11.3 introduced whatever is causing an issue here.

@benoit-pierre
Copy link
Member Author

benoit-pierre commented Nov 17, 2018

And if you remove QtDBus from the blacklist: osx/app_resources/dist_blacklist.txt:22: **/*DBus*?
(We keep it on other platforms).

@morinted
Copy link
Member

Confirmed, removing that from the blacklist and everything's working fine with the PR as-is. Thank you for your patience.

@benoit-pierre benoit-pierre merged commit bf8a598 into openstenoproject:master Nov 18, 2018
@benoit-pierre benoit-pierre deleted the update_requirements branch November 18, 2018 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants