packaging for mingw64 / MSYS2

[totaam]

Issue migrated from trac ticket # 1883

component: packaging | priority: major | resolution: fixed

2018-06-21 08:52:01: antoine created the issue

---

We should be able to submit this as a package to MSYS2: [https://github.com/msys2/msys2/wiki/Creating-Packages].

[totaam]

2018-06-21 11:12:12: antoine changed owner from antoine to smo

[totaam]

2018-06-21 11:12:12: antoine changed title from packaging for mingw to packaging for mingw64 / MSYS2

[totaam]

2018-06-21 11:12:12: antoine commented

---

Python build changes mostly completed in r19684.
We can now install xpra locally from an MSYS2 shell with:

rm -fr dist/*
python ./setup.py clean
python ./setup.py install --prefix=./dist --share=share/xpra --with-scripts
rsync -rplogt dist/* ${MINGW_PREFIX}/

Still TODO:

• install both python2 and python3 builds?
• figure out the dependencies we want to declare (ie: not opencv..)
• create some packages for the most important python dependencies (those should be easy): rencode, lz4, PyOpenGL + PyOpenGL_accelerate, cryptography
• fix audio subprocess command failures

@smo: can you help with any of this?

[totaam]

2018-07-10 16:14:37: antoine commented

---

Update from the transition to python 3.7 (#1878) done by msys2:

• OpenGL_accelerate needs patching to build: pyopengl: Python 3.7 Build failure
• websockify: we need our patch set - really wished I had merged this upstream now..

[totaam]

2018-08-03 14:50:04: antoine commented

---

• for websockify with python 3.7 on win32, see [/attachment/ticket/1568/websockify-0.8-py37.tar.xz].
• for paramiko / pynacl see [ssh client integration via paramiko #1646#comment:8]
• r20010 html5 client path fixup

[totaam]

2018-08-04 19:54:52: @totaam commented

---

• r20022: use mingw packages for some python dependencies
• for python-lzo see python-lzo 2.09 for win32 #1044#comment:9
• would be very useful to have libyuv (as per building all the dependencies from source on win32 #678#comment:40 - much easier now that libyuv builds the DLL by default and without errors) - every other installation step is now fully automated, see Building MSWindows
• pyopengl_accelerate can be built from source against python 3.7 (with a recent enough version of Cython), but only from a git copy (source here: [https://github.com/mcfletch/pyopengl]) and after removing the ".c" file to ensure they get re-generated - since the pyopengl version should match, build that from the same source tree (just remove the SGIX directories as those contain the "async" keyword which is now illegal - we don't need those anyway)
• lz4 is crucial, we need the workaround from r20171 (python-lz4 2.1 is broken, again #1929#comment:1) so it gets packaged properly (I'm not blaming cx_Freeze for this particular breakage)

[totaam]

2018-08-25 07:44:18: @totaam commented

---

python-zeroconf (original feature ticket: #1644) dropped support for python2 in version 0.20 (see TypeError: unsupported operand type(s) for &: 'str' and 'int' and python2-zeroconf: updated to 0.20.0, but 0.20.0 dropped python2 support), but something is not right and using easy_install will pickup the latest (broken) version, so we have to be specific about it: r20193. No such problems with python3.

[totaam]

2018-08-26 17:24:58: antoine commented

---

Work in progress lives here: github thesmo MINGW-packages

[totaam]

2018-10-16 06:28:05: antoine commented

---

Here's a slightly more tricky one: pygtkglext as per #678#comment:27 (2 patches and a workaround for the install step), it was MIA from my 64-bit build VM, fortunately the new "native" backend was working OK so I didn't notice!

[totaam]

2018-10-21 19:34:32: antoine commented

---

Minor update: rencode 1.0.6, see r20754

[totaam]

2018-11-13 09:03:09: antoine commented

---

@smo: see #2004#comment:3 for libyuv.

[totaam]

2018-11-28 15:29:59: smo commented

---

Still waiting on this merge.
[https://github.com/Alexpux/MINGW-packages/pull/4595]

[totaam]

2018-11-28 15:34:39: antoine commented

---

Can you update what has been merged and replace easy_install with pacman in the setup and update scripts?

[totaam]

2018-11-29 12:30:45: antoine commented

---

Found that paramiko, comtypes, netifaces and rencode are available in MSYS2 now, so r21147 switches to that.

[totaam]

2019-01-18 03:58:55: smo commented

---

Here is a link to all the closed merge requests.

https://github.com/Alexpux/MINGW-packages/pulls?q=is%3Apr+author%3Athesmo+is%3Aclosed

nvidia-ml-py seems to be the only open one still

[totaam]

2019-01-25 09:39:16: antoine commented

---

Useful link: thesmo pull requests

r21483 switches to MSYS2 packages for: websocket-client, setproctitle, pyu2f, ldap, ldap3, bcrypt and pynacl. Also zeroconf, but only for python3 (see below, comment:5).

Important libraries left to package / fix:

• lz4 - crucial, see comment:4
• libyuv - see comment:10
• websockify - see comment:3
• nvidia-ml-py - pending: [https://github.com/Alexpux/MINGW-packages/pull/4595]
• zeroconf should be using version 0.19 for python2 as per comment:5
• pyopengl: should be packaging a version newer than 3.1.0, 3.1.1 needs patches for python3 support, as per comment:2
• pygtkglext - see comment:8 (maybe we can just forget about this one since 2.5 will be using the "native" opengl backend by default)

[totaam]

2019-01-27 16:12:36: @totaam commented

---

With the two build files above, I can generate a working installation of xpra.

Still TODO:

• apply fixes from comment:15
• merge those two packages
• provide a better way of generating the /etc and /share/xpra directories, without doing a fake build (even if it is a minimal one, doing the cx_Freeze step is expensive and unnecessary - also requires cx_Freeze which we don't really need for anything)
• is there a way to make a truly generic noarch package? the contents of the mingw-w64-x86_64-xpra-common are not architecture dependent at all, we want a mingw-w64-xpra-common instead

[totaam]

2019-01-31 16:04:58: antoine commented

---

Sent pull request for python-lz4 + python-pkgconfig (needed by python-lz4): [https://github.com/Alexpux/MINGW-packages/pull/4937].

[totaam]

2019-02-01 14:56:58: antoine changed status from new to assigned

[totaam]

2019-02-01 14:56:58: antoine changed owner from smo to antoine

[totaam]

2019-02-01 14:56:58: antoine commented

---

The pull request has been merged, I have made a new one for python-nvidia-ml - the one above was broken for python3.

Since 2.5 will drop websockify (#2121) and make "native" the default opengl backend (r21382), there aren't many important packages left to fix / add from comment:15:

• libyuv as per comment:10
• pyopengl as per comment:2
• zeroconf as per comment:5 - no idea how to deal with this one..

[totaam]

2019-02-01 16:15:38: antoine commented

---

pull request for python-zeroconf: [https://github.com/Alexpux/MINGW-packages/pull/4941] with one build file for each python version.

[totaam]

2019-02-04 15:41:28: antoine commented

---

As of r21532 + r21533 + r21534, minimal builds don't include anything at all, not even any python bits. Adding --with-data we get just the extra bits we need for packaging "xpra-common".

[totaam]

2019-02-05 14:24:29: antoine commented

---

Pull request for libyuv: [https://github.com/Alexpux/MINGW-packages/pull/4948].

[totaam]

2019-02-05 15:55:06: antoine commented

---

Pull request for fixing pyopengl: [https://github.com/Alexpux/MINGW-packages/pull/4949]

[totaam]

2019-02-06 05:25:07: antoine commented

---

Merged upstream:

• python-opengl fixes merged - moving opengl-accelerate to pyopengl accelerate for msys2 #2134
• python-zeroconf fixes

[totaam]

2019-02-07 03:45:36: antoine commented

---

pyopengl-accelerate pull request (added to pyopengl): [https://github.com/Alexpux/MINGW-packages/pull/4954]

[totaam]

2019-02-08 04:52:47: antoine commented

---

Merged upstream:

• pyopengl-accelerate
• python-nvidia-ml

Updated setup and removed python update script: r21590.

Only things left for this ticket:

• libyuv upstream merge (comment:21)
• submit xpra package merge request

[totaam]

2019-02-13 07:54:34: antoine commented

---

The install changes caused a regression: #2144#comment:1.

[totaam]

2019-02-23 06:54:22: antoine commented

---

For python-gssapi and winkerberos (#1691), moving to a new ticket: #2170

[totaam]

2019-02-26 13:41:51: antoine commented

---

libyuv has been merged upstream and we now use it: r21892

[totaam]

2019-03-08 15:42:25: antoine commented

---

Pull request for xpra package: [https://github.com/msys2/MINGW-packages/pull/5037]

[totaam]

2019-03-13 06:13:43: antoine changed status from assigned to closed

[totaam]

2019-03-13 06:13:43: antoine set resolution to fixed

[totaam]

2019-03-13 06:13:43: antoine commented

---

And finally, xpra is in MSYS2: commit d6fe4115fbc605a9684991ffef1a4085894320f3.

[totaam]

2019-03-20 06:17:16: antoine commented

---

Update to 2.5 release: [https://github.com/msys2/MINGW-packages/pull/5095].

[Biswa96]

New source tarballs in xpra.org/src do not contain xpra/platform/win32/setappid{.cpp, .h} files.

[totaam]

@Biswa96 thanks for the heads up.
Fixed by the commit above, these will be included in future releases.