setup a proper build infrastructure for win32 builds

[totaam]

Developing on win32 is a nightmare and we rely on far too many sources for all the DLLs we need.
We should setup a cross compile environment based on mingw so that anyone can just follow some simple (?) steps and obtain all the components, built from source, with or without debugging symbols, etc
Including:

• gtk
• gstreamer
• x264
• libav
• webp
  etc..
  Then we can also automate the build and include that from the buildbot.

This should make the code more reliable/maintainable and may well help in solving a number of ugly win32 tickets:

• try newer gstreamer win32 builds #299
• Sound stops working and gives errors when turned on and off using --speaker-codec wavpack. #297
• opengl rendering on osx #226
• opengl rendering on win32 #227
• gmp and mpfr build for win32 - needed for pycrypto fastmath #373

[totaam]

This is becoming more important so we can ensure that we don't ship vulnerable media libraries (see #544)

[totaam]

Here are some of the mingw packages we will need, available on Fedora 20:

• base stuff: mingw32-winpthreads, mingw32-xz, mingw32-xz-libs
• libraries: mingw32-orc, mingw32-mpfr, mingw32-liboil, mingw32-gmp
• gtk: mingw32-gtk2, mingw32-gtkglext (no pygtkglext...)
• gstreamer: mingw32-gstreamer, mingw32-gstreamer-plugins-bad-free, mingw32-gstreamer-plugins-good
• media libraries: mingw32-wavpack, mingw32-flac, mingw32-libvorbis}, mingw32-libwebp

Things we should be able to compile from the mingw shell:

• x264
• ffmpeg
• lame, libmad, aac

Potential problems:

• Python is likely to be a big problem, see: cross and native build of python for mingw* hosts - this may help: mingw-and-ndk
• all the setuptools bits (PyOpenGL, Cython, numpy, lz4, pycrypto, Pillow...
• packaging... could be tricky too

Maybe we can mix cross-compilation and native versions?
Or maybe we should start considering moving to Python 3 + native clients?, as this may well be easier to build with a cross compiler (see python-mingw)

[totaam]

OK, here are a few things that do build. Loosely based on the list of dependencies found here for OSX builds.
Unless otherwise stated, just run mingw32-configure && make && make install in the source:

• yasm (needed by ffmpeg)
• x264: add --enable-shared --enable-static --enable-win32thread
• lame
• libmad: remove --fforce-mem from the CFLAGS in the Makefile to avoid: unrecognized command line option '-fforce-mem'
• faac: add --without-mp4v2
• speex
• gst-plugins-ugly
• gst-plugins-bad: add --disable-dccp --disable-librfb
• ffmpeg:

wget http://www.ffmpeg.org/releases/ffmpeg-2.2.1.tar.bz2
tar -jxf ffmpeg-2.2.1.tar.bz2
cd ffmpeg-2.2.1
./configure \
    --prefix=/usr/i686-w64-mingw32/sys-root/mingw
    --arch=x86 --target-os=mingw32 --enable-cross-compile \
    --enable-runtime-cpudetect --enable-memalign-hack --enable-gpl \
    --disable-everything \
    --enable-swscale --enable-libx264 --enable-decoder=h264 \
    --enable-libvpx --enable-decoder=vp8 --enable-decoder=vp9 \
    --enable-decoder=hevc \
    --disable-protocol=tcp --disable-protocol=rtp \
    --disable-filter=aformat --disable-filter=crop --disable-filter=setpts \
    --disable-filter=anull --disable-filter=format --disable-filter=trim \
    --disable-filter=atrim --disable-filter=null \
    --disable-programs --disable-avfilter
make && make install

Things that do not build (yet?):

• twolame
• faad2
• libvpx

[totaam]

2014-04-21 10:14:22: totaam uploaded file Python-2.7.6-1_mingw.diff (142.8 KiB)

patch for cross compiling Python 2.7.6 with mingw

[totaam]

With the patch above (found here - see also: cross compiling python for windows with mingw32), I can compile Python 2.7.6! (only it didn't find windres so I edited the Makefile to point it to /usr/bin/i686-w64-mingw32-windres)
Then we Configure wine to be able to run the new Python.exe. We can then install setuptools:

sudo /usr/i686-w64-mingw32/sys-root/mingw/bin/python.exe ./setup.py install

Find out that Python has been built without SSL support ("unknown url type: https") and install mingw32-openssl to rebuild it with https support...

Then we can proceed with the python dependencies using the newly installed /opt/i686-w64-mingw32/sys-root/mingw/bin/easy_install.exe and add: pyasn1, lz4. (the others need more work..)

[totaam]

Trying (and failing) to build a simple extension (netifaces):

• we need to patch this header:

--- /opt/i686-w64-mingw32/sys-root/mingw/include/python2.7/pyconfig.h
+++ /opt/i686-w64-mingw32/sys-root/mingw/include/python2.7/pyconfig.h
@@ -1256,7 +1256,7 @@
 /* #undef size_t */
 
 /* Define to `int' if <sys/socket.hdoes not define. */
-#define socklen_t int
+// #define socklen_t int
 
 /* Define to `int' if <sys/types.hdoesn't define. */
 #define uid_t int

• tell setuptools where to find the compiler:

CC=/usr/bin/i686-w64-mingw32-gcc

Then I can try to run the build:

/opt/i686-w64-mingw32/sys-root/mingw/bin/python.exe ./setup.py build --compiler=mingw32

Which fails to find Python.h... and adding it by hand for now only fails a little bit later... Copying all the headers to the source directory is easier than fighting wine / unix paths, but it eventually still fails.

[totaam]

2014-04-27 13:56:14: antoine commented

---

This is such a pain that maybe #90 is a better way forward?

One problem with this solution is that GTK3 binaries for MS Windows are almost as out of date as the GTK2 version (over 2 years without update..), see GTK Download for Windows:

• GTK+ 3.6.4 (current release is 3.12)
• libpng 1.5.14 (see update libpng and libjpeg in binary builds #544 for why this is a very bad thing for us)

So even with this one, we will have to try to rebuild a fair amount of stuff.
There are some newer builds here: [http://win32builder.gnome.org/], but no idea how well they work and how official they are... what a mess.

[totaam]

I think building pygtk2 from source will just be too hard (cross compiling is a no go because of all the extensions that need to be compiled - already hard enough even without the cross compiling part), especially if we need to update some of the libraries for security reasons (ie: libpng.

So I am closing this as WONTFIX for now, hoping that we can move win32 clients to GTK3 (#90).

[totaam]

Re-opening, GTK3 is going nowhere fast.

[totaam]

2016-04-25 11:30:25: antoine commented

---

@smo: hexchat build?

[totaam]

There are 3 tickets that should be consolidated into one: #917, #678 and #300

[totaam]

Will follow up in #678.