>= Qt-5.10.1 everywhere: pre-requisite for HTTP2 in our packages

[michaelstingl]

After releasing 2.3.3 client with Qt-5.6.2 for Win, Mac and Linux we plan to release a 2.4.0 client with many new features soon.

We plan a smaller 2.5.0 release with Qt-5.10.1 for all platforms:

• Linux
• Windows
• macOS

We can try to tweak our build infrastructure to make (experimental) Qt-5.9.x builds available for testing earlier. I will check with @jnweiger …

[tessus]

I'd be happy to test a client 2.3.3 or 2.4 with QT-5.9.x and http2.

[michaelstingl]

@tessus Thank you for your offer to help testing. Make sure you are on the testpilot mailing list.

At the moment, ownCloud automated build systems can't build 2.3.3 or 2.4 with QT-5.9.x. We plan a ownCloud-2.5.0.????-nightly* with QT-5.9.x after the 2.4.0 release.

As a workaround, you could build your own 2.3.3 or 2.4 with QT-5.9.x.

[tessus]

At the moment, ownCloud automated build systems can't build 2.3.3 or 2.4 with QT-5.9.x. We plan a ownCloud-2.5.0.????-nightly* with QT-5.9.x after the 2.4.0 release.

I didn't say you should create automated builds every day. I asked for one build for 2.3.3 (and maybe another one with 2.4) with QT 5.9.x
Even a manual build should be an easy task on your build machines.

As a workaround, you could build your own 2.3.3 or 2.4 with QT-5.9.x.

I do not have the resources. That's my whole point. Otherwise I would do it gladly.

[tessus]

It seems that 2.5.0 won't be released for another few months, thus I really ask you guys to provide a separate 2.4.0 version with QT 5.9.x as well.

I really think that it should be easy with your build infrastructure to create one additional package once. As mentioned before I don't have the resources of 250GB to compile QT and the client.

[michaelstingl]

@tessus I'm very sorry, there is some delay. But it's still the plan to provide first 2.5.0 pre-release builds with QT 5.9.x short after the 2.4.0 release. (2.4.0 is in beta) So at least, there should be builds available for testing in the not so far future.

[tessus]

Well, a 2.5.0 pre-release is not a stable version or am I wrong?

[guruz]

Have not investigated deeper but we should check it:

<guruz> from which Qt release is HTTP2 considered good and stable?
<annulen> surely not before 5.10.1
<guruz> annulen: really? :)
<annulen> I can list you bugs which prevent using it in QtWebKit
<annulen> but for controlled server earlier versions may be fine too
<annulen> QTBUG-64359, QTBUG-63722, QTBUG-64721
<qt_gerrit> QTBUG-64359, Facebook login fails with enabled HTTP/2 but works otherwise - https://bugreports.qt.io/browse/QTBUG-64359 (Closed)
<qt_gerrit> QTBUG-64359, Slow Requests Using HTTP2 - https://bugreports.qt.io/browse/QTBUG-63722 (Closed)
<qt_gerrit> QTBUG-64359, HTTP/2 request with invalid host name does not end in HostNotFound error - https://bugreports.qt.io/browse/QTBUG-64721 (Closed)
<annulen> all recent stuff
<annulen> you should also be aware of QTBUG-61397 with Qt < 5.10
<qt_gerrit> annulen: HTTP/2: connection to http:// host (that does not support http2) is always failing - https://bugreports.qt.io/browse/QTBUG-61397 (Closed)
<annulen> also I'm going to disable HTTP/2 for OpenSSL < 1.0.2 (i.e. lacking NPN) because of misbehaving server found in QTBUG-65849
<qt_gerrit> annulen: HTTP/2: Invalid frame size - https://bugreports.qt.io/browse/QTBUG-65849 (Closed)
<annulen> also, you cannot negotiate it on macOS with SecureTransport, because it doesn't allow controlling ALPN, so only direct connections are possible
<annulen> if you were asking if 5.10.1 is really bulletproof or it has more hidden bugs, idk

Thanks @annulen

[tessus]

That's just great. I would like to place a bet: I bet the client won't have http2 before 2020.

[annulen]

FWIW, if you limit what server-side implementation of HTTP/2 are supported, you can dodge misbehaving server cases like in QTBUG-65849, and probably QTBUG-63722. QTBUG-64359 is not a problem if you are not sending multiple Set-Cookie header in same request (but is a showstopper otheriwse). QTBUG-61397 is not a problem if you enable HTTP/2 only for https

[tessus]

Using http2 on non-https connections is idiotic anyway. I'm not even sure, if h2c is supported by all browsers.
Furthermore, even considering http connections (and related bugs) is a waste of time. Who in their right mind would run a cloud system with http these days? Sorry to be blunt, but developers should concentrate their efforts towards sane setups.

You know, I could probably drive my car with 4 flat tires for a while too. But I am sure that nobody would ask someone to actually mount 4 flat tires.

[annulen]

Ok, if you are fine with broken 'http://' when http/2 is enabled, you are left with just 3 bugs. Oh, and QTBUG-64359 is fixed in 5.9.4

[tessus]

I'm fine with broken http. However I'm not fine with broken https. As mentioned several times in this thread (and a few others I think), I do not have enough free space to compile Qt and the client myself, thus I depend on the official binaries, which in turn tells me that this is what is going to happen:

In a few months, the devs will use Qt 5.9.2 and most likely deactivate the http2 code path due to the bugs in Qt.
In a year or 2 the devs will finally use a Qt version that does not have any serious bugs when it comes to https and http2. If lucky, devs will activate the http2 code path again.
So realistically I will have a client that works with http2 in 2020.

P.S.: please also see #5470 (comment). So my estimate of 4 years to create a http2 capable client are not off. :-(

[jnweiger]

@michaelstingl Should we relabel this ticket to say "Qt-5.10.x everywhere ..." ? It seems 5.9.x is not the HTTP2 prerequisite we need.

@tessus There is no motivation in what you say. I'll ignore that for now. Sorry.

[michaelstingl]

@guruz @ogoffart What is your preferred Qt for the final 2.5.0 ?

[ogoffart]

@michaelstingl 5.10.x would be the best, independently of http/2

[guruz]

(At least qt 5.9.2 seems to have issues with http2... #6285 (comment) .. or that's on server side but i guess not?)

[annulen]

This might be caused by https://bugreports.qt.io/browse/QTBUG-63722, I didn't hit it myself but there was report http://lists.qt-project.org/pipermail/interest/2018-January/029247.html

So, better check with 5.10.1

[guruz]

@annulen thanks, i guess it was 64359 (#6285 (comment))

[jnweiger]

@guruz @annulen http://download.qt.io/official_releases/qt/5.10/ does not have 5.10.1 source modules. Do you have a recommendation where to download them?

[ogoffart]

@jnweiger 5.10.1 is not yet released. In the mean time, you can prepare the build with 5.10.0, and upgrading to 5.10.1 should be easy.

[jnweiger]

@ogoffart okay. So 5.10.0 is correct for now. Builds for Linux with 5.10.0 are ready for test: https://build.opensuse.org/package/show/isv:ownCloud:testpilot:daily:master/testpilotcloud-client

[jnweiger]

5.10.1 arrived in SUSE OBS: https://build.opensuse.org/project/monitor/isv:ownCloud:Qt5101

[guruz]

Better macOS package here, the other one might be broken on some OS:
https://download.owncloud.com/desktop/testing/ownCloud-qt5.10.1-2.4.2.951824.pkg

[tessus]

Awesome, just came back from vacation (w/o Internet) and there it was. The long awaited h2 client.

Already running it on my system and it looks great. The connections are done via h2 and I haven't seen any issues so far.

Thank you !!!

[dschmidt]

@tessus Glad to hear! Just out of curiosity, which build(s) did you test?

[tessus]

@dschmidt sorry, my bad. I've always talked about macOS thus I never considered adding that info in my last comment.

I used the link @guruz provided:

https://download.owncloud.com/desktop/testing/ownCloud-qt5.10.1-2.4.2.951824.pkg

Update: double my bad. I never mentioned macOS in this ticket. :-) shame on me.

[dschmidt]

Alright, thanks for the feedback!

[tessus]

I've noticed a strange request in my Apache server-status:

Protocol	VHost	Request
http/1.1	REMOVED:443	PROPFIND /remote.php/dav/files/REMOVED/ HTTP/2.0
h2	REMOVED:443	done, streams: 0/1/1/0/0 (open/recv/resp/push/rst)

The first request does not make any sense. It shows that protocol http/1.1 is used, yet it also indicates HTTP/2.0 in the request itself.

Has someone else seen this as well?

[guruz]

@tessus Try to see it in a log?

SERVER_PROTOCOL	The protocol used by the request (e.g. HTTP/1.1). In some types of internal subrequests, this variable has the value INCLUDED.

https://httpd.apache.org/docs/trunk/expr.html#vars

Also, when you click the 🔒 button in the desktop client UI, do you see an entry for HTTP/2?

[tessus]

Try to see it in a log

Hmm, unfortunately this is not possible. I'm filtering out PROPFIND and they make it never into any log on my server. That's because these entries flooded my logs (every 1-5 seconds).

when you click the 🔒 button in the desktop client UI, do you see an entry for HTTP/2?

Yes, I do.

[yunlhan]

I tried to build the windows client through an opensuse dev enironment as documented here, no modification of build.sh. However the client I built is with Qt 5.6.2 and very old openssl 1.0.2h. The official client is with a newer Qt version and openssl 1.1g. What is the recommended modification of build script to build against the lastest Qt and openssl? Thanks.

[ogoffart]

@yunlhan Don't use the cross build, use the native build: https://doc.owncloud.org/desktop/2.3/building.html#windows-development-build
You can use either mingw or msvc. 2.5 packages will likely be compiled with MSVC.

[dschmidt]

@yunlhan Yes, you are right. Maintaining the cross build infrastructure was a lot of work, that's one of the reasons we're switching to msvc. That's also why we won't update it to 5.10 anymore.
I'm sorry that there is no documentation that's as easy and complete as the cross build yet. It might help (or confuse you more) to look at the appveyor.yml in our root folder. We use KDE's craft to install our dependencies. There's absolutely no need to use it though.

[yunlhan]

@ogoffart @dschmidt Thank you guys.

If I go for development build, does that mean I have to compile for each machine I have? Say I have two windows machines, then I have to set up the environment and build the client on each of them?

Also, is it because client version 2.4+ and 2.5 with Qt 5.10 are still in development, the documentation does not keep up with the latest github code? Is there any plan to update the build documentation to reflect the fact that MSVC is being used?

[dschmidt]

Yes, of course - we will update the build documentation at some point.

You don't need to build it on every machine, if you use KDE craft you can possibly just use the portable packager and unzip it on every machine. Unfortunately you cannot use our old NSIS scripts for a native build in the current state, as it has all sorts of cross building paths hardcoded. The new MSI scripts are not public currently.

cc @michaelstingl

[yunlhan]

@dschmidt Thank you! I am looking forward to the newly updated documentation and MSI building scripts in the repo.