Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

html5 client improvements: refactoring, mpeg4, scrolling, etc #1341

Closed
totaam opened this issue Oct 19, 2016 · 30 comments
Closed

html5 client improvements: refactoring, mpeg4, scrolling, etc #1341

totaam opened this issue Oct 19, 2016 · 30 comments

Comments

@totaam
Copy link
Collaborator

totaam commented Oct 19, 2016 •
edited
Loading

See also:
@totaam
Copy link
Collaborator Author

totaam commented Oct 20, 2016 •
edited
Loading

Updates:

Still todo:

  • make forge.js optional so we don't load it unless we need it

@totaam
Copy link
Collaborator Author

totaam commented Oct 31, 2016 •
edited
Loading

Updates:

  • rename scripts: r14270
  • identation: tabs vs spaces, r14282 + r14293
  • move generic functions to add "Utilities.js", split initialization code into functions mirroring the python code: r14292
  • connect page and options parsing: r14243, r14255 + r14256, r14278
  • fixes: r14247
  • allow insecure passwords: r14253 (and remove sound log spam)
  • add support for "window-move-resize" and "configure-override-redirect": r14279
  • notifications support: r14281
  • empty "desktop-size" handler (avoids warnings): r14280

@totaam
Copy link
Collaborator Author

totaam commented Nov 3, 2016 •
edited
Loading

r14387 enables scrolling, but this won't fire as it is because h264 is still disabled by default, maybe mpeg4 video needs doing first? (#1107)

@totaam
Copy link
Collaborator Author

totaam commented Nov 3, 2016

2016-11-03 17:53:31: antoine uploaded file html5-mpeg4video.patch (4.6 KiB)

try to use the mpeg4 video from #1107

@totaam
Copy link
Collaborator Author

totaam commented Nov 4, 2016

2016-11-04 18:25:21: antoine uploaded file html5-mpeg4video-v2.patch (7.3 KiB)

updated patch

@totaam
Copy link
Collaborator Author

totaam commented Nov 4, 2016 •
edited
Loading

Useful pointers for html5 video:

Not video related: browser feature compat table.

@totaam
Copy link
Collaborator Author

totaam commented Nov 5, 2016

2016-11-05 07:00:19: antoine uploaded file html5-mpeg4video-v3.patch (44.7 KiB)

this works! (on Firefox, with one frame of latency, no cleanup code, etc..)

@totaam
Copy link
Collaborator Author

totaam commented Nov 5, 2016

2016-11-05 12:54:27: antoine uploaded file html5-mp3.patch (8.1 KiB)

support mp3 sound via MediaSource (google chrome only...)

@totaam
Copy link
Collaborator Author

totaam commented Nov 7, 2016

2016-11-07 13:32:19: antoine uploaded file html5-sound.patch (6.2 KiB)

try more codecs to attempt to play in all browsers

@totaam
Copy link
Collaborator Author

totaam commented Nov 7, 2016

2016-11-07 13:33:44: antoine uploaded file html5-mediasource.patch (16.2 KiB)

video and audio through mediasource api

@totaam
Copy link
Collaborator Author

totaam commented Nov 10, 2016

2016-11-10 10:52:49: antoine uploaded file html5-mediasource-v2.patch (22.3 KiB)

gives 3 working audio codecs with google-chrome, still none with Firefox

@totaam
Copy link
Collaborator Author

totaam commented Nov 11, 2016

2016-11-11 15:23:01: antoine uploaded file flush-video-frame.patch (3.9 KiB)

ugly PoC hack for flushing frames from the ffmpeg encoder

@totaam
Copy link
Collaborator Author

totaam commented Nov 11, 2016

2016-11-11 15:35:52: antoine uploaded file html5-mediasource-v4.patch (75.4 KiB)

updates: buffer audio (now works with firefox), flush video (video shown is up to date)

@totaam
Copy link
Collaborator Author

totaam commented Nov 11, 2016 •
edited
Loading

As of the patch above:

  • sound works on both google-chrome and Firefox
  • video no longer lags by one frame (see hacky temporary flush-vdeo-frame patch above)

Still TODO:

  • re-work the frame flush hack: maybe merge it with the b-frame flush code? (same intent - but needs a full frame in memory, regular flush does not - regular flush could be converted to that and made more reliable as a result: no more errors on flushing IDR frame 0)
  • choose mp3 for firefox, vorbis for chrome by default?
  • only enable video encodings for browsers that are known to work (just Firefox for now?)
  • let users select the video encoding on the advanced connection tab
  • ensure we call the frame decode callbacks once the frame is decoded (keep them in a queue and find an event that fires: "updateend"? maybe schedule a timer to ensure we don't lose any?)
  • fix ffmpeg h264 low quality (bandwidth settings too low?)
  • fix video with google-chrome, display more than just the first frame! (no idea how, shows "Video rendering in low delay mode" and "pipeline_state=kPlaying")
  • try to figure out the audio latency and report it back to the server for av-sync
  • use workers for processing packets
  • canvas2d: pass alpha flag? use webgl?
    etc

PS: using XPRA_SAVE_TO_FILE=1 then using ffmpeg -i test-vp8+webm-4.webm out%03d.png I can inspect individual frames as PNG and see that the stream is always up to date. So the problem is due to video buffering on the client side...

@totaam
Copy link
Collaborator Author

totaam commented Nov 14, 2016 •
edited
Loading

Large update for handling sound via mediasource with aurora as fallback done in r14412 + r14413 + r14414 (mostly svn subclipse client f--- up!)

The video still lags, but this is not a problem with the video, the same happens with jpeg or png with Firefox! (the patch above just happened to work most of the time by repainting unnecessarily)

@totaam
Copy link
Collaborator Author

totaam commented Nov 14, 2016

2016-11-14 17:08:25: antoine uploaded file html5-mpeg4video-v4.patch (59.3 KiB)

patch updated for already merged audio bits

@totaam
Copy link
Collaborator Author

totaam commented Nov 15, 2016 •
edited
Loading

Improvements and fixes:

  • aurora updated to latest version, added flac support: r14421 + r14422 + r14423
  • native video via mediasource api (experimental): r14427 (see long commit message for details)
  • video related tweaks and improvements: r14426 (oops, incomplete until r14427), r14425, r14428

About the status of the Broadway decoder:

  • got updated to the latest version, a few fixes made to the code
  • decodes frames synchronously which is great
  • still pretty fast, and maybe we can make it faster still by tuning x264 and raising the min-speed (html client only), see Correct x264 encoder settings for Broadway to decode
  • fails for video sizes greater than ~1MPixel - should be fixable:
  • Firefox:
RangeError: invalid array length
Decoder/onPicFun<()                         Decoder.js:138
bound ()                                    self-hosted
_broadwayOnPictureDecoded()                 Decoder.js:63
kb()                                        Decoder.js:69
decode()                                    Decoder.js:329
paint()                                     Window.js:931
XpraClient.prototype._process_draw()        Client.js:1377
XpraClient.prototype._route_packet()        Client.js:339
XpraProtocolWorkerHost.prototype.open/<()   Protocol.js:39
  • chrome:
Decoder.js:138 Uncaught RangeError: Source is too large
    at Uint8Array.set (native)
    at Decoder.<anonymous> (http://localhost:14500/js/lib/broadway/Decoder.js:138:25)
    at _broadwayOnPictureDecoded (http://localhost:14500/js/lib/broadway/Decoder.js:63:53425)
    at Object.kb [as _broadwayPlayStream] (http://localhost:14500/js/lib/broadway/Decoder.js:69:3425)
    at Decoder.decode (http://localhost:14500/js/lib/broadway/Decoder.js:329:16)
    at XpraWindow.paint (http://localhost:14500/js/Window.js:912:25)
    at XpraClient._process_draw (http://localhost:14500/js/Client.js:1377:7)
    at XpraProtocolWorkerHost.XpraClient._route_packet [as packet_handler] (http://localhost:14500/js/Client.js:339:3)
    at Worker.<anonymous> (http://localhost:14500/js/Protocol.js:39:9)

About the mediasource video api:

  • browsers buffer too much with the mediasource api... how can we prevent this?
  • maybe we need frame duration to get firefox to play at a predictable rate?
  • chrome claims to be playing the video (and shows the first 2 frames), but stalls..

Still TODO:

  • offload to worker - important
  • fix broadway decoding errors for >1MPixel
  • invalid server host error is not user friendly
  • callback should tell the server in case of image decoding failures
  • video scaling should be easy to implement using putImageData
  • could add vp8 software decoding using [https://github.com/brion/ogv.js/]

@totaam
Copy link
Collaborator Author

totaam commented Nov 16, 2016 •
edited
Loading

Lots more improvements in r14434:

  • remote logging enabled by default
  • audio-codec validation and fallback
  • fix pointer-events - will IE need another band-aid? [https://github.com/kmewhort/pointer_events_polyfill]
    etc

Notes:

  • improving the host error would mean passing the message from the network worker to the client.close function

@totaam
Copy link
Collaborator Author

totaam commented Nov 17, 2016 •
edited
Loading

More:

@totaam
Copy link
Collaborator Author

totaam commented Nov 18, 2016 •
edited
Loading

Mistake: scaling was only added today in r14444

@totaam
Copy link
Collaborator Author

totaam commented Nov 23, 2016 •
edited
Loading

Important update: r14474 (+ r14476 + r14477 derp fixup) worker stuff, see commit message for details.

Still TODO:

  • is there a better way of adding bytes to the buffer than copying them one-by-one in websocket.onmessage?
  • the special case code for the 'draw' zero-copy packets is ugly, made worse by the "scroll" extra-special case, will probably leave it as it is.. but here are some ideas:
  • send "scroll" draw data using a binary blob (ie: an array of array of ints?) so it can be handled like normal binary pixel data - meh, doesn't solve the "draw" special case
  • forcing the server to send all blobs using raw packets so that we can wrap them all as Uint8Array in the protocol receive codepath, this doesn't solve the special cases in the message queue handlers... - but this can be used already with XPRA_INLINE_SIZE=0 xpra start .., works ok and removes one special case.. (but would also cause backwards compatibility issues with older servers)

@totaam
Copy link
Collaborator Author

totaam commented Nov 24, 2016 •
edited
Loading

Make sure we empty the queues fix: r14480.
Maybe process_receive_queue should be using a while loop too rather than using recursion.

PS: new task: #1372, the html5 client should live in its own package.

More thoughts on the byte-copying: I'm just guessing that this is costly because it was the case with the python code when I had profiled it, many years ago now.
Maybe the javascript engines are really really clever and optimize it somehow? But I doubt it can be as fast as memcpy.. which is already a penalty.
It may be that most clients have enough CPU power to spare to do all this unnecessary copying, but every little saving helps.

That's why I had added code to send the packet header and data to the socket separately when the packets are big enough (r246 - 5 years ago!, 64KB is the threshold in the current code), this avoids concatenating them before sending which causes memory churn. And that's using a fast memcpy for doing the copying!

One way of dealing with this would be to queue the "Uint8Array"s directly, and then work a bit harder to extract the header from them. It sounds complicated, but in almost all cases, the header will be contained in a single array, with the data either appended to it or in the next array element(s).
I'm sure concatenating multiple arrays into one big array is going to be faster than byte-copying.
I'm not sure how much TCP packet aggregation websockets end up doing. It probably depends on all sorts of things from OS network configuration, congestion, browser, etc.
We may need to re-queue left-over data from each process_receive_queue iteration, this can be stored in a dedicated variable.

Another note: do we really need to run the "process_xxxx_queue" timers all the time?
Seems wasteful. Can't we just start scheduling the timers when we add elements to their queue (if not scheduled already) and stop them when the queue becomes empty?

@totaam
Copy link
Collaborator Author

totaam commented Nov 24, 2016 •
edited
Loading

Much improved encoding selection for the html5 client with no video encodings available: r14485.

A bigger problem: paints have been completely broken since r14474.

@totaam
Copy link
Collaborator Author

totaam commented Nov 25, 2016 •
edited
Loading

Paint fixed in r14490, ready for testing.

@totaam
Copy link
Collaborator Author

totaam commented Dec 22, 2016

2016-12-22 00:41:09: afarr commented

Tested some with 1.0 r14570 fedora 23 server - using both Chrome (55) on windows 8.1 and... well, Opera (41) on OSX 10.12.1.

With the default settings this html5 client is really really working well.

There are some probably not related to this ticket issues with scrolling events being lost when the session is shifted to a browser that's significantly smaller than the originally connected one (anecdotally, re-sizing the browser so that the canvas was large enough that the server-side chrome browser's scroll bar was visible, the scroll events began being caught); OSX laptop tracpad events result in dizzyingly & unmanageably fast scrolling (something about the tracpad sending more events with a smaller delta, apparently); and the fact that one has to remember that browser shortcuts, like control-t, will be captured by the local browser rather than being passed through to the server-side applications.

On to more relevant issues to this ticket...

  • Still seeing that Decoder.js:138 Uncaught RangeError: Source is too large error mentioned above when video is enabled... had the server-side browser at about 1600x575px and was apparently still too large a source when scrolling.

  • With video enabled it looks like the h264 events start to back up, and it's not too long before I start seeing Warning: delayed region timeouts on the server side.

2016-12-21 14:51:21,509 client display size is 1749x1106 with 1 screen:
2016-12-21 14:51:21,509   HTML (463x293 mm - DPI: 95x95)
2016-12-21 14:51:21,509     Canvas
2016-12-21 14:51:24,735 Error: expiring 6 missing damage ACKs,
2016-12-21 14:51:24,736  connection may be closed or closing,
2016-12-21 14:51:24,736  sequence numbers missing: 199, 200, 201, 203, 213, 217
2016-12-21 14:51:32,740 Error: expiring 1 missing damage ACK,
2016-12-21 14:51:32,740  connection may be closed or closing,
2016-12-21 14:51:32,740  sequence numbers missing: 220
2016-12-21 14:51:48,959 Warning: delayed region timeout
2016-12-21 14:51:48,960  region is 15 seconds old, will retry - bad connection?
2016-12-21 14:51:48,961  30 late responses:
2016-12-21 14:51:48,961   234 h264: 66s
2016-12-21 14:51:48,961   245 h264: 65s
2016-12-21 14:51:48,961   272 h264: 58s
2016-12-21 14:51:48,961   276 h264: 58s
2016-12-21 14:51:48,961   280 h264: 58s
2016-12-21 14:51:48,961   281 h264: 58s
2016-12-21 14:51:48,961   282 h264: 58s
2016-12-21 14:51:48,961   283 h264: 57s
2016-12-21 14:51:48,961   287 h264: 55s
2016-12-21 14:51:48,962   295 h264: 55s
2016-12-21 14:51:48,962   543 h264: 52s
2016-12-21 14:51:48,962   754 h264: 51s
2016-12-21 14:51:48,962  1022 h264: 49s
2016-12-21 14:51:48,962  1062 h264: 46s
2016-12-21 14:51:48,962  1068 h264: 42s
2016-12-21 14:51:48,962  1083 h264: 39s
2016-12-21 14:51:48,962  1103 h264: 37s
2016-12-21 14:51:48,962  1119 h264: 36s
2016-12-21 14:51:48,962  1120 h264: 36s
2016-12-21 14:51:48,963  1124 h264: 32s
2016-12-21 14:51:48,963  1129 h264: 27s
2016-12-21 14:51:48,963  1134 h264: 27s
2016-12-21 14:51:48,963  1139 h264: 27s
2016-12-21 14:51:48,963  1142 h264: 25s
2016-12-21 14:51:48,963  1147 h264: 24s
2016-12-21 14:51:48,963  1222 h264: 23s
2016-12-21 14:51:48,963  1233 h264: 22s
2016-12-21 14:51:48,963  1259 h264: 22s
2016-12-21 14:51:48,963  1284 h264: 21s
2016-12-21 14:51:48,963  1318 h264: 18s
2016-12-21 14:52:04,050 Warning: delayed region timeout
2016-12-21 14:52:04,051  region is 15 seconds old, will retry - bad connection?
2016-12-21 14:52:04,052  30 late responses:
2016-12-21 14:52:04,052   234 h264: 81s
2016-12-21 14:52:04,053   245 h264: 80s
2016-12-21 14:52:04,053   272 h264: 73s
2016-12-21 14:52:04,053   276 h264: 73s
2016-12-21 14:52:04,054   280 h264: 73s
2016-12-21 14:52:04,054   281 h264: 73s
2016-12-21 14:52:04,054   282 h264: 73s
...

... the painting at this point becomes extremely choppy and the entire session often becomes non-responsive for as much as 10 long seconds (occasionally something like a minute). I'll post a screenshot of a point I got it frozen at.

  • Trying the mp3, it is stable, but does seem to run further and further out of sync, reaching some 20 seconds off after 10-15 minutes of playing (it took about 20 seconds for sound to stop after pausing a video... and then it took about 10-15 seconds to re-start again once the video was started again... at which point sync was just an abstract concept).

  • Trying the vorbis sound I find that the sound only lasts a minute or two before the audio queue apparently starts overflowing and the media stream becomes "undefined" client side.

Client console logs:

startup complete
Client.js:123 audio media source open
Client.js:123 using audio codec string for vorbis+mka: audio/webm; codecs="vorbis"
Client.js:123 audio: requesting vorbis+mka stream from the server
Client.js:588 server connection is OK
Client.js:123 audio start of mediasource vorbis+mka stream
Client.js:958 audio play!
Client.js:123 close_video: video_source_buffer=undefined, media_source=undefined, video=undefined

Server logs:

2016-12-21 12:00:04,075 sound source using audio codec vorbis
2016-12-21 12:00:04,075 sound source using container format matroska
[31605:31656:1221/120010.466497:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120010.468056:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120010.468322:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120010.468485:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120010.468614:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120010.468749:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
2016-12-21 12:00:11,156 client 1: close_video: video_source_buffer=undefined, media_source=undefined, video=undefined
[1:1:1221/120201.455646:ERROR:KeyboardEventManager.cpp(424)] Not implemented reached in static bool blink::KeyboardEventManager::currentCapsLockState()
2016-12-21 12:03:42,130 client 1: close_video: video_source_buffer=undefined, media_source=undefined, video=undefined
2016-12-21 12:03:53,866 client 1: audio queue overflowing: 250, stopping
2016-12-21 12:03:53,867 client 1: close_audio: audio_source_buffer=[object SourceBuffer], media_source=[object MediaSource], video=[object HTMLAudioElement]
2016-12-21 12:03:53,873 sound source stopping
2016-12-21 12:03:54,842 client 1: audio error
2016-12-21 12:03:54,843 client 1: close_audio: audio_source_buffer=null, media_source=null, video=null
[31605:31656:1221/120408.173414:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120408.173835:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120408.173895:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120408.173930:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120408.173970:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
[31605:31656:1221/120408.174002:ERROR:browser_gpu_channel_host_factory.cc(113)] Failed to launch GPU process.
2016-12-21 12:04:08,835 client 1: close_video: video_source_buffer=undefined, media_source=undefined, video=undefined
2016-12-21 12:04:59,822 client 1: close_video: video_source_buffer=undefined, media_source=undefined, video=undefined

Refreshing the browser tab that is connecting to the server, the sound will restart... but it crashes soon after again.

  • Trying with the opus codec I'm promptly informed in server logs that opus isn't supported on my server and it fails over to vorbis, and behaves as above.

  • Didn't try wav yet.

Not sure if you'd like any particular logs from the server side for any of these issues... or if you're ready to start making tickets for things like tracpads or non-responsive drop menu buttons in a javascript dialog box.

I'll re-assign this to you to look at and let me know what you want to pursue here and what you might want broken out into a more specific ticket.

@totaam
Copy link
Collaborator Author

totaam commented Dec 22, 2016

2016-12-22 00:42:09: afarr uploaded file ticket1341_chrome-with-video-enabled_scrolling-arstechnica_1600x575px-server-side-browser.PNG (479.0 KiB)

Screenshot of scrolling with video
ticket1341_chrome-with-video-enabled_scrolling-arstechnica_1600x575px-server-side-browser.PNG

@totaam
Copy link
Collaborator Author

totaam commented Dec 27, 2016 •
edited
Loading

  • the sound issues should be tracked in #845: see #845#comment:21.
  • scroll events: I will try to reproduce
  • osx trackpad handling: moved to osx trackpad sends too many events with HTML5 client #1390
  • one has to remember that browser shortcuts, like control-t, will be captured by the local browser rather than being passed through to the server-side applications : we try to capture as many of the shortcuts as we can, but chrome does not allow control-t to be intercepted: Ctrl keys: Ctrl-N, Ctrl-T and Ctrl-W no longer available to client side JavaScript in Chrome4 (available in FF3.x, IE7/8 and Chrome3)
  • video issues: please report OS / browser / native vs legacy video support - which will vary a lot.
  • Uncaught RangeError: Source is too large is only a problem with the non-native decoding - and I'm still not sure what can be done about this, r14617 will catch those errors and report them to the server which will then re-send the frame as png or jpeg, so at least the screen updates won't freeze - this probably explains the delayed region timeouts

@totaam
Copy link
Collaborator Author

totaam commented Jan 27, 2017

2017-01-27 00:41:52: afarr commented

Managed to make some time to test with both OSX 10.12.1 and Windows 8.1 against a 1.0.2 14823 fedora 25 server.

As mentioned in #845, the sound seems to have no real problems aside from the poor network I'm using.

Testing with Chrome 55 and Opera 42 on the OSX and Chrome 56 (beta) on the Windows 8.1, I saw about the same performance on all. Wav would cut out within seconds (not hugely surprisingly); while the opus and vorbis both performed similarly, remaining stable until the client hit spinners and a ping timeout after about 3-5 minutes; and the mp3 (no mention of legacy) seemed to just run and run and clients didn't ping out for 10-20 minutes with that sound running.

As a comparison, I connected an OSX 1.0.1 14723 client (with default presumably vorbis codec) and it took about the same 3-5 minutes to ping timeout.

I did run into a not-at-all-verbose sound error with the Windows 8.1 & Chrome 56, but wasn't able to reproduce at all.

2017-01-24 15:11:35,121 client 14: audio queue overflowing: 250, stopping
2017-01-24 15:11:35,126 client 14: close_audio: audio_source_buffer=[object SourceBuffer], media_source=[object MediaSource], video=[object HTMLAudioElement]
2017-01-24 15:11:35,129 client 14: audio error
2017-01-24 15:11:35,131 client 14: close_audio: audio_source_buffer=null, media_source=null, video=null
2017-01-24 15:11:35,143 sound source stopping

The scrolling, meanwhile, was very good with the defaults, but once I enabled experimental video (h264 I presume) and scrolled the canvas seemed to 'break up into little boxes', each of which seemed to be updating separately... presumably the scrolling code having issues... and I began seeing the 'delayed region timeout' errors. No sign of any new errors though.

I'll try to get a 2.0 build working so I can test that a bit too.

@totaam
Copy link
Collaborator Author

totaam commented Jan 31, 2017

2017-01-31 01:41:46: afarr commented

Ok, with a little help from Smo, got the 2.0 (14909) server running and, as a bonus, seem to have upgraded to a less bandwidth-light network.

With that said, made some time to test sound with OSX 10.12.1 and Chrome 56 (the beta).

Looks like it's much more stable with all the codecs (well, didn't make time to try wav yet).

Vorbis and opus seemed about as stable as the (non-legacy) mp3 (vorbis didn't even timeout, must've found a hole in the bandwidth usage crowds).

Didn't manage to try with other browsers on OSX or any on windows ... but I'd be very surprised, after what I saw, to have them behave any differently (well, except maybe better).

Watching my ping times as I was testing, it looked like most of the issues I did see corresponded to super-jitter spikes and timeouts... like:

64 bytes from 10.0.32.138: icmp_seq=517 ttl=64 time=20.302 ms
Request timeout for icmp_seq 518
64 bytes from 10.0.32.138: icmp_seq=519 ttl=64 time=24.950 ms
64 bytes from 10.0.32.138: icmp_seq=520 ttl=64 time=21.777 ms
64 bytes from 10.0.32.138: icmp_seq=521 ttl=64 time=22.435 ms
64 bytes from 10.0.32.138: icmp_seq=522 ttl=64 time=22.710 ms
64 bytes from 10.0.32.138: icmp_seq=523 ttl=64 time=25.820 ms
64 bytes from 10.0.32.138: icmp_seq=524 ttl=64 time=29.762 ms
64 bytes from 10.0.32.138: icmp_seq=525 ttl=64 time=22.227 ms
Request timeout for icmp_seq 526
64 bytes from 10.0.32.138: icmp_seq=526 ttl=64 time=1012.982 ms
64 bytes from 10.0.32.138: icmp_seq=527 ttl=64 time=50.985 ms
64 bytes from 10.0.32.138: icmp_seq=528 ttl=64 time=231.962 ms

So that seems stable as is.

Turning the video encoding on, I'm still seeing (expected I expect) the RangeError issue, though rather pretty and regular.

2017-01-30 17:16:33,314 Warning: client decoding error:
2017-01-30 17:16:33,315  
2017-01-30 17:16:33,315 client 1: error painting
2017-01-30 17:16:33,316 client 1: h264
2017-01-30 17:16:33,316 client 1: RangeError: Source is too large

I did manage to trigger one interesting error, but otherwise the behavior seems to be exactly as with the 1.0.2 server (canvas breaking up into little boxes when scrolling, etc).

2017-01-30 17:16:33,337 Warning: client decoding error:
2017-01-30 17:16:33,338  
2017-01-30 17:16:33,338 Error: failed to encode x264 video frame:
2017-01-30 17:16:33,338  image pixels does not have 3 planes! (found 921600)
2017-01-30 17:16:33,338  source: XShmImageWrapper(BGRX: 20, 131, 640, 360)
2017-01-30 17:16:33,339  options: {'av-sync': True, 'scroll': True}
2017-01-30 17:16:33,339  encoder:
2017-01-30 17:16:33,339    profile             : baseline
2017-01-30 17:16:33,339    generation          : 247
2017-01-30 17:16:33,339    delayed             : 0
2017-01-30 17:16:33,339    height              : 360
2017-01-30 17:16:33,340    max-size            : (8192, 4096)
2017-01-30 17:16:33,340    preset              : medium
2017-01-30 17:16:33,340    frames              : 0
2017-01-30 17:16:33,340    quality             : 58
2017-01-30 17:16:33,340    lossless            : False
2017-01-30 17:16:33,340    frame-types         : {}
2017-01-30 17:16:33,340    width               : 640
2017-01-30 17:16:33,340    speed               : 34
2017-01-30 17:16:33,341    source              : unknown
2017-01-30 17:16:33,341    version             : 148
2017-01-30 17:16:33,341    tune                : zerolatency
2017-01-30 17:16:33,341    src_format          : YUV420P
2017-01-30 17:16:33,341    formats             : ['YUV422P', 'RGB', 'BGRX', 'BGR', 'YUV420P', 'BGRA', 'YUV444P']
2017-01-30 17:16:33,341    b-frames            : 0
2017-01-30 17:16:33,354 client 1: error painting
2017-01-30 17:16:33,355 client 1: h264
2017-01-30 17:16:33,355 client 1: RangeError: Source is too large
2017-01-30 17:16:33,356 Warning: client decoding error:
2017-01-30 17:16:33,356  
2017-01-30 17:16:33,356 client 1: error painting
2017-01-30 17:16:33,357 client 1: h264
2017-01-30 17:16:33,357 client 1: RangeError: Source is too large

I suspect that, once that new issue is looked at, this ticket might be able to be closed and replaced by specific tickets for the video and such... unless you'd like me to check with firefox as a client first?

@totaam
Copy link
Collaborator Author

totaam commented Jan 31, 2017 •
edited
Loading

  • let's deal with the sound stuff in #845 - and we may just be able to close that one too
  • the RangeError: Source is too large stuff is expected - maybe Dave can help us fix that?
  • Error: failed to encode x264 video frame: had been seen before (see Seeing avcodec errors when playing video #1346#comment:9), it looks like a race condition caused by the client decoding error - maybe r14916 will help us avoid it (but maybe not in all cases..), either way - let's not worry too much about this one unless we start hitting it under "normal" conditions

Let's follow up in #1424, #1463

@totaam totaam closed this as completed Jan 31, 2017
This was referenced Jan 22, 2021
Closed
Closed
Closed
Closed
Open
@totaam totaam mentioned this issue May 22, 2021
Closed
@totaam totaam mentioned this issue May 4, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@totaam