out of memory / gl_paint_planar error and eventual client crash #1111

totaam · 2016-02-04T01:55:12Z

Issue migrated from trac ticket # 1111

component: client | priority: critical | resolution: fixed | keywords: intel opengl

2016-02-04 01:55:12: afarr created the issue

Before I mention anything more, the answer is yes: I am forcing --opengl=on with an Intel HD 4000 graphics card... and I do suspect that may be the underlying culprit here.

Reporting in case there might be more to it... and so that others may be able to find the answer (not to mention adding to the list of reasons Intel should, potentially, address their drivers).

0.17.0 r11776 win32 client against 0.17.0 r11814 fedora 23 server.

Server side, I had gotten a number of Warning: client decoding error: failed to get a gl context errors (which I half ignored) a few hours before the crash.

A couple of hours later, I got the following message half a dozen times, before the client crashed:
2016-02-03 17:18:53,201 Warning: client decoding error: gl_paint_planar error: GLError( err=1285, description # 'out of memory', baseOperationglTexSubImage2D )
2016-02-03 17:19:13,600 Warning: delayed region timeout
2016-02-03 17:19:13,601  region is 15 seconds old, will retry - bad connection?
Client side, I got a spew of the same traceback over and over again before crash:
Traceback (most recent call last):
  File "xpra\client\gl\gl_window_backing_base.pyc", line 730, in gl_paint_planar
  File "xpra\client\gl\gl_window_backing_base.pyc", line 790, in update_planar_textures
  File "latebind.pyx", line 32, in OpenGL_accelerate.latebind.LateBind.__call__ (c:\Users\mcfletch\OpenGL-dev\OpenGL-ctypes\OpenGL_accelerate\src\latebind.c:989)
  File "wrapper.pyx", line 318, in OpenGL_accelerate.wrapper.Wrapper.__call__ (c:\Users\mcfletch\OpenGL-dev\OpenGL-ctypes\OpenGL_accelerate\src\wrapper.c:6561)
GLError: GLError(
        err = 1285,
        description = 'out of memory',
        baseOperation = glTexSubImage2D,
        pyArgs = (
                GL_TEXTURE_RECTANGLE_ARB,
                0,
                0,
                0,
                640,
                362,
                GL_LUMINANCE,
                GL_UNSIGNED_BYTE,
                <memory at 0x08542530>,
        ),
        cArgs = (
                GL_TEXTURE_RECTANGLE_ARB,
                0,
                0,
                0,
                640,
                362,
                GL_LUMINANCE,
                GL_UNSIGNED_BYTE,
                <memory at 0x08542530>,
        ),
        cArguments = (
                GL_TEXTURE_RECTANGLE_ARB,
                0,
                0,
                0,
                640,
                362,
                GL_LUMINANCE,
                GL_UNSIGNED_BYTE,
                <memory at 0x08542530>,
        )
)
2016-02-03 17:19:09,612 screen size change: will reinit the windows
2016-02-03 17:19:14,051 WM_DWMCOMPOSITIONCHANGED: Desktop Window Manager composition has been enabled or disabled: 0 / 0

(Xpra:10952): GtkGLExt-WARNING **: cannot create GdkGLContext

Traceback (most recent call last):
  File "xpra\client\gl\gtk2\gl_window_backing.pyc", line 40, in gl_expose_event
  File "xpra\client\gl\gtk_compat.pyc", line 91, in __enter__
TypeError: Gdk.Drawable.gl_begin() argument 1 must be gtk.gdkgl.Context, not None
2016-02-03 17:19:34,418 sound output stopping
If this looks like something other than just another reason why the opengl is greylisted for Intel graphics cards, let me know if there's any flags that might be of use to get any more details (I might be able to repro). If not, well... I guess it can be closed promptly.

The text was updated successfully, but these errors were encountered:

totaam · 2016-02-04T02:27:55Z

2016-02-04 02:27:55: antoine changed owner from antoine to afarr

totaam · 2016-02-04T02:27:55Z

2016-02-04 02:27:55: antoine commented

The cause is pretty obvious in the stacktrace: out of memory.

The real question is: where is all the memory gone? Do we have a leak?

totaam · 2016-02-27T13:27:45Z

2016-02-27 13:27:45: antoine changed priority from major to critical

totaam · 2016-02-27T13:27:45Z

2016-02-27 13:27:45: antoine commented

Raising - a memory leak would be bad!

totaam · 2016-05-18T01:34:11Z

2016-05-18 01:34:11: afarr changed owner from afarr to antoine

totaam · 2016-05-18T01:34:11Z

2016-05-18 01:34:11: afarr commented

Set up a new fedora 23 vm with only 512 MB of memory, and tried to repeat/repro the out of memory error with no luck.

Tested using 0.18.0 r12578 windows client (your repo) with --opengl=on against a 0.18.0 12577 fedora 23 server (also from your repo).

Testing with that weak vm for a server did trigger a number of other issues though... which I wasn't able to repro with a vm server with a reasonable amount of resources.

#1198 - a 'tuple' object AttributeError (which might've been related to adjusting desktop-scaling to < 100%).

#1199 - h264 encoding errors both server and client side triggered when loading videos (or perhaps anything that will use considerable h264) while desktop-scaling < 100%.

#1200 - a gtk_common.error.XError that may've been triggered only by a combination of some variety of other issues with the poorly resourced server.

In any case, it doesn't look like the av-sync-delta adjustments were the cause of whatever I did... and I haven't seen it at any other time while tweaking with av-sync-delta (which I've done more than a little bit)... so I'm inclined to guess that it is related to the forced opengl (somehow).

I'll assign back to you to wade through those other issues as you get time, but if you have any other ideas for trying to repro, pass it back.

totaam · 2016-08-10T06:32:09Z

2016-08-10 06:32:09: antoine changed owner from antoine to afarr

totaam · 2016-08-10T06:32:09Z

2016-08-10 06:32:09: antoine changed title from Getting gl_paint_planar error and eventual client crash while adjusting av-sync-delta for testing to out of memory / gl_paint_planar error and eventual client crash

totaam · 2016-08-10T06:32:09Z

2016-08-10 06:32:09: antoine commented

r13291 fixes a bug which could have manifested itself as a memory leak of opengl resources on the client. But only when toggling opengl on and off... which should be a rare thing.
r13292 improves the error handling for cannot create GdkGLContext

I don't think there's anything else we should do with this ticket: enable opengl on intel at your own risk, run with at least 512MB of ram, etc..

totaam · 2016-08-16T23:46:38Z

2016-08-16 23:46:38: afarr changed status from new to closed

totaam · 2016-08-16T23:46:38Z

2016-08-16 23:46:38: afarr set resolution to fixed

totaam · 2016-08-16T23:46:38Z

2016-08-16 23:46:38: afarr commented

Started a 1.0 r13348 server and got a memory warning:
2016-08-16 15:43:51,761 Warning: only 489MB total system memory available
2016-08-16 15:43:51,761  this may not be enough to run a server
Between that and users forcing opengl on... I agree, not much more to do, let the users go crazy if they want.

Closing.

totaam · 2016-11-11T23:49:32Z

2016-11-11 23:49:32: afarr commented

Additional note:

Testing with 1.0 r14397 win client, with intel 4000 graphics and --opengl=on, against a 1.0 r14404 fedora 23 server, I'm again seeing similar opengl errors.

Server side (with -d stats,compress,x264, and also capturing client) errors:

2016-11-11 15:41:03,508 x264 close context 0x0
2016-11-11 15:41:03,528 compress:  18.5ms for 1319x912  pixels at    0,0    for wid=2     using  jpeg with ratio   5.2% ( 4698KB to   246KB), sequence   603, client_options={'quality': 94}
2016-11-11 15:41:03,534 compress:   2.9ms for 1319x147  pixels at    0,49   for wid=2     using  jpeg with ratio   5.7% (  757KB to    43KB), sequence   604, client_options={'quality': 94}
2016-11-11 15:41:03,630 compress:  20.2ms for 1319x912  pixels at    0,0    for wid=2     using  jpeg with ratio   5.2% ( 4698KB to   243KB), sequence   605, client_options={'quality': 94}
2016-11-11 15:41:03,670 compress:   7.5ms for  866x630  pixels at    0,0    for wid=2     using  jpeg with ratio   4.2% ( 2131KB to    89KB), sequence   606, client_options={'quality': 94}
2016-11-11 15:41:03,872 client 7: gtk2.GLWindowBacking(2, (1715, 910), GBRP).gl_paint_planar(..) error: GLError(
2016-11-11 15:41:03,874 client 7: 	err = 1286,
2016-11-11 15:41:03,875 client 7: 	baseOperation = glClear,
2016-11-11 15:41:03,875 client 7: 	cArguments = (GL_COLOR_BUFFER_BIT,)
2016-11-11 15:41:03,876 client 7: )
2016-11-11 15:41:03,876 client 7:   File "xpra\client\gl\gl_window_backing_base.pyc", line 853, in gl_paint_planar
2016-11-11 15:41:03,876 client 7:   File "xpra\client\gl\gl_window_backing_base.pyc", line 442, in gl_init
2016-11-11 15:41:03,876 client 7:   File "errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (c:\Users\mcfletch\OpenGL-dev\OpenGL-ctypes\OpenGL_accelerate\src\errorchecker.c:1218)
2016-11-11 15:41:03,877 Warning: client decoding error: gl_paint_planar error: GLError( err=1286, baseOperation = glClear )
2016-11-11 15:41:03,877 client 7: gtk2.GLWindowBacking(2, (1715, 910), GBRP).gl_paint_planar(..) error: GLError(
2016-11-11 15:41:03,877 client 7: 	err = 1286,
2016-11-11 15:41:03,877 client 7: 	baseOperation = glClear,
2016-11-11 15:41:03,877 client 7: 	cArguments = (GL_COLOR_BUFFER_BIT,)
2016-11-11 15:41:03,878 client 7: )
2016-11-11 15:41:03,878 client 7:   File "xpra\client\gl\gl_window_backing_base.pyc", line 853, in gl_paint_planar
2016-11-11 15:41:03,878 client 7:   File "xpra\client\gl\gl_window_backing_base.pyc", line 442, in gl_init
2016-11-11 15:41:03,878 client 7:   File "errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (c:\Users\mcfletch\OpenGL-dev\OpenGL-ctypes\OpenGL_accelerate\src\errorchecker.c:1218)
2016-11-11 15:41:03,878 Warning: client decoding error: gl_paint_planar error: GLError( err=1286, baseOperation = glClear )
2016-11-11 15:41:03,916 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:03,919 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:03,949 compress: 163.6ms for  854x480  pixels at   10,150  for wid=2     using   png with ratio  16.7% ( 1601KB to   266KB), sequence   607, client_options={'compress_level': 3}
2016-11-11 15:41:03,957 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:03,958 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:03,975 compress:  22.9ms for 1702x910  pixels at    0,0    for wid=2     using  jpeg with ratio   5.5% ( 6050KB to   332KB), sequence   608, client_options={'quality': 94}
2016-11-11 15:41:04,004 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:04,067 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:04,133 compress: 158.3ms for 1714x910  pixels at    0,0    for wid=2     using   png with ratio  13.4% ( 6092KB to   818KB), sequence   609, client_options={'compress_level': 3}
2016-11-11 15:41:04,158 compress:  23.4ms for 1714x910  pixels at    0,0    for wid=2     using  jpeg with ratio   5.4% ( 6092KB to   326KB), sequence   610, client_options={'quality': 94}
2016-11-11 15:41:04,178 compress:  19.9ms for 1715x910  pixels at    0,0    for wid=2     using  jpeg with ratio   5.4% ( 6096KB to   326KB), sequence   611, client_options={'quality': 94}
2016-11-11 15:41:04,210 compress:  26.2ms for 1714x910  pixels at    0,0    for wid=2     using  jpeg with ratio   4.9% ( 6092KB to   298KB), sequence   612, client_options={'quality': 94}

...

2016-11-11 15:41:06,960 compress: 148.9ms for 1713x910  pixels at    0,0    for wid=2     using   png with ratio  15.9% ( 6089KB to   970KB), sequence   634, client_options={'compress_level': 3}
2016-11-11 15:41:06,968 x264 context=0x7f9608581a80, YUV420P 1140x606  quality=37, speed=84, source=unknown
2016-11-11 15:41:06,968  preset=veryfast, profile=high10, tune=zerolatency
2016-11-11 15:41:06,969  me=DIA, me_range=16, mv_range=-1, b-frames=0, max delayed frames=0
2016-11-11 15:41:06,969  vfr-input=0, lookahead=-1, sync-lookahead=0, mb-tree=0, bframe-adaptive=FAST
2016-11-11 15:41:06,969  threads=auto, sliced-threads=True
2016-11-11 15:41:06,987 x264 encode YUV420P frame     0 as  IDR slice with 7 nals, tune=zerolatency, total   49454 bytes, keyframe=True , delayed=0
2016-11-11 15:41:06,987 compress:  25.7ms for 1712x910  pixels at    0,0    for wid=2     using  h264 with ratio   0.8% ( 6089KB to    48KB), sequence   635, client_options={'pts': 0, 'frame': 0, 'csc': 'YUV420P', 'scaled_size': (1140, 606), 'type': 'IDR', 'quality': 37, 'speed': 84}
2016-11-11 15:41:07,062 Warning: client decoding error: opengl RGB paint error: GLError(
	err = 1286,
	baseOperation = glClear,
	cArguments = (GL_COLOR_BUFFER_BIT,)
)
2016-11-11 15:41:07,067 client 7: gtk2.GLWindowBacking(2, (1713, 910), GBRP).gl_paint_planar(..) error: GLError(
2016-11-11 15:41:07,067 client 7: 	err = 1286,
2016-11-11 15:41:07,067 client 7: 	baseOperation = glClear,
2016-11-11 15:41:07,067 client 7: 	cArguments = (GL_COLOR_BUFFER_BIT,)
2016-11-11 15:41:07,068 client 7: )
2016-11-11 15:41:07,068 client 7:   File "xpra\client\gl\gl_window_backing_base.pyc", line 853, in gl_paint_planar
2016-11-11 15:41:07,068 client 7:   File "xpra\client\gl\gl_window_backing_base.pyc", line 442, in gl_init
2016-11-11 15:41:07,069 client 7:   File "errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (c:\Users\mcfletch\OpenGL-dev\OpenGL-ctypes\OpenGL_accelerate\src\errorchecker.c:1218)
2016-11-11 15:41:07,069 Warning: client decoding error: gl_paint_planar error: GLError( err=1286, baseOperation = glClear )
2016-11-11 15:41:07,147 compress: 139.8ms for 1713x910  pixels at    0,0    for wid=2     using   png with ratio  11.0% ( 6089KB to   671KB), sequence   636, client_options={'compress_level': 3}
2016-11-11 15:41:07,147 x264 close context 0x7f9608581a80

Not sure if these merit a re-opening of this ticket, but I'll log them here.

totaam · 2016-11-12T07:52:49Z

2016-11-12 07:52:49: antoine changed status from closed to reopened

totaam · 2016-11-12T07:52:49Z

2016-11-12 07:52:49: antoine removed resolution (was fixed)

totaam · 2016-11-12T08:08:24Z

2016-11-12 08:08:24: antoine commented

This has nothing to do with this ticket as there is no "out of memory" in these new error messages.

Please file a separate ticket for this and make sure to check to see if this is a regression by testing with older versions: we have always used glClear(GL_COLOR_BUFFER_BIT) but newer builds do this twice because we have 2 buffers in order to support scrolling (#1252). The error is on the first call, so this probably doesn't matter. Other changes are tracked in #679.
Error 1286 is INVALID_FRAMEBUFFER_OPERATION_EXT. Does r14405 help?
It looks like this bug: OpenGL invalid framebuffer operation after glClear(GL_COLOR_BUFFER_BIT), which looks like a driver problem... (we could also try to use glCheckFramebufferStatus with drivers that support it, maybe add an offscreen test to the gl_check code?)
Does it ever render anything with opengl enabled on this chipset / driver version?

The Intel drivers should still be greylisted so this should be less critical.
(please attach opengl debugging information to the new ticket)

totaam · 2016-11-18T00:38:41Z

2016-11-18 00:38:41: afarr changed status from reopened to closed

totaam · 2016-11-18T00:38:41Z

2016-11-18 00:38:41: afarr set resolution to fixed

totaam · 2016-11-18T00:38:41Z

2016-11-18 00:38:41: afarr commented

Info on new forced opengl error moved to #1358. I'll re-close this one.

totaam closed this as completed Nov 18, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

out of memory / gl_paint_planar error and eventual client crash #1111

out of memory / gl_paint_planar error and eventual client crash #1111

totaam commented Feb 4, 2016

totaam commented Feb 4, 2016

totaam commented Feb 4, 2016

totaam commented Feb 27, 2016

totaam commented Feb 27, 2016

totaam commented May 18, 2016

totaam commented May 18, 2016

totaam commented Aug 10, 2016

totaam commented Aug 10, 2016

totaam commented Aug 10, 2016

totaam commented Aug 16, 2016

totaam commented Aug 16, 2016

totaam commented Aug 16, 2016

totaam commented Nov 11, 2016

totaam commented Nov 12, 2016

totaam commented Nov 12, 2016

totaam commented Nov 12, 2016

totaam commented Nov 18, 2016

totaam commented Nov 18, 2016

totaam commented Nov 18, 2016

out of memory / gl_paint_planar error and eventual client crash #1111

out of memory / gl_paint_planar error and eventual client crash #1111

Comments

totaam commented Feb 4, 2016

2016-02-04 01:55:12: afarr created the issue

totaam commented Feb 4, 2016

2016-02-04 02:27:55: antoine changed owner from antoine to afarr

totaam commented Feb 4, 2016

2016-02-04 02:27:55: antoine commented

totaam commented Feb 27, 2016

2016-02-27 13:27:45: antoine changed priority from major to critical

totaam commented Feb 27, 2016

2016-02-27 13:27:45: antoine commented

totaam commented May 18, 2016

2016-05-18 01:34:11: afarr changed owner from afarr to antoine

totaam commented May 18, 2016

2016-05-18 01:34:11: afarr commented

totaam commented Aug 10, 2016

2016-08-10 06:32:09: antoine changed owner from antoine to afarr

totaam commented Aug 10, 2016

2016-08-10 06:32:09: antoine changed title from Getting gl_paint_planar error and eventual client crash while adjusting av-sync-delta for testing to out of memory / gl_paint_planar error and eventual client crash

totaam commented Aug 10, 2016

2016-08-10 06:32:09: antoine commented

totaam commented Aug 16, 2016

2016-08-16 23:46:38: afarr changed status from new to closed

totaam commented Aug 16, 2016

2016-08-16 23:46:38: afarr set resolution to fixed

totaam commented Aug 16, 2016

2016-08-16 23:46:38: afarr commented

totaam commented Nov 11, 2016

2016-11-11 23:49:32: afarr commented

totaam commented Nov 12, 2016

2016-11-12 07:52:49: antoine changed status from closed to reopened

totaam commented Nov 12, 2016

2016-11-12 07:52:49: antoine removed resolution (was fixed)

totaam commented Nov 12, 2016

2016-11-12 08:08:24: antoine commented

totaam commented Nov 18, 2016

2016-11-18 00:38:41: afarr changed status from reopened to closed

totaam commented Nov 18, 2016

2016-11-18 00:38:41: afarr set resolution to fixed

totaam commented Nov 18, 2016

2016-11-18 00:38:41: afarr commented