vp9 encoder crash

[totaam]

Issue migrated from trac ticket # 962

component: encodings | priority: critical | resolution: fixed | keywords: vp9 crash

2015-08-23 16:26:39: antoine created the issue

---

Spotted whilst testing for #937.
I was hoping this sort of thing would have been caught in #832.

0.15.5 server started with:

XPRA_FORCE_CSC_MODE=YUV420P \
    xpra start --csc-modules=swscale

Trunk client started with (see #937 for env vars):

XPRA_CSC_CYTHON_YUV420P_COLORSPACES="RGBX" XPRA_PIXMAP_INDIRECT_BGR=0 \
    xpra attach --no-mmap --encoding=vp9 --opengl=no --csc-module=cython

Hit segfault twice, got backtrace the second time:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f929a98e700 (LWP 26070)]
0x00007f92afff0c60 in write_modes_sb () from /usr/lib64/xpra/libvpx.so.2
(gdb) bt
#0  0x00007f92afff0c60 in write_modes_sb () at /usr/lib64/xpra/libvpx.so.2
#1  0x00007f92afff0d2f in write_modes_sb () at /usr/lib64/xpra/libvpx.so.2
#2  0x00007f92afff0cfe in write_modes_sb () at /usr/lib64/xpra/libvpx.so.2
#3  0x00007f92afff197c in vp9_pack_bitstream () at /usr/lib64/xpra/libvpx.so.2
#4  0x00007f92b0037c7f in encode_frame_to_data_rate () at /usr/lib64/xpra/libvpx.so.2
#5  0x00007f92b0039906 in vp9_get_compressed_data () at /usr/lib64/xpra/libvpx.so.2
#6  0x00007f92affd8d12 in encoder_encode () at /usr/lib64/xpra/libvpx.so.2
#7  0x00007f92aff2a1e4 in vpx_codec_encode () at /usr/lib64/xpra/libvpx.so.2
#8  0x00007f92b034e6ea in __pyx_f_4xpra_6codecs_3vpx_7encoder_7Encoder_do_compress_image (__pyx_v_self=0x7f929a125200, __pyx_v_pic_in=<optimized out>, __pyx_v_strides=<optimized out>) at xpra/codecs/vpx/encoder.c:7195
#9  0x00007f92b03496c0 in __pyx_pw_4xpra_6codecs_3vpx_7encoder_7Encoder_27compress_image (__pyx_v_options=<optimized out>, __pyx_v_speed=<optimized out>, __pyx_v_quality=<optimized out>, __pyx_v_image=<optimized out>, __pyx_v_self=<optimized out>) at xpra/codecs/vpx/encoder.c:6577
#10 0x00007f92b03496c0 in __pyx_pw_4xpra_6codecs_3vpx_7encoder_7Encoder_27compress_image (__pyx_v_self=<optimized out>, __pyx_args=<optimized out>, __pyx_kwds=<optimized out>) at xpra/codecs/vpx/encoder.c:6272
...

Both the client and servers have the latest libvpx-xpra installed:

window[2].encoder=vpx
window[2].encoder.abi_version=10
window[2].encoder.build_config=--prefix=/usr --libdir=/usr/lib64/xpra --enable-pic --disable-install-docs --disable-install-bins --enable-shared --enable-vp8 --enable-vp9 --enable-realtime-only --enable-runtime-cpu-detect
window[2].encoder.encoding=vp9
window[2].encoder.encodings=('vp8', 'vp9')
window[2].encoder.libvpx14=True
window[2].encoder.max_threads=3
window[2].encoder.src_format=YUV420P
window[2].encoder.version=v1.4.0
window[2].encoder.vp8.colorspaces=('YUV420P',)
window[2].encoder.vp9.colorspaces=('YUV420P', 'YUV444P')

[totaam]

2015-08-23 16:28:09: antoine changed status from new to assigned

[totaam]

2015-08-23 16:28:09: antoine edited the issue description

[totaam]

2015-08-30 11:04:02: antoine changed status from assigned to closed

[totaam]

2015-08-30 11:04:02: antoine set resolution to fixed

[totaam]

2015-08-30 11:04:02: antoine commented

---

Not seen since. Closing.