qrenderdoc crashes when attempting to view a frame on (theory) on a hiDPI display.

[bradc6]

Description

qrenderdoc crashes when using a when attempting to view a capture on Ubuntu.
Dump available @ https://www.dropbox.com/s/kp87hqjynms2rys/tutorial20_2018.06.21_04.57.34_frame225.rdc?dl=0

Repro steps

1)Open qrenderdoc
2)Launch a OpenGL application
3)Capture a frame
4)Attempt to inspect the captured frame by double clicking it
-->Crashed with a segmentation fault

Environment

*Renderdoc master from source @ 67037ed

• RenderDoc build: Built from source
• Operating System: Ubuntu 18.04 64bit
• API: OpenGL 4.2

Currently I have one hiDPI (4k) display and a 1080p display connected. I suspect that may have something to do with it as this was not a problem before installing the monitor. Below is the text from the console. The OpenGL app is just a sample so I can give that to you as well...

Thank you for the awesome software :)
-Brad

##Console output
user@comp:~/dev/renderdoc/build$ ./bin/qrenderdoc
Log - QRenderDoc initialising.
Gtk-Message: 21:57:30.614: GtkDialog mapped without a transient parent. This is discouraged.
Texture Array Sample
Log - Got remote handshake: tutorial20 [16274]
Vertex Shader Filepath: /home/brad/dev/OpenGLSuperBible6/Source/Tutorial20/Shaders/Main.glsl.vert
Fragment Shader Filepath: /home/brad/dev/OpenGLSuperBible6/Source/Tutorial20/Shaders/Main.glsl.frag
Log - Used API: OpenGL (Not presenting & supported)
Log - Used API: OpenGL (Presenting & supported)
Log - Got a new capture: 0 (time 1529557058) 163972 byte thumbnail
Log - Opening RDCFile /tmp/RenderDoc/tutorial20_2018.06.21_04.57.34_frame225.rdc
Log - Opening RDCFile /tmp/RenderDoc/tutorial20_2018.06.21_04.57.34_frame225.rdc
Log - Vendor checks for 46 (NVIDIA Corporation / GeForce GTX 980/PCIe/SSE2 / 4.6.0 NVIDIA 390.48)
Log - Running GL replay on: NVIDIA Corporation / GeForce GTX 980/PCIe/SSE2 / 4.6.0 NVIDIA 390.48
Log - [0]: GL_AMD_multi_draw_indirect, [1]: GL_AMD_seamless_cubemap_per_texture, [2]: GL_AMD_vertex_shader_viewport_index, [3]: GL_AMD_vertex_shader_layer, [4]: GL_ARB_arrays_of_arrays, [5]: GL_ARB_base_instance, [6]: GL_ARB_bindless_texture, [7]: GL_ARB_blend_func_extended, [8]: GL_ARB_buffer_storage, [9]: GL_ARB_clear_buffer_object, [10]: GL_ARB_clear_texture, [11]: GL_ARB_clip_control, [12]: GL_ARB_color_buffer_float, [13]: GL_ARB_compressed_texture_pixel_storage, [14]: GL_ARB_conservative_depth, [15]: GL_ARB_compute_shader, [16]: GL_ARB_compute_variable_group_size, [17]: GL_ARB_conditional_render_inverted, [18]: GL_ARB_copy_buffer, [19]: GL_ARB_copy_image, [20]: GL_ARB_cull_distance, [21]: GL_ARB_debug_output, [22]: GL_ARB_depth_buffer_float, [23]: GL_ARB_depth_clamp, [24]: GL_ARB_depth_texture, [25]: GL_ARB_derivative_control, [26]: GL_ARB_direct_state_access, [27]: GL_ARB_draw_buffers, [28]: GL_ARB_draw_buffers_blend, [29]: GL_ARB_draw_indirect, [30]: GL_ARB_draw_elements_base_vertex, [31]: GL_ARB_draw_instanced, [32]: GL_ARB_enhanced_layouts, [33]: GL_ARB_ES2_compatibility, [34]: GL_ARB_ES3_compatibility, [35]: GL_ARB_ES3_1_compatibility, [36]: GL_ARB_ES3_2_compatibility, [37]: GL_ARB_explicit_attrib_location, [38]: GL_ARB_explicit_uniform_location, [39]: GL_ARB_fragment_coord_conventions, [40]: GL_ARB_fragment_layer_viewport, [41]: GL_ARB_fragment_program, [42]: GL_ARB_fragment_program_shadow, [43]: GL_ARB_fragment_shader, [44]: GL_ARB_fragment_shader_interlock, [45]: GL_ARB_framebuffer_no_attachments, [46]: GL_ARB_framebuffer_object, [47]: GL_ARB_framebuffer_sRGB, [48]: GL_ARB_geometry_shader4, [49]: GL_ARB_get_program_binary, [50]: GL_ARB_get_texture_sub_image, [51]: GL_ARB_gl_spirv, [52]: GL_ARB_gpu_shader5, [53]: GL_ARB_gpu_shader_fp64, [54]: GL_ARB_gpu_shader_int64, [55]: GL_ARB_half_float_pixel, [56]: GL_ARB_half_float_vertex, [57]: GL_ARB_imaging, [58]: GL_ARB_indirect_parameters, [59]: GL_ARB_instanced_arrays, [60]: GL_ARB_internalformat_query, [61]: GL_ARB_internalformat_query2, [62]: GL_ARB_invalidate_subdata, [63]: GL_ARB_map_buffer_alignment, [64]: GL_ARB_map_buffer_range, [65]: GL_ARB_multi_bind, [66]: GL_ARB_multi_draw_indirect, [67]: GL_ARB_multisample, [68]: GL_ARB_multitexture, [69]: GL_ARB_occlusion_query, [70]: GL_ARB_occlusion_query2, [71]: GL_ARB_parallel_shader_compile, [72]: GL_ARB_pipeline_statistics_query, [73]: GL_ARB_pixel_buffer_object, [74]: GL_ARB_point_parameters, [75]: GL_ARB_point_sprite, [76]: GL_ARB_polygon_offset_clamp, [77]: GL_ARB_post_depth_coverage, [78]: GL_ARB_program_interface_query, [79]: GL_ARB_provoking_vertex, [80]: GL_ARB_query_buffer_object, [81]: GL_ARB_robust_buffer_access_behavior, [82]: GL_ARB_robustness, [83]: GL_ARB_sample_locations, [84]: GL_ARB_sample_shading, [85]: GL_ARB_sampler_objects, [86]: GL_ARB_seamless_cube_map, [87]: GL_ARB_seamless_cubemap_per_texture, [88]: GL_ARB_separate_shader_objects, [89]: GL_ARB_shader_atomic_counter_ops, [90]: GL_ARB_shader_atomic_counters, [91]: GL_ARB_shader_ballot, [92]: GL_ARB_shader_bit_encoding, [93]: GL_ARB_shader_clock, [94]: GL_ARB_shader_draw_parameters, [95]: GL_ARB_shader_group_vote, [96]: GL_ARB_shader_image_load_store, [97]: GL_ARB_shader_image_size, [98]: GL_ARB_shader_objects, [99]: GL_ARB_shader_precision, [100]: GL_ARB_shader_storage_buffer_object,
Log - [101]: GL_ARB_shader_subroutine, [102]: GL_ARB_shader_texture_image_samples, [103]: GL_ARB_shader_texture_lod, [104]: GL_ARB_shading_language_100, [105]: GL_ARB_shader_viewport_layer_array, [106]: GL_ARB_shading_language_420pack, [107]: GL_ARB_shading_language_include, [108]: GL_ARB_shading_language_packing, [109]: GL_ARB_shadow, [110]: GL_ARB_sparse_buffer, [111]: GL_ARB_sparse_texture, [112]: GL_ARB_sparse_texture2, [113]: GL_ARB_sparse_texture_clamp, [114]: GL_ARB_spirv_extensions, [115]: GL_ARB_stencil_texturing, [116]: GL_ARB_sync, [117]: GL_ARB_tessellation_shader, [118]: GL_ARB_texture_barrier, [119]: GL_ARB_texture_border_clamp, [120]: GL_ARB_texture_buffer_object, [121]: GL_ARB_texture_buffer_object_rgb32, [122]: GL_ARB_texture_buffer_range, [123]: GL_ARB_texture_compression, [124]: GL_ARB_texture_compression_bptc, [125]: GL_ARB_texture_compression_rgtc, [126]: GL_ARB_texture_cube_map, [127]: GL_ARB_texture_cube_map_array, [128]: GL_ARB_texture_env_add, [129]: GL_ARB_texture_env_combine, [130]: GL_ARB_texture_env_crossbar, [131]: GL_ARB_texture_env_dot3, [132]: GL_ARB_texture_filter_anisotropic, [133]: GL_ARB_texture_filter_minmax, [134]: GL_ARB_texture_float, [135]: GL_ARB_texture_gather, [136]: GL_ARB_texture_mirror_clamp_to_edge, [137]: GL_ARB_texture_mirrored_repeat, [138]: GL_ARB_texture_multisample, [139]: GL_ARB_texture_non_power_of_two, [140]: GL_ARB_texture_query_levels, [141]: GL_ARB_texture_query_lod, [142]: GL_ARB_texture_rectangle, [143]: GL_ARB_texture_rg, [144]: GL_ARB_texture_rgb10_a2ui, [145]: GL_ARB_texture_stencil8, [146]: GL_ARB_texture_storage, [147]: GL_ARB_texture_storage_multisample, [148]: GL_ARB_texture_swizzle, [149]: GL_ARB_texture_view, [150]: GL_ARB_timer_query, [151]: GL_ARB_transform_feedback2, [152]: GL_ARB_transform_feedback3, [153]: GL_ARB_transform_feedback_instanced, [154]: GL_ARB_transform_feedback_overflow_query, [155]: GL_ARB_transpose_matrix, [156]: GL_ARB_uniform_buffer_object, [157]: GL_ARB_vertex_array_bgra, [158]: GL_ARB_vertex_array_object, [159]: GL_ARB_vertex_attrib_64bit, [160]: GL_ARB_vertex_attrib_binding, [161]: GL_ARB_vertex_buffer_object, [162]: GL_ARB_vertex_program, [163]: GL_ARB_vertex_shader, [164]: GL_ARB_vertex_type_10f_11f_11f_rev, [165]: GL_ARB_vertex_type_2_10_10_10_rev, [166]: GL_ARB_viewport_array, [167]: GL_ARB_window_pos, [168]: GL_ATI_draw_buffers, [169]: GL_ATI_texture_float, [170]: GL_ATI_texture_mirror_once, [171]: GL_S3_s3tc, [172]: GL_EXT_texture_env_add, [173]: GL_EXT_abgr, [174]: GL_EXT_bgra, [175]: GL_EXT_bindable_uniform, [176]: GL_EXT_blend_color, [177]: GL_EXT_blend_equation_separate, [178]: GL_EXT_blend_func_separate, [179]: GL_EXT_blend_minmax, [180]: GL_EXT_blend_subtract, [181]: GL_EXT_compiled_vertex_array, [182]: GL_EXT_Cg_shader, [183]: GL_EXT_depth_bounds_test, [184]: GL_EXT_direct_state_access, [185]: GL_EXT_draw_buffers2, [186]: GL_EXT_draw_instanced, [187]: GL_EXT_draw_range_elements, [188]: GL_EXT_fog_coord, [189]: GL_EXT_framebuffer_blit, [190]: GL_EXT_framebuffer_multisample, [191]: GL_EXTX_framebuffer_mixed_formats, [192]: GL_EXT_framebuffer_multisample_blit_scaled, [193]: GL_EXT_framebuffer_object, [194]: GL_EXT_framebuffer_sRGB, [195]: GL_EXT_geometry_shader4, [196]: GL_EXT_gpu_program_parameters, [197]: GL_EXT_gpu_shader4, [198]: GL_EXT_multi_draw_arrays, [199]: GL_EXT_packed_depth_stencil, [200]: GL_EXT_packed_float,
Log - [201]: GL_EXT_packed_pixels, [202]: GL_EXT_pixel_buffer_object, [203]: GL_EXT_point_parameters, [204]: GL_EXT_polygon_offset_clamp, [205]: GL_EXT_post_depth_coverage, [206]: GL_EXT_provoking_vertex, [207]: GL_EXT_raster_multisample, [208]: GL_EXT_rescale_normal, [209]: GL_EXT_secondary_color, [210]: GL_EXT_separate_shader_objects, [211]: GL_EXT_separate_specular_color, [212]: GL_EXT_shader_image_load_formatted, [213]: GL_EXT_shader_image_load_store, [214]: GL_EXT_shader_integer_mix, [215]: GL_EXT_shadow_funcs, [216]: GL_EXT_sparse_texture2, [217]: GL_EXT_stencil_two_side, [218]: GL_EXT_stencil_wrap, [219]: GL_EXT_texture3D, [220]: GL_EXT_texture_array, [221]: GL_EXT_texture_buffer_object, [222]: GL_EXT_texture_compression_dxt1, [223]: GL_EXT_texture_compression_latc, [224]: GL_EXT_texture_compression_rgtc, [225]: GL_EXT_texture_compression_s3tc, [226]: GL_EXT_texture_cube_map, [227]: GL_EXT_texture_edge_clamp, [228]: GL_EXT_texture_env_combine, [229]: GL_EXT_texture_env_dot3, [230]: GL_EXT_texture_filter_anisotropic, [231]: GL_EXT_texture_filter_minmax, [232]: GL_EXT_texture_integer, [233]: GL_EXT_texture_lod, [234]: GL_EXT_texture_lod_bias, [235]: GL_EXT_texture_mirror_clamp, [236]: GL_EXT_texture_object, [237]: GL_EXT_texture_shared_exponent, [238]: GL_EXT_texture_sRGB, [239]: GL_EXT_texture_sRGB_decode, [240]: GL_EXT_texture_storage, [241]: GL_EXT_texture_swizzle, [242]: GL_EXT_timer_query, [243]: GL_EXT_transform_feedback2, [244]: GL_EXT_vertex_array, [245]: GL_EXT_vertex_array_bgra, [246]: GL_EXT_vertex_attrib_64bit, [247]: GL_EXT_window_rectangles, [248]: GL_EXT_x11_sync_object, [249]: GL_EXT_import_sync_object, [250]: GL_NV_robustness_video_memory_purge, [251]: GL_IBM_rasterpos_clip, [252]: GL_IBM_texture_mirrored_repeat, [253]: GL_KHR_context_flush_control, [254]: GL_KHR_debug, [255]: GL_EXT_memory_object, [256]: GL_EXT_memory_object_fd, [257]: GL_KHR_parallel_shader_compile, [258]: GL_KHR_no_error, [259]: GL_KHR_robust_buffer_access_behavior, [260]: GL_KHR_robustness, [261]: GL_EXT_semaphore, [262]: GL_EXT_semaphore_fd, [263]: GL_KTX_buffer_region, [264]: GL_NV_alpha_to_coverage_dither_control, [265]: GL_NV_bindless_multi_draw_indirect, [266]: GL_NV_bindless_multi_draw_indirect_count, [267]: GL_NV_bindless_texture, [268]: GL_NV_blend_equation_advanced, [269]: GL_NV_blend_equation_advanced_coherent, [270]: GL_NV_blend_minmax_factor, [271]: GL_NV_blend_square, [272]: GL_NV_command_list, [273]: GL_NV_compute_program5, [274]: GL_NV_conditional_render, [275]: GL_NV_conservative_raster, [276]: GL_NV_conservative_raster_dilate, [277]: GL_NV_copy_depth_to_color, [278]: GL_NV_copy_image, [279]: GL_NV_depth_buffer_float, [280]: GL_NV_depth_clamp, [281]: GL_NV_draw_texture, [282]: GL_NV_draw_vulkan_image, [283]: GL_NV_ES1_1_compatibility, [284]: GL_NV_ES3_1_compatibility, [285]: GL_NV_explicit_multisample, [286]: GL_NV_fence, [287]: GL_NV_fill_rectangle, [288]: GL_NV_float_buffer, [289]: GL_NV_fog_distance, [290]: GL_NV_fragment_coverage_to_color, [291]: GL_NV_fragment_program, [292]: GL_NV_fragment_program_option, [293]: GL_NV_fragment_program2, [294]: GL_NV_fragment_shader_interlock, [295]: GL_NV_framebuffer_mixed_samples, [296]: GL_NV_framebuffer_multisample_coverage, [297]: GL_NV_geometry_shader4, [298]: GL_NV_geometry_shader_passthrough, [299]: GL_NV_gpu_program4, [300]: GL_NV_internalformat_sample_query,
Log - [301]: GL_NV_gpu_program4_1, [302]: GL_NV_gpu_program5, [303]: GL_NV_gpu_program5_mem_extended, [304]: GL_NV_gpu_program_fp64, [305]: GL_NV_gpu_shader5, [306]: GL_NV_half_float, [307]: GL_NV_light_max_exponent, [308]: GL_NV_multisample_coverage, [309]: GL_NV_multisample_filter_hint, [310]: GL_NV_occlusion_query, [311]: GL_NV_packed_depth_stencil, [312]: GL_NV_parameter_buffer_object, [313]: GL_NV_parameter_buffer_object2, [314]: GL_NV_path_rendering, [315]: GL_NV_path_rendering_shared_edge, [316]: GL_NV_pixel_data_range, [317]: GL_NV_point_sprite, [318]: GL_NV_primitive_restart, [319]: GL_NV_query_resource, [320]: GL_NV_query_resource_tag, [321]: GL_NV_register_combiners, [322]: GL_NV_register_combiners2, [323]: GL_NV_sample_locations, [324]: GL_NV_sample_mask_override_coverage, [325]: GL_NV_shader_atomic_counters, [326]: GL_NV_shader_atomic_float, [327]: GL_NV_shader_atomic_fp16_vector, [328]: GL_NV_shader_atomic_int64, [329]: GL_NV_shader_buffer_load, [330]: GL_NV_shader_storage_buffer_object, [331]: GL_NV_texgen_reflection, [332]: GL_NV_texture_barrier, [333]: GL_NV_texture_compression_vtc, [334]: GL_NV_texture_env_combine4, [335]: GL_NV_texture_multisample, [336]: GL_NV_texture_rectangle, [337]: GL_NV_texture_rectangle_compressed, [338]: GL_NV_texture_shader, [339]: GL_NV_texture_shader2, [340]: GL_NV_texture_shader3, [341]: GL_NV_transform_feedback, [342]: GL_NV_transform_feedback2, [343]: GL_NV_uniform_buffer_unified_memory, [344]: GL_NV_vdpau_interop, [345]: GL_NV_vertex_array_range, [346]: GL_NV_vertex_array_range2, [347]: GL_NV_vertex_attrib_integer_64bit, [348]: GL_NV_vertex_buffer_unified_memory, [349]: GL_NV_vertex_program, [350]: GL_NV_vertex_program1_1, [351]: GL_NV_vertex_program2, [352]: GL_NV_vertex_program2_option, [353]: GL_NV_vertex_program3, [354]: GL_NV_viewport_array2, [355]: GL_NV_viewport_swizzle, [356]: GL_NVX_conditional_render, [357]: GL_NVX_gpu_memory_info, [358]: GL_NVX_nvenc_interop, [359]: GL_NV_shader_thread_group, [360]: GL_NV_shader_thread_shuffle, [361]: GL_KHR_blend_equation_advanced, [362]: GL_KHR_blend_equation_advanced_coherent, [363]: GL_SGIS_generate_mipmap, [364]: GL_SGIS_texture_lod, [365]: GL_SGIX_depth_texture, [366]: GL_SGIX_shadow, [367]: GL_SUN_slice_accum,
Log - Function pointers available:
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111111111111111111111111111111111111111111111
Log - 1111111111111111111111000000000000000000000000000000000000000000
Log - Created device.
Warning - FBOs are shared on this implementation
Log - GLSL version 460
Log - nVidia GPU detected - no counters available
Log - Created replay driver.
Log - Timer chunk initialisation - 191.786 ms
Log - QRenderDoc - renderer created for "/tmp/RenderDoc/tutorial20_2018.06.21_04.57.34_frame225.rdc"
Segmentation fault (core dumped)

[baldurk]

Hi, thanks for the report.

In this case the capture itself is fairly innocuous so I don't think it's the cause of the problem. Can you post the callstack where it crashes for you? That's a better place to start debugging.

[bradc6]

Is there any particular format/method that you prefer? I can give you a core dump if you want.

[Switching to Thread 0x7fffbe9f8700 (LWP 12503)]
0x00007ffff6157c5e in ResourceId::operator< (this=0x48b4864cc5589f0, u=...)
    at /home/brad/dev/renderdoc/renderdoc/api/replay/renderdoc_replay.h:289
289	  bool operator<(const ResourceId u) const { return id < u.id; }
(gdb) bt
#0  0x00007ffff6157c5e in ResourceId::operator<(ResourceId) const (this=0x48b4864cc5589f0, u=...)
    at /home/brad/dev/renderdoc/renderdoc/api/replay/renderdoc_replay.h:289
#1  0x00007ffff615a119 in std::less<ResourceId>::operator()(ResourceId const&, ResourceId const&) const (this=0x7fffbe9f7b60, __x=..., __y=...)
    at /usr/include/c++/7/bits/stl_function.h:386
#2  0x00007ffff6204a9e in std::_Rb_tree<ResourceId, ResourceId, std::_Identity<ResourceId>, std::less<ResourceId>, std::allocator<ResourceId> >::equal_range(ResourceId const&) (this=0x7fffbe9f7b60, __k=...)
    at /usr/include/c++/7/bits/stl_tree.h:1940
#3  0x00007ffff620476e in std::_Rb_tree<ResourceId, ResourceId, std::_Identity<ResourceId>, std::less<ResourceId>, std::allocator<ResourceId> >::erase(ResourceId const&) (this=0x7fffbe9f7b60, __x=...)
    at /usr/include/c++/7/bits/stl_tree.h:2500
#4  0x00007ffff62043a9 in std::set<ResourceId, std::less<ResourceId>, std::allocator<ResourceId> >::erase(ResourceId const&) (this=0x7fffbe9f7b60, __x=...)
    at /usr/include/c++/7/bits/stl_set.h:675
#5  0x00007ffff6c6142e in ReplayController::FreeTargetResource(ResourceId) (this=0x7fffbe9f75a0, id=...)
    at /home/brad/dev/renderdoc/renderdoc/replay/replay_controller.cpp:1830
#6  0x0000555555db1302 in CaptureContext::<lambda(IReplayController*)>::operator()(IReplayController *) const (__closure=0x7fffb8007858, r=0x7fffac0008d0)
    at ../../qrenderdoc/Code/CaptureContext.cpp:267```

[baldurk]

What commit sha is that from? I'm trying to tally up the line numbers - particularly that one on CaptureContext.cpp:267 but it doesn't seem to be 67037ed from the original post or any other recent commit. I don't remember there ever being a call to FreeTargetResource from CaptureContext.cpp.

Also FYI when posting code-like text it's a good idea to surround it with ``` so that it's formatted better and doesn't end up referencing very old github issues :).

[bradc6]

I updated to the latest 1.x branch and it's working correctly.
Thank you @baldurk

Closing this issue.