Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable sanitizer on macOS #5809

Merged
merged 8 commits into from
Mar 10, 2022
Merged

Enable sanitizer on macOS #5809

merged 8 commits into from
Mar 10, 2022

Conversation

philippemnoel
Copy link
Owner

@philippemnoel philippemnoel commented Mar 10, 2022

Ticket(s) Closed

This is a small improvement to our CI that @fhvwy did a while back, and we never merged/finished. The fix is described here:

google/sanitizers#1026

Need to explicitly specify detect_leaks on macOS, and use clang++ for it to work.

@github-actions github-actions bot added 📁 Repo: workflows This PR/Issue modifies /workflows code 📁 Repo: protocol This PR/Issue modifies /protocol code labels Mar 10, 2022
@philippemnoel philippemnoel marked this pull request as ready for review March 10, 2022 15:02
@fhvwy fhvwy force-pushed the mark/use-sanitisers-on-macos branch from 4f3e257 to ba4eea6 Compare March 10, 2022 15:07
@codecov
Copy link

codecov bot commented Mar 10, 2022

Codecov Report

Merging #5809 (cd76d7f) into dev (efa3b62) will decrease coverage by 5.68%.
The diff coverage is 100.00%.

❗ Current head cd76d7f differs from pull request most recent head c5fbc0f. Consider uploading reports for the commit c5fbc0f to get more accurate results

@@            Coverage Diff             @@
##              dev    #5809      +/-   ##
==========================================
- Coverage   61.59%   55.91%   -5.69%     
==========================================
  Files         168      194      +26     
  Lines       21457    25893    +4436     
  Branches       31       31              
==========================================
+ Hits        13217    14477    +1260     
- Misses       8028    11204    +3176     
  Partials      212      212              
Flag Coverage Δ *Carryforward flag
backend-services 51.10% <ø> (ø) Carriedforward from b233ff3
backend-webserver 88.15% <ø> (ø) Carriedforward from b233ff3
frontend-core-ts 69.13% <ø> (ø) Carriedforward from b233ff3
protocol 54.17% <100.00%> (-7.02%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files Coverage Δ
protocol/whist/network/ringbuffer.c 41.94% <100.00%> (-0.21%) ⬇️
protocol/whist/utils/lodepng.c 87.50% <0.00%> (-1.93%) ⬇️
protocol/whist/file/file_synchronizer.c 68.51% <0.00%> (-1.81%) ⬇️
protocol/whist/video/codec/decode.c 34.60% <0.00%> (-1.46%) ⬇️
protocol/whist/utils/os_utils.c 8.13% <0.00%> (-1.39%) ⬇️
protocol/whist/video/ltr.c 86.02% <0.00%> (-1.13%) ⬇️
protocol/client/whist_client.c 11.23% <0.00%> (-0.93%) ⬇️
protocol/client/network.c 7.86% <0.00%> (-0.28%) ⬇️
protocol/whist/logging/error_monitor.c 32.50% <0.00%> (-0.26%) ⬇️
... and 73 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update efa3b62...c5fbc0f. Read the comment docs.

@fhvwy fhvwy force-pushed the mark/use-sanitisers-on-macos branch from ba4eea6 to e60d9b8 Compare March 10, 2022 15:18
@philippemnoel philippemnoel self-assigned this Mar 10, 2022
@github-actions
Copy link

Protocol End-to-End Streaming Test Results - 2022_03_10@17-31-41 UTC

Results compared to branch dev

Experiment 0 - Network conditions: normal

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-31-41 UTC 2022_03_10@00-24-27 UTC
network_conditions normal normal
using_two_instances True True
branch_name mark/use-sanitisers-on-macos dev
github_sha cd76d7f efd1f06
server_hang_detected False False
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_FPS_SKIPPED 120 0.183 0.658 -0.475 (-72.200%) ⬇️
VIDEO_FPS_RENDERED 120 57.392 58.150 -0.758 (-1.300%) ⬇️
VIDEO_END_TO_END_LATENCY 6038 53.774 54.460 -0.686 (-1.300%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 12 1087.000 470.000 617.000 (131.300%) ⬆️
VIDEO_INTER_FRAME_QP 6710 16.206 16.365 -0.159 (-1.000%) ⬇️
MAX_VIDEO_INTER_FRAME_QP 12 32.000 33.000 -1.000 (-3.000%) ⬇️
VIDEO_INTRA_FRAME_QP 2 29.500 23.000 6.500 (28.300%) ⬆️
MAX_VIDEO_INTRA_FRAME_QP 1 33.000 26.000 7.000 (26.900%) ⬆️
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 1 0.042 0.042 -
AUDIO_RECEIVE_TIME 11710 0.000 0.000 -
MAX_AUDIO_RECEIVE_TIME 12 0.029 0.029 -
AUDIO_FPS_SKIPPED 120 0.183 0.658 -0.475 (-72.200%) ⬇️
READ_PACKET_TIME_TCP 4886 3.682 3.749 -0.067 (-1.800%) ⬇️
MAX_READ_PACKET_TIME_TCP 12 11.968 12.167 -0.199 (-1.600%) ⬇️
READ_PACKET_TIME_UDP 208948 0.571 0.565 0.006 (1.100%) ⬆️
MAX_READ_PACKET_TIME_UDP 12 11.706 11.521 0.185 (1.600%) ⬆️
HANDLE_SERVER_MESSAGE_TIME_TCP 3 0.018 0.017 0.001 (5.900%) ⬆️
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 1 0.023 0.017 0.006 (35.300%) ⬆️
CURSOR_UPDATE_TIME 6 0.164 0.430 -0.266 (-61.900%) ⬇️
MAX_CURSOR_UPDATE_TIME 1 0.273 2.220 -1.947 (-87.700%) ⬇️
VIDEO_DECODE_SEND_PACKET_TIME 6886 8.760 9.189 -0.429 (-4.700%) ⬇️
MAX_VIDEO_DECODE_SEND_PACKET_TIME 12 42.663 52.256 -9.593 (-18.400%) ⬇️
VIDEO_DECODE_GET_FRAME_TIME 108936 0.001 0.001 -
MAX_VIDEO_DECODE_GET_FRAME_TIME 12 0.196 0.251 -0.055 (-21.900%) ⬇️
VIDEO_FPS_RENDERED 120 57.392 58.150 -0.758 (-1.300%) ⬇️
VIDEO_END_TO_END_LATENCY 6038 53.774 54.460 -0.686 (-1.300%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 12 1087.000 470.000 617.000 (131.300%) ⬆️
MIN_VIDEO_END_TO_END_LATENCY 12 12.000 8.000 4.000 (50.000%) ⬆️
VIDEO_RECEIVE_TIME 6887 0.000 0.000 -
MAX_VIDEO_RECEIVE_TIME 12 0.014 0.015 -0.001 (-6.700%) ⬇️
VIDEO_RENDER_TIME 12170 0.964 0.994 -0.030 (-3.000%) ⬇️
MAX_VIDEO_RENDER_TIME 12 30.501 30.979 -0.478 (-1.500%) ⬇️
VIDEO_TIME_BETWEEN_FRAMES 6037 19.391 19.475 -0.084 (-0.400%) ⬇️
MAX_VIDEO_TIME_BETWEEN_FRAMES 12 207.420 203.674 3.746 (1.800%) ⬆️
NOTIFICATIONS_RECEIVED 120 0.000 0.000 -
CLIENT_CPU_USAGE 102 14.558 14.310 0.248 (1.700%) ⬆️
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_ENCODE_TIME 11931 0.196 0.209 -0.013 (-6.200%) ⬇️
MAX_AUDIO_ENCODE_TIME 12 2.587 1.722 0.865 (50.200%) ⬆️
HANDLE_USERINPUT_TIME 1 0.018 0.016 0.002 (12.500%) ⬆️
MAX_HANDLE_USERINPUT_TIME 1 0.018 0.016 0.002 (12.500%) ⬆️
THROTTLED_PACKET_DELAY 346118 0.057 0.059 -0.002 (-3.400%) ⬇️
MAX_THROTTLED_PACKET_DELAY 12 6.139 5.629 0.510 (9.100%) ⬆️
THROTTLED_PACKET_DELAY_RATE 346118 0.000 0.000 -
MAX_THROTTLED_PACKET_DELAY_RATE 12 1.192 1.197 -0.005 (-0.400%) ⬇️
VIDEO_CAPTURE_CREATE_TIME 1 2341.988 276.956 2065.032 (745.600%) ⬆️
MAX_VIDEO_CAPTURE_CREATE_TIME 1 2341.988 276.956 2065.032 (745.600%) ⬆️
VIDEO_CAPTURE_UPDATE_TIME 2 30.033 27.794 2.239 (8.100%) ⬆️
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 60.053 55.564 4.489 (8.100%) ⬆️
VIDEO_CAPTURE_SCREEN_TIME 3309 3.711 3.708 0.003 (0.100%) ⬆️
MAX_VIDEO_CAPTURE_SCREEN_TIME 12 76.569 72.585 3.984 (5.500%) ⬆️
VIDEO_CAPTURE_TRANSFER_TIME 6712 3.069 3.473 -0.404 (-11.600%) ⬇️
MAX_VIDEO_CAPTURE_TRANSFER_TIME 12 23.000 25.766 -2.766 (-10.700%) ⬇️
VIDEO_ENCODER_UPDATE_TIME 11 8.122 9.971 -1.849 (-18.500%) ⬇️
MAX_VIDEO_ENCODER_UPDATE_TIME 2 87.824 83.010 4.814 (5.800%) ⬆️
VIDEO_ENCODE_TIME 6711 9.868 9.897 -0.029 (-0.300%) ⬇️
MAX_VIDEO_ENCODE_TIME 12 14.947 15.226 -0.279 (-1.800%) ⬇️
VIDEO_FPS_SENT 120 55.925 56.858 -0.933 (-1.600%) ⬇️
VIDEO_FPS_SKIPPED_IN_CAPTURE 120 0.358 0.400 -0.042 (-10.500%) ⬇️
VIDEO_FRAME_SIZE 6711 30889.349 30823.391 65.958 (0.200%) ⬆️
MAX_VIDEO_FRAME_SIZE 12 345575.000 436342.000 -90767.000 (-20.800%) ⬇️
VIDEO_FRAME_PROCESSING_TIME 6711 13.819 14.227 -0.408 (-2.900%) ⬇️
MAX_VIDEO_FRAME_PROCESSING_TIME 12 82.552 59.255 23.297 (39.300%) ⬆️
GET_CURSOR_TIME 6711 0.531 0.524 0.007 (1.300%) ⬆️
MAX_GET_CURSOR_TIME 12 29.184 17.326 11.858 (68.400%) ⬆️
VIDEO_INTER_FRAME_QP 6710 16.206 16.365 -0.159 (-1.000%) ⬇️
MAX_VIDEO_INTER_FRAME_QP 12 32.000 33.000 -1.000 (-3.000%) ⬇️
VIDEO_INTRA_FRAME_QP 2 29.500 23.000 6.500 (28.300%) ⬆️
MAX_VIDEO_INTRA_FRAME_QP 1 33.000 26.000 7.000 (26.900%) ⬆️
VIDEO_SEND_TIME 6711 3.161 3.150 0.011 (0.300%) ⬆️
MAX_VIDEO_SEND_TIME 12 85.396 65.230 20.166 (30.900%) ⬆️
DBUS_MSGS_RECEIVED 120 0.758 1.125 -0.367 (-32.600%) ⬇️
SERVER_CPU_USAGE 99 57.324 56.653 0.671 (1.200%) ⬆️

Experiment 1 - Network conditions: 1Mbit,none,none

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-41-46 UTC 2022_03_10@00-32-11 UTC
network_conditions 1Mbit,none,none 1Mbit,none,none
using_two_instances True True
branch_name mark/use-sanitisers-on-macos dev
github_sha cd76d7f efd1f06
server_hang_detected False False
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_FPS_SKIPPED 80 0.438 0.487 -0.049 (-10.100%) ⬇️
VIDEO_FPS_RENDERED 80 2.600 2.150 0.450 (20.900%) ⬆️
VIDEO_END_TO_END_LATENCY 207 3384.957 3780.906 -395.949 (-10.500%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 8 8428.000 8056.000 372.000 (4.600%) ⬆️
VIDEO_INTER_FRAME_QP 2132 28.617 27.840 0.777 (2.800%) ⬆️
MAX_VIDEO_INTER_FRAME_QP 13 40.000 40.000 -
VIDEO_INTRA_FRAME_QP 13 31.923 32.000 -0.077 (-0.200%) ⬇️
MAX_VIDEO_INTRA_FRAME_QP 9 46.000 43.000 3.000 (7.000%) ⬆️
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 9 0.201 0.208 -0.007 (-3.400%) ⬇️
AUDIO_RECEIVE_TIME 415 0.001 0.002 -0.001 (-50.000%) ⬇️
MAX_AUDIO_RECEIVE_TIME 8 0.032 0.040 -0.008 (-20.000%) ⬇️
AUDIO_FPS_SKIPPED 80 0.438 0.487 -0.049 (-10.100%) ⬇️
READ_PACKET_TIME_TCP 2122 3.415 3.436 -0.021 (-0.600%) ⬇️
MAX_READ_PACKET_TIME_TCP 8 11.889 11.902 -0.013 (-0.100%) ⬇️
READ_PACKET_TIME_UDP 9922 5.232 5.114 0.118 (2.300%) ⬆️
MAX_READ_PACKET_TIME_UDP 8 11.494 11.998 -0.504 (-4.200%) ⬇️
HANDLE_SERVER_MESSAGE_TIME_TCP 10 0.015 0.021 -0.006 (-28.600%) ⬇️
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 8 0.023 0.038 -0.015 (-39.500%) ⬇️
CURSOR_UPDATE_TIME 13 0.375 0.292 0.083 (28.400%) ⬆️
MAX_CURSOR_UPDATE_TIME 8 4.394 2.817 1.577 (56.000%) ⬆️
VIDEO_DECODE_SEND_PACKET_TIME 208 7.603 10.158 -2.555 (-25.200%) ⬇️
MAX_VIDEO_DECODE_SEND_PACKET_TIME 8 22.456 19.446 3.010 (15.500%) ⬆️
VIDEO_DECODE_GET_FRAME_TIME 137510 0.001 0.001 -
MAX_VIDEO_DECODE_GET_FRAME_TIME 8 0.054 0.054 -
VIDEO_FPS_RENDERED 80 2.600 2.150 0.450 (20.900%) ⬆️
VIDEO_END_TO_END_LATENCY 207 3384.957 3780.906 -395.949 (-10.500%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 8 8428.000 8056.000 372.000 (4.600%) ⬆️
MIN_VIDEO_END_TO_END_LATENCY 8 232.000 509.000 -277.000 (-54.400%) ⬇️
VIDEO_RECEIVE_TIME 208 0.000 0.001 -0.001 (-100.000%) ⬇️
MAX_VIDEO_RECEIVE_TIME 8 0.001 0.012 -0.011 (-91.700%) ⬇️
VIDEO_RENDER_TIME 700 1.082 0.979 0.103 (10.500%) ⬆️
MAX_VIDEO_RENDER_TIME 8 31.350 36.171 -4.821 (-13.300%) ⬇️
VIDEO_TIME_BETWEEN_FRAMES 206 517.678 609.450 -91.772 (-15.100%) ⬇️
MAX_VIDEO_TIME_BETWEEN_FRAMES 8 11054.678 9113.582 1941.096 (21.300%) ⬆️
NOTIFICATIONS_RECEIVED 80 0.000 0.000 -
CLIENT_CPU_USAGE 43 39.284 23.579 15.705 (66.600%) ⬆️
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_ENCODE_TIME 12972 0.182 0.190 -0.008 (-4.200%) ⬇️
MAX_AUDIO_ENCODE_TIME 13 1.858 1.209 0.649 (53.700%) ⬆️
HANDLE_USERINPUT_TIME 1 0.016 0.020 -0.004 (-20.000%) ⬇️
MAX_HANDLE_USERINPUT_TIME 1 0.016 0.020 -0.004 (-20.000%) ⬇️
THROTTLED_PACKET_DELAY 46541 1.004 0.959 0.045 (4.700%) ⬆️
MAX_THROTTLED_PACKET_DELAY 13 7.273 5.969 1.304 (21.800%) ⬆️
THROTTLED_PACKET_DELAY_RATE 46541 0.002 0.002 -
MAX_THROTTLED_PACKET_DELAY_RATE 13 1.241 1.238 0.003 (0.200%) ⬆️
VIDEO_CAPTURE_CREATE_TIME 1 333.885 4368.136 -4034.251 (-92.400%) ⬇️
MAX_VIDEO_CAPTURE_CREATE_TIME 1 333.885 4368.136 -4034.251 (-92.400%) ⬇️
VIDEO_CAPTURE_UPDATE_TIME 2 43.775 41.996 1.779 (4.200%) ⬆️
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 87.537 83.983 3.554 (4.200%) ⬆️
VIDEO_CAPTURE_SCREEN_TIME 1356 3.602 3.610 -0.008 (-0.200%) ⬇️
MAX_VIDEO_CAPTURE_SCREEN_TIME 13 50.013 54.745 -4.732 (-8.600%) ⬇️
VIDEO_CAPTURE_TRANSFER_TIME 2145 3.145 3.502 -0.357 (-10.200%) ⬇️
MAX_VIDEO_CAPTURE_TRANSFER_TIME 13 23.159 24.272 -1.113 (-4.600%) ⬇️
VIDEO_ENCODER_UPDATE_TIME 20 4.667 4.009 0.658 (16.400%) ⬆️
MAX_VIDEO_ENCODER_UPDATE_TIME 10 90.523 77.237 13.286 (17.200%) ⬆️
VIDEO_ENCODE_TIME 2145 10.311 10.045 0.266 (2.600%) ⬆️
MAX_VIDEO_ENCODE_TIME 13 15.165 15.061 0.104 (0.700%) ⬆️
VIDEO_FPS_SENT 130 16.500 15.215 1.285 (8.400%) ⬆️
VIDEO_FPS_SKIPPED_IN_CAPTURE 130 0.638 0.331 0.307 (92.700%) ⬆️
VIDEO_FRAME_SIZE 2145 8547.924 9066.709 -518.785 (-5.700%) ⬇️
MAX_VIDEO_FRAME_SIZE 13 224960.000 133595.000 91365.000 (68.400%) ⬆️
VIDEO_FRAME_PROCESSING_TIME 2145 19.888 19.629 0.259 (1.300%) ⬆️
MAX_VIDEO_FRAME_PROCESSING_TIME 13 146.941 113.251 33.690 (29.700%) ⬆️
GET_CURSOR_TIME 2145 0.652 0.659 -0.007 (-1.100%) ⬇️
MAX_GET_CURSOR_TIME 13 17.775 13.207 4.568 (34.600%) ⬆️
VIDEO_INTER_FRAME_QP 2132 28.617 27.840 0.777 (2.800%) ⬆️
MAX_VIDEO_INTER_FRAME_QP 13 40.000 40.000 -
VIDEO_INTRA_FRAME_QP 13 31.923 32.000 -0.077 (-0.200%) ⬇️
MAX_VIDEO_INTRA_FRAME_QP 9 46.000 43.000 3.000 (7.000%) ⬆️
VIDEO_SEND_TIME 2144 16.701 16.812 -0.111 (-0.700%) ⬇️
MAX_VIDEO_SEND_TIME 13 155.390 130.381 25.009 (19.200%) ⬆️
DBUS_MSGS_RECEIVED 130 1.085 1.215 -0.130 (-10.700%) ⬇️
SERVER_CPU_USAGE 46 51.085 52.969 -1.884 (-3.600%) ⬇️

Experiment 2 - Network conditions: 1Mbit,none,0.1

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-45-37 UTC 2022_03_10@00-36-06 UTC
network_conditions 1Mbit,none,0.1 1Mbit,none,0.1
using_two_instances True True
branch_name mark/use-sanitisers-on-macos dev
github_sha cd76d7f efd1f06
server_hang_detected False False
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_FPS_SKIPPED 30 0.467 0.487 -0.020 (-4.100%) ⬇️
VIDEO_FPS_RENDERED 30 3.067 2.075 0.992 (47.800%) ⬆️
VIDEO_END_TO_END_LATENCY 90 3020.911 3571.560 -550.649 (-15.400%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 3 7769.000 8416.000 -647.000 (-7.700%) ⬇️
VIDEO_INTER_FRAME_QP 959 28.619 28.512 0.107 (0.400%) ⬆️
MAX_VIDEO_INTER_FRAME_QP 6 40.000 40.000 -
VIDEO_INTRA_FRAME_QP 7 35.286 34.000 1.286 (3.800%) ⬆️
MAX_VIDEO_INTRA_FRAME_QP 4 46.000 46.000 -
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 4 0.171 0.142 0.029 (20.400%) ⬆️
AUDIO_RECEIVE_TIME 187 0.001 0.002 -0.001 (-50.000%) ⬇️
MAX_AUDIO_RECEIVE_TIME 3 0.027 0.036 -0.009 (-25.000%) ⬇️
AUDIO_FPS_SKIPPED 30 0.467 0.487 -0.020 (-4.100%) ⬇️
READ_PACKET_TIME_TCP 845 3.334 3.401 -0.067 (-2.000%) ⬇️
MAX_READ_PACKET_TIME_TCP 3 11.881 11.999 -0.118 (-1.000%) ⬇️
READ_PACKET_TIME_UDP 4016 5.142 5.256 -0.114 (-2.200%) ⬇️
MAX_READ_PACKET_TIME_UDP 3 11.470 11.574 -0.104 (-0.900%) ⬇️
HANDLE_SERVER_MESSAGE_TIME_TCP 4 0.021 0.015 0.006 (40.000%) ⬆️
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 3 0.027 0.025 0.002 (8.000%) ⬆️
CURSOR_UPDATE_TIME 5 0.622 0.236 0.386 (163.600%) ⬆️
MAX_CURSOR_UPDATE_TIME 3 2.747 2.358 0.389 (16.500%) ⬆️
VIDEO_DECODE_SEND_PACKET_TIME 92 7.019 9.173 -2.154 (-23.500%) ⬇️
MAX_VIDEO_DECODE_SEND_PACKET_TIME 3 17.807 19.173 -1.366 (-7.100%) ⬇️
VIDEO_DECODE_GET_FRAME_TIME 54873 0.001 0.001 -
MAX_VIDEO_DECODE_GET_FRAME_TIME 3 0.021 0.168 -0.147 (-87.500%) ⬇️
VIDEO_FPS_RENDERED 30 3.067 2.075 0.992 (47.800%) ⬆️
VIDEO_END_TO_END_LATENCY 90 3020.911 3571.560 -550.649 (-15.400%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 3 7769.000 8416.000 -647.000 (-7.700%) ⬇️
MIN_VIDEO_END_TO_END_LATENCY 3 222.000 205.000 17.000 (8.300%) ⬆️
VIDEO_RECEIVE_TIME 92 0.000 0.000 -
MAX_VIDEO_RECEIVE_TIME 3 0.001 0.003 -0.002 (-66.700%) ⬇️
VIDEO_RENDER_TIME 265 0.970 1.146 -0.176 (-15.400%) ⬇️
MAX_VIDEO_RENDER_TIME 3 28.719 28.611 0.108 (0.400%) ⬆️
VIDEO_TIME_BETWEEN_FRAMES 89 385.917 644.325 -258.408 (-40.100%) ⬇️
MAX_VIDEO_TIME_BETWEEN_FRAMES 3 8182.096 13914.926 -5732.830 (-41.200%) ⬇️
NOTIFICATIONS_RECEIVED 30 0.000 0.000 -
CLIENT_CPU_USAGE 17 25.629 36.572 -10.943 (-29.900%) ⬇️
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_ENCODE_TIME 11946 0.180 0.193 -0.013 (-6.700%) ⬇️
MAX_AUDIO_ENCODE_TIME 12 2.631 2.419 0.212 (8.800%) ⬆️
HANDLE_USERINPUT_TIME 1 0.014 0.020 -0.006 (-30.000%) ⬇️
MAX_HANDLE_USERINPUT_TIME 1 0.014 0.020 -0.006 (-30.000%) ⬇️
THROTTLED_PACKET_DELAY 22595 0.943 1.041 -0.098 (-9.400%) ⬇️
MAX_THROTTLED_PACKET_DELAY 6 8.555 6.322 2.233 (35.300%) ⬆️
THROTTLED_PACKET_DELAY_RATE 22595 0.002 0.002 -
MAX_THROTTLED_PACKET_DELAY_RATE 6 1.422 1.240 0.182 (14.700%) ⬆️
VIDEO_CAPTURE_CREATE_TIME 1 355.062 345.229 9.833 (2.800%) ⬆️
MAX_VIDEO_CAPTURE_CREATE_TIME 1 355.062 345.229 9.833 (2.800%) ⬆️
VIDEO_CAPTURE_UPDATE_TIME 2 41.784 36.278 5.506 (15.200%) ⬆️
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 83.553 72.528 11.025 (15.200%) ⬆️
VIDEO_CAPTURE_SCREEN_TIME 602 3.717 3.690 0.027 (0.700%) ⬆️
MAX_VIDEO_CAPTURE_SCREEN_TIME 6 100.303 34.103 66.200 (194.100%) ⬆️
VIDEO_CAPTURE_TRANSFER_TIME 966 3.106 3.509 -0.403 (-11.500%) ⬇️
MAX_VIDEO_CAPTURE_TRANSFER_TIME 6 25.138 23.951 1.187 (5.000%) ⬆️
VIDEO_ENCODER_UPDATE_TIME 10 9.635 4.232 5.403 (127.700%) ⬆️
MAX_VIDEO_ENCODER_UPDATE_TIME 5 95.025 81.472 13.553 (16.600%) ⬆️
VIDEO_ENCODE_TIME 966 10.492 10.256 0.236 (2.300%) ⬆️
MAX_VIDEO_ENCODE_TIME 6 15.943 15.801 0.142 (0.900%) ⬆️
VIDEO_FPS_SENT 120 8.042 16.469 -8.427 (-51.200%) ⬇️
VIDEO_FPS_SKIPPED_IN_CAPTURE 120 0.483 0.600 -0.117 (-19.500%) ⬇️
VIDEO_FRAME_SIZE 965 8336.255 8546.100 -209.845 (-2.500%) ⬇️
MAX_VIDEO_FRAME_SIZE 6 69582.000 65348.000 4234.000 (6.500%) ⬆️
VIDEO_FRAME_PROCESSING_TIME 965 20.049 19.857 0.192 (1.000%) ⬆️
MAX_VIDEO_FRAME_PROCESSING_TIME 6 74.343 87.921 -13.578 (-15.400%) ⬇️
GET_CURSOR_TIME 966 0.789 0.675 0.114 (16.900%) ⬆️
MAX_GET_CURSOR_TIME 6 13.950 20.010 -6.060 (-30.300%) ⬇️
VIDEO_INTER_FRAME_QP 959 28.619 28.512 0.107 (0.400%) ⬆️
MAX_VIDEO_INTER_FRAME_QP 6 40.000 40.000 -
VIDEO_INTRA_FRAME_QP 7 35.286 34.000 1.286 (3.800%) ⬆️
MAX_VIDEO_INTRA_FRAME_QP 4 46.000 46.000 -
VIDEO_SEND_TIME 964 16.233 16.735 -0.502 (-3.000%) ⬇️
MAX_VIDEO_SEND_TIME 6 80.200 95.084 -14.884 (-15.700%) ⬇️
DBUS_MSGS_RECEIVED 120 0.975 0.862 0.113 (13.100%) ⬆️
SERVER_CPU_USAGE 21 58.338 52.830 5.508 (10.400%) ⬆️

Experiment 3 - Network conditions: 1Mbit,100,0.1

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-49-14 UTC 2022_03_09@22-56-24 UTC
network_conditions 1Mbit,100,0.1 1Mbit,100,0.1
using_two_instances True True
branch_name mark/use-sanitisers-on-macos dev
github_sha cd76d7f efd1f06
server_hang_detected False False
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_FPS_SKIPPED 50 0.38 0.438 -0.058 (-13.200%) ⬇️
VIDEO_FPS_RENDERED 50 2.88 1.800 1.080 (60.000%) ⬆️
VIDEO_END_TO_END_LATENCY 142 2078506374505629.75 2049638230414915.500 28868144090714.250 (1.400%) ⬆️
MAX_VIDEO_END_TO_END_LATENCY 5 18446744073708232.00 18446744073707840.000 392.000 (0.000%) ⬆️
VIDEO_INTER_FRAME_QP 1436 28.18 28.775 -0.595 (-2.100%) ⬇️
MAX_VIDEO_INTER_FRAME_QP 9 40.00 40.000 -
VIDEO_INTRA_FRAME_QP 10 31.80 32.895 -1.095 (-3.300%) ⬇️
MAX_VIDEO_INTRA_FRAME_QP 7 46.00 43.000 3.000 (7.000%) ⬆️
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 6 1.043 0.617 0.426 (69.000%) ⬆️
AUDIO_RECEIVE_TIME 315 0.001 0.002 -0.001 (-50.000%) ⬇️
MAX_AUDIO_RECEIVE_TIME 5 0.026 0.054 -0.028 (-51.900%) ⬇️
AUDIO_FPS_SKIPPED 50 0.380 0.438 -0.058 (-13.200%) ⬇️
READ_PACKET_TIME_TCP 1534 3.435 3.463 -0.028 (-0.800%) ⬇️
MAX_READ_PACKET_TIME_TCP 5 11.901 11.888 0.013 (0.100%) ⬆️
READ_PACKET_TIME_UDP 7187 5.218 5.267 -0.049 (-0.900%) ⬇️
MAX_READ_PACKET_TIME_UDP 5 11.507 11.706 -0.199 (-1.700%) ⬇️
HANDLE_SERVER_MESSAGE_TIME_TCP 5 0.018 0.016 0.002 (12.500%) ⬆️
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 5 0.027 0.019 0.008 (42.100%) ⬆️
CURSOR_UPDATE_TIME 8 0.539 0.283 0.256 (90.500%) ⬆️
MAX_CURSOR_UPDATE_TIME 5 3.832 2.638 1.194 (45.300%) ⬆️
VIDEO_DECODE_SEND_PACKET_TIME 144 7.580 9.466 -1.886 (-19.900%) ⬇️
MAX_VIDEO_DECODE_SEND_PACKET_TIME 5 21.358 29.247 -7.889 (-27.000%) ⬇️
VIDEO_DECODE_GET_FRAME_TIME 100327 0.001 0.001 -
MAX_VIDEO_DECODE_GET_FRAME_TIME 5 0.139 0.032 0.107 (334.400%) ⬆️
VIDEO_FPS_RENDERED 50 2.880 1.800 1.080 (60.000%) ⬆️
VIDEO_END_TO_END_LATENCY 142 2078506374505629.750 2049638230414915.500 28868144090714.250 (1.400%) ⬆️
MAX_VIDEO_END_TO_END_LATENCY 5 18446744073708232.000 18446744073707840.000 392.000 (0.000%) ⬆️
MIN_VIDEO_END_TO_END_LATENCY 5 447.000 315.000 132.000 (41.900%) ⬆️
VIDEO_RECEIVE_TIME 144 0.000 0.000 -
MAX_VIDEO_RECEIVE_TIME 5 0.001 0.001 -
VIDEO_RENDER_TIME 470 1.020 1.206 -0.186 (-15.400%) ⬇️
MAX_VIDEO_RENDER_TIME 5 28.931 28.966 -0.035 (-0.100%) ⬇️
VIDEO_TIME_BETWEEN_FRAMES 141 498.202 750.021 -251.819 (-33.600%) ⬇️
MAX_VIDEO_TIME_BETWEEN_FRAMES 5 10852.144 13882.684 -3030.540 (-21.800%) ⬇️
NOTIFICATIONS_RECEIVED 50 0.000 0.000 -
CLIENT_CPU_USAGE 29 32.928 44.132 -11.204 (-25.400%) ⬇️
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (dev) Delta
AUDIO_ENCODE_TIME 12971 0.181 0.192 -0.011 (-5.700%) ⬇️
MAX_AUDIO_ENCODE_TIME 13 1.813 2.672 -0.859 (-32.100%) ⬇️
HANDLE_USERINPUT_TIME 1 0.017 0.018 -0.001 (-5.600%) ⬇️
MAX_HANDLE_USERINPUT_TIME 1 0.017 0.018 -0.001 (-5.600%) ⬇️
THROTTLED_PACKET_DELAY 33468 0.956 0.986 -0.030 (-3.000%) ⬇️
MAX_THROTTLED_PACKET_DELAY 9 6.898 6.304 0.594 (9.400%) ⬆️
THROTTLED_PACKET_DELAY_RATE 33468 0.002 0.002 -
MAX_THROTTLED_PACKET_DELAY_RATE 9 1.239 1.238 0.001 (0.100%) ⬆️
VIDEO_CAPTURE_CREATE_TIME 1 306.570 326.817 -20.247 (-6.200%) ⬇️
MAX_VIDEO_CAPTURE_CREATE_TIME 1 306.570 326.817 -20.247 (-6.200%) ⬇️
VIDEO_CAPTURE_UPDATE_TIME 2 48.969 37.770 11.199 (29.700%) ⬆️
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 97.922 75.527 22.395 (29.700%) ⬆️
VIDEO_CAPTURE_SCREEN_TIME 927 3.738 3.661 0.077 (2.100%) ⬆️
MAX_VIDEO_CAPTURE_SCREEN_TIME 9 54.810 34.696 20.114 (58.000%) ⬆️
VIDEO_CAPTURE_TRANSFER_TIME 1446 3.183 3.518 -0.335 (-9.500%) ⬇️
MAX_VIDEO_CAPTURE_TRANSFER_TIME 9 24.842 23.788 1.054 (4.400%) ⬆️
VIDEO_ENCODER_UPDATE_TIME 14 5.887 4.489 1.398 (31.100%) ⬆️
MAX_VIDEO_ENCODER_UPDATE_TIME 6 80.416 86.909 -6.493 (-7.500%) ⬇️
VIDEO_ENCODE_TIME 1446 10.501 9.948 0.553 (5.600%) ⬆️
MAX_VIDEO_ENCODE_TIME 9 16.093 15.522 0.571 (3.700%) ⬆️
VIDEO_FPS_SENT 130 11.115 15.336 -4.221 (-27.500%) ⬇️
VIDEO_FPS_SKIPPED_IN_CAPTURE 130 0.569 0.593 -0.024 (-4.000%) ⬇️
VIDEO_FRAME_SIZE 1445 8905.194 8831.327 73.867 (0.800%) ⬆️
MAX_VIDEO_FRAME_SIZE 9 97756.000 146183.000 -48427.000 (-33.100%) ⬇️
VIDEO_FRAME_PROCESSING_TIME 1445 20.063 19.785 0.278 (1.400%) ⬆️
MAX_VIDEO_FRAME_PROCESSING_TIME 9 84.392 97.528 -13.136 (-13.500%) ⬇️
GET_CURSOR_TIME 1446 0.696 0.692 0.004 (0.600%) ⬆️
MAX_GET_CURSOR_TIME 9 11.880 10.373 1.507 (14.500%) ⬆️
VIDEO_INTER_FRAME_QP 1436 28.180 28.775 -0.595 (-2.100%) ⬇️
MAX_VIDEO_INTER_FRAME_QP 9 40.000 40.000 -
VIDEO_INTRA_FRAME_QP 10 31.800 32.895 -1.095 (-3.300%) ⬇️
MAX_VIDEO_INTRA_FRAME_QP 7 46.000 43.000 3.000 (7.000%) ⬆️
VIDEO_SEND_TIME 1444 16.729 16.605 0.124 (0.700%) ⬆️
MAX_VIDEO_SEND_TIME 9 85.192 100.233 -15.041 (-15.000%) ⬇️
DBUS_MSGS_RECEIVED 130 0.962 0.979 -0.017 (-1.700%) ⬇️
SERVER_CPU_USAGE 31 53.955 52.826 1.129 (2.100%) ⬆️

Results compared to branch mark/use-sanitisers-on-macos

Experiment 0 - Network conditions: normal

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-31-41 UTC 2022_03_10@15-54-29 UTC
network_conditions normal normal
using_two_instances True True
branch_name mark/use-sanitisers-on-macos mark/use-sanitisers-on-macos
github_sha cd76d7f 8fac639
server_hang_detected False False
server_failure False False
client_failure False False
client_server_connection_failure False False
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_FPS_SKIPPED 120 0.183 0.458 -0.275 (-60.000%) ⬇️
VIDEO_FPS_RENDERED 120 57.392 57.942 -0.550 (-0.900%) ⬇️
VIDEO_END_TO_END_LATENCY 6038 53.774 54.178 -0.404 (-0.700%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 12 1087.000 931.000 156.000 (16.800%) ⬆️
VIDEO_INTER_FRAME_QP 6710 16.206 16.369 -0.163 (-1.000%) ⬇️
MAX_VIDEO_INTER_FRAME_QP 12 32.000 34.000 -2.000 (-5.900%) ⬇️
VIDEO_INTRA_FRAME_QP 2 29.500 29.000 0.500 (1.700%) ⬆️
MAX_VIDEO_INTRA_FRAME_QP 1 33.000 32.000 1.000 (3.100%) ⬆️
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 1 0.042 0.043 -0.001 (-2.300%) ⬇️
AUDIO_RECEIVE_TIME 11710 0.000 0.000 -
MAX_AUDIO_RECEIVE_TIME 12 0.029 0.026 0.003 (11.500%) ⬆️
AUDIO_FPS_SKIPPED 120 0.183 0.458 -0.275 (-60.000%) ⬇️
READ_PACKET_TIME_TCP 4886 3.682 3.666 0.016 (0.400%) ⬆️
MAX_READ_PACKET_TIME_TCP 12 11.968 11.929 0.039 (0.300%) ⬆️
READ_PACKET_TIME_UDP 208948 0.571 0.570 0.001 (0.200%) ⬆️
MAX_READ_PACKET_TIME_UDP 12 11.706 11.587 0.119 (1.000%) ⬆️
HANDLE_SERVER_MESSAGE_TIME_TCP 3 0.018 0.021 -0.003 (-14.300%) ⬇️
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 1 0.023 0.030 -0.007 (-23.300%) ⬇️
CURSOR_UPDATE_TIME 6 0.164 0.107 0.057 (53.300%) ⬆️
MAX_CURSOR_UPDATE_TIME 1 0.273 0.280 -0.007 (-2.500%) ⬇️
VIDEO_DECODE_SEND_PACKET_TIME 6886 8.760 9.002 -0.242 (-2.700%) ⬇️
MAX_VIDEO_DECODE_SEND_PACKET_TIME 12 42.663 49.225 -6.562 (-13.300%) ⬇️
VIDEO_DECODE_GET_FRAME_TIME 108936 0.001 0.001 -
MAX_VIDEO_DECODE_GET_FRAME_TIME 12 0.196 0.164 0.032 (19.500%) ⬆️
VIDEO_FPS_RENDERED 120 57.392 57.942 -0.550 (-0.900%) ⬇️
VIDEO_END_TO_END_LATENCY 6038 53.774 54.178 -0.404 (-0.700%) ⬇️
MAX_VIDEO_END_TO_END_LATENCY 12 1087.000 931.000 156.000 (16.800%) ⬆️
MIN_VIDEO_END_TO_END_LATENCY 12 12.000 13.000 -1.000 (-7.700%) ⬇️
VIDEO_RECEIVE_TIME 6887 0.000 0.000 -
MAX_VIDEO_RECEIVE_TIME 12 0.014 0.015 -0.001 (-6.700%) ⬇️
VIDEO_RENDER_TIME 12170 0.964 1.105 -0.141 (-12.800%) ⬇️
MAX_VIDEO_RENDER_TIME 12 30.501 33.101 -2.600 (-7.900%) ⬇️
VIDEO_TIME_BETWEEN_FRAMES 6037 19.391 19.591 -0.200 (-1.000%) ⬇️
MAX_VIDEO_TIME_BETWEEN_FRAMES 12 207.420 573.290 -365.870 (-63.800%) ⬇️
NOTIFICATIONS_RECEIVED 120 0.000 0.000 -
CLIENT_CPU_USAGE 102 14.558 14.224 0.334 (2.300%) ⬆️
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_ENCODE_TIME 11931 0.196 0.198 -0.002 (-1.000%) ⬇️
MAX_AUDIO_ENCODE_TIME 12 2.587 1.964 0.623 (31.700%) ⬆️
HANDLE_USERINPUT_TIME 1 0.018 0.016 0.002 (12.500%) ⬆️
MAX_HANDLE_USERINPUT_TIME 1 0.018 0.016 0.002 (12.500%) ⬆️
THROTTLED_PACKET_DELAY 346118 0.057 0.058 -0.001 (-1.700%) ⬇️
MAX_THROTTLED_PACKET_DELAY 12 6.139 5.140 0.999 (19.400%) ⬆️
THROTTLED_PACKET_DELAY_RATE 346118 0.000 0.000 -
MAX_THROTTLED_PACKET_DELAY_RATE 12 1.192 1.194 -0.002 (-0.200%) ⬇️
VIDEO_CAPTURE_CREATE_TIME 1 2341.988 351.435 1990.553 (566.400%) ⬆️
MAX_VIDEO_CAPTURE_CREATE_TIME 1 2341.988 351.435 1990.553 (566.400%) ⬆️
VIDEO_CAPTURE_UPDATE_TIME 2 30.033 48.812 -18.779 (-38.500%) ⬇️
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 60.053 97.606 -37.553 (-38.500%) ⬇️
VIDEO_CAPTURE_SCREEN_TIME 3309 3.711 3.770 -0.059 (-1.600%) ⬇️
MAX_VIDEO_CAPTURE_SCREEN_TIME 12 76.569 40.107 36.462 (90.900%) ⬆️
VIDEO_CAPTURE_TRANSFER_TIME 6712 3.069 3.077 -0.008 (-0.300%) ⬇️
MAX_VIDEO_CAPTURE_TRANSFER_TIME 12 23.000 23.448 -0.448 (-1.900%) ⬇️
VIDEO_ENCODER_UPDATE_TIME 11 8.122 7.485 0.637 (8.500%) ⬆️
MAX_VIDEO_ENCODER_UPDATE_TIME 2 87.824 80.768 7.056 (8.700%) ⬆️
VIDEO_ENCODE_TIME 6711 9.868 9.759 0.109 (1.100%) ⬆️
MAX_VIDEO_ENCODE_TIME 12 14.947 16.585 -1.638 (-9.900%) ⬇️
VIDEO_FPS_SENT 120 55.925 56.558 -0.633 (-1.100%) ⬇️
VIDEO_FPS_SKIPPED_IN_CAPTURE 120 0.358 0.533 -0.175 (-32.800%) ⬇️
VIDEO_FRAME_SIZE 6711 30889.349 30951.386 -62.037 (-0.200%) ⬇️
MAX_VIDEO_FRAME_SIZE 12 345575.000 436160.000 -90585.000 (-20.800%) ⬇️
VIDEO_FRAME_PROCESSING_TIME 6711 13.819 13.723 0.096 (0.700%) ⬆️
MAX_VIDEO_FRAME_PROCESSING_TIME 12 82.552 88.813 -6.261 (-7.000%) ⬇️
GET_CURSOR_TIME 6711 0.531 0.541 -0.010 (-1.800%) ⬇️
MAX_GET_CURSOR_TIME 12 29.184 22.928 6.256 (27.300%) ⬆️
VIDEO_INTER_FRAME_QP 6710 16.206 16.369 -0.163 (-1.000%) ⬇️
MAX_VIDEO_INTER_FRAME_QP 12 32.000 34.000 -2.000 (-5.900%) ⬇️
VIDEO_INTRA_FRAME_QP 2 29.500 29.000 0.500 (1.700%) ⬆️
MAX_VIDEO_INTRA_FRAME_QP 1 33.000 32.000 1.000 (3.100%) ⬆️
VIDEO_SEND_TIME 6711 3.161 3.210 -0.049 (-1.500%) ⬇️
MAX_VIDEO_SEND_TIME 12 85.396 95.309 -9.913 (-10.400%) ⬇️
DBUS_MSGS_RECEIVED 120 0.758 1.067 -0.309 (-29.000%) ⬇️
SERVER_CPU_USAGE 99 57.324 56.327 0.997 (1.800%) ⬆️

Experiment 1 - Network conditions: 1Mbit,none,none

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-41-46 UTC not found
network_conditions 1Mbit,none,none not found
using_two_instances True not found
branch_name mark/use-sanitisers-on-macos not found
github_sha cd76d7f not found
server_hang_detected False not found
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_FPS_SKIPPED 80 0.438 N/A N/A
VIDEO_FPS_RENDERED 80 2.600 N/A N/A
VIDEO_END_TO_END_LATENCY 207 3384.957 N/A N/A
MAX_VIDEO_END_TO_END_LATENCY 8 8428.000 N/A N/A
VIDEO_INTER_FRAME_QP 2132 28.617 N/A N/A
MAX_VIDEO_INTER_FRAME_QP 13 40.000 N/A N/A
VIDEO_INTRA_FRAME_QP 13 31.923 N/A N/A
MAX_VIDEO_INTRA_FRAME_QP 9 46.000 N/A N/A
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 9 0.201 N/A N/A
AUDIO_RECEIVE_TIME 415 0.001 N/A N/A
MAX_AUDIO_RECEIVE_TIME 8 0.032 N/A N/A
AUDIO_FPS_SKIPPED 80 0.438 N/A N/A
READ_PACKET_TIME_TCP 2122 3.415 N/A N/A
MAX_READ_PACKET_TIME_TCP 8 11.889 N/A N/A
READ_PACKET_TIME_UDP 9922 5.232 N/A N/A
MAX_READ_PACKET_TIME_UDP 8 11.494 N/A N/A
HANDLE_SERVER_MESSAGE_TIME_TCP 10 0.015 N/A N/A
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 8 0.023 N/A N/A
CURSOR_UPDATE_TIME 13 0.375 N/A N/A
MAX_CURSOR_UPDATE_TIME 8 4.394 N/A N/A
VIDEO_DECODE_SEND_PACKET_TIME 208 7.603 N/A N/A
MAX_VIDEO_DECODE_SEND_PACKET_TIME 8 22.456 N/A N/A
VIDEO_DECODE_GET_FRAME_TIME 137510 0.001 N/A N/A
MAX_VIDEO_DECODE_GET_FRAME_TIME 8 0.054 N/A N/A
VIDEO_FPS_RENDERED 80 2.600 N/A N/A
VIDEO_END_TO_END_LATENCY 207 3384.957 N/A N/A
MAX_VIDEO_END_TO_END_LATENCY 8 8428.000 N/A N/A
MIN_VIDEO_END_TO_END_LATENCY 8 232.000 N/A N/A
VIDEO_RECEIVE_TIME 208 0.000 N/A N/A
MAX_VIDEO_RECEIVE_TIME 8 0.001 N/A N/A
VIDEO_RENDER_TIME 700 1.082 N/A N/A
MAX_VIDEO_RENDER_TIME 8 31.350 N/A N/A
VIDEO_TIME_BETWEEN_FRAMES 206 517.678 N/A N/A
MAX_VIDEO_TIME_BETWEEN_FRAMES 8 11054.678 N/A N/A
NOTIFICATIONS_RECEIVED 80 0.000 N/A N/A
CLIENT_CPU_USAGE 43 39.284 N/A N/A
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_ENCODE_TIME 12972 0.182 N/A N/A
MAX_AUDIO_ENCODE_TIME 13 1.858 N/A N/A
HANDLE_USERINPUT_TIME 1 0.016 N/A N/A
MAX_HANDLE_USERINPUT_TIME 1 0.016 N/A N/A
THROTTLED_PACKET_DELAY 46541 1.004 N/A N/A
MAX_THROTTLED_PACKET_DELAY 13 7.273 N/A N/A
THROTTLED_PACKET_DELAY_RATE 46541 0.002 N/A N/A
MAX_THROTTLED_PACKET_DELAY_RATE 13 1.241 N/A N/A
VIDEO_CAPTURE_CREATE_TIME 1 333.885 N/A N/A
MAX_VIDEO_CAPTURE_CREATE_TIME 1 333.885 N/A N/A
VIDEO_CAPTURE_UPDATE_TIME 2 43.775 N/A N/A
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 87.537 N/A N/A
VIDEO_CAPTURE_SCREEN_TIME 1356 3.602 N/A N/A
MAX_VIDEO_CAPTURE_SCREEN_TIME 13 50.013 N/A N/A
VIDEO_CAPTURE_TRANSFER_TIME 2145 3.145 N/A N/A
MAX_VIDEO_CAPTURE_TRANSFER_TIME 13 23.159 N/A N/A
VIDEO_ENCODER_UPDATE_TIME 20 4.667 N/A N/A
MAX_VIDEO_ENCODER_UPDATE_TIME 10 90.523 N/A N/A
VIDEO_ENCODE_TIME 2145 10.311 N/A N/A
MAX_VIDEO_ENCODE_TIME 13 15.165 N/A N/A
VIDEO_FPS_SENT 130 16.500 N/A N/A
VIDEO_FPS_SKIPPED_IN_CAPTURE 130 0.638 N/A N/A
VIDEO_FRAME_SIZE 2145 8547.924 N/A N/A
MAX_VIDEO_FRAME_SIZE 13 224960.000 N/A N/A
VIDEO_FRAME_PROCESSING_TIME 2145 19.888 N/A N/A
MAX_VIDEO_FRAME_PROCESSING_TIME 13 146.941 N/A N/A
GET_CURSOR_TIME 2145 0.652 N/A N/A
MAX_GET_CURSOR_TIME 13 17.775 N/A N/A
VIDEO_INTER_FRAME_QP 2132 28.617 N/A N/A
MAX_VIDEO_INTER_FRAME_QP 13 40.000 N/A N/A
VIDEO_INTRA_FRAME_QP 13 31.923 N/A N/A
MAX_VIDEO_INTRA_FRAME_QP 9 46.000 N/A N/A
VIDEO_SEND_TIME 2144 16.701 N/A N/A
MAX_VIDEO_SEND_TIME 13 155.390 N/A N/A
DBUS_MSGS_RECEIVED 130 1.085 N/A N/A
SERVER_CPU_USAGE 46 51.085 N/A N/A

Experiment 2 - Network conditions: 1Mbit,none,0.1

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-45-37 UTC not found
network_conditions 1Mbit,none,0.1 not found
using_two_instances True not found
branch_name mark/use-sanitisers-on-macos not found
github_sha cd76d7f not found
server_hang_detected False not found
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_FPS_SKIPPED 30 0.467 N/A N/A
VIDEO_FPS_RENDERED 30 3.067 N/A N/A
VIDEO_END_TO_END_LATENCY 90 3020.911 N/A N/A
MAX_VIDEO_END_TO_END_LATENCY 3 7769.000 N/A N/A
VIDEO_INTER_FRAME_QP 959 28.619 N/A N/A
MAX_VIDEO_INTER_FRAME_QP 6 40.000 N/A N/A
VIDEO_INTRA_FRAME_QP 7 35.286 N/A N/A
MAX_VIDEO_INTRA_FRAME_QP 4 46.000 N/A N/A
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 4 0.171 N/A N/A
AUDIO_RECEIVE_TIME 187 0.001 N/A N/A
MAX_AUDIO_RECEIVE_TIME 3 0.027 N/A N/A
AUDIO_FPS_SKIPPED 30 0.467 N/A N/A
READ_PACKET_TIME_TCP 845 3.334 N/A N/A
MAX_READ_PACKET_TIME_TCP 3 11.881 N/A N/A
READ_PACKET_TIME_UDP 4016 5.142 N/A N/A
MAX_READ_PACKET_TIME_UDP 3 11.470 N/A N/A
HANDLE_SERVER_MESSAGE_TIME_TCP 4 0.021 N/A N/A
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 3 0.027 N/A N/A
CURSOR_UPDATE_TIME 5 0.622 N/A N/A
MAX_CURSOR_UPDATE_TIME 3 2.747 N/A N/A
VIDEO_DECODE_SEND_PACKET_TIME 92 7.019 N/A N/A
MAX_VIDEO_DECODE_SEND_PACKET_TIME 3 17.807 N/A N/A
VIDEO_DECODE_GET_FRAME_TIME 54873 0.001 N/A N/A
MAX_VIDEO_DECODE_GET_FRAME_TIME 3 0.021 N/A N/A
VIDEO_FPS_RENDERED 30 3.067 N/A N/A
VIDEO_END_TO_END_LATENCY 90 3020.911 N/A N/A
MAX_VIDEO_END_TO_END_LATENCY 3 7769.000 N/A N/A
MIN_VIDEO_END_TO_END_LATENCY 3 222.000 N/A N/A
VIDEO_RECEIVE_TIME 92 0.000 N/A N/A
MAX_VIDEO_RECEIVE_TIME 3 0.001 N/A N/A
VIDEO_RENDER_TIME 265 0.970 N/A N/A
MAX_VIDEO_RENDER_TIME 3 28.719 N/A N/A
VIDEO_TIME_BETWEEN_FRAMES 89 385.917 N/A N/A
MAX_VIDEO_TIME_BETWEEN_FRAMES 3 8182.096 N/A N/A
NOTIFICATIONS_RECEIVED 30 0.000 N/A N/A
CLIENT_CPU_USAGE 17 25.629 N/A N/A
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_ENCODE_TIME 11946 0.180 N/A N/A
MAX_AUDIO_ENCODE_TIME 12 2.631 N/A N/A
HANDLE_USERINPUT_TIME 1 0.014 N/A N/A
MAX_HANDLE_USERINPUT_TIME 1 0.014 N/A N/A
THROTTLED_PACKET_DELAY 22595 0.943 N/A N/A
MAX_THROTTLED_PACKET_DELAY 6 8.555 N/A N/A
THROTTLED_PACKET_DELAY_RATE 22595 0.002 N/A N/A
MAX_THROTTLED_PACKET_DELAY_RATE 6 1.422 N/A N/A
VIDEO_CAPTURE_CREATE_TIME 1 355.062 N/A N/A
MAX_VIDEO_CAPTURE_CREATE_TIME 1 355.062 N/A N/A
VIDEO_CAPTURE_UPDATE_TIME 2 41.784 N/A N/A
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 83.553 N/A N/A
VIDEO_CAPTURE_SCREEN_TIME 602 3.717 N/A N/A
MAX_VIDEO_CAPTURE_SCREEN_TIME 6 100.303 N/A N/A
VIDEO_CAPTURE_TRANSFER_TIME 966 3.106 N/A N/A
MAX_VIDEO_CAPTURE_TRANSFER_TIME 6 25.138 N/A N/A
VIDEO_ENCODER_UPDATE_TIME 10 9.635 N/A N/A
MAX_VIDEO_ENCODER_UPDATE_TIME 5 95.025 N/A N/A
VIDEO_ENCODE_TIME 966 10.492 N/A N/A
MAX_VIDEO_ENCODE_TIME 6 15.943 N/A N/A
VIDEO_FPS_SENT 120 8.042 N/A N/A
VIDEO_FPS_SKIPPED_IN_CAPTURE 120 0.483 N/A N/A
VIDEO_FRAME_SIZE 965 8336.255 N/A N/A
MAX_VIDEO_FRAME_SIZE 6 69582.000 N/A N/A
VIDEO_FRAME_PROCESSING_TIME 965 20.049 N/A N/A
MAX_VIDEO_FRAME_PROCESSING_TIME 6 74.343 N/A N/A
GET_CURSOR_TIME 966 0.789 N/A N/A
MAX_GET_CURSOR_TIME 6 13.950 N/A N/A
VIDEO_INTER_FRAME_QP 959 28.619 N/A N/A
MAX_VIDEO_INTER_FRAME_QP 6 40.000 N/A N/A
VIDEO_INTRA_FRAME_QP 7 35.286 N/A N/A
MAX_VIDEO_INTRA_FRAME_QP 4 46.000 N/A N/A
VIDEO_SEND_TIME 964 16.233 N/A N/A
MAX_VIDEO_SEND_TIME 6 80.200 N/A N/A
DBUS_MSGS_RECEIVED 120 0.975 N/A N/A
SERVER_CPU_USAGE 21 58.338 N/A N/A

Experiment 3 - Network conditions: 1Mbit,100,0.1

Experiment metadata - Expand here
Experiment metadata:
Key (this run) Value (this run) Value (compared run)
start_time 2022_03_10@17-49-14 UTC not found
network_conditions 1Mbit,100,0.1 not found
using_two_instances True not found
branch_name mark/use-sanitisers-on-macos not found
github_sha cd76d7f not found
server_hang_detected False not found
server_failure False not found
client_failure False not found
client_server_connection_failure False not found
Experiment results - Expand here
SUMMARY OF MOST INTERESTING METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_FPS_SKIPPED 50 0.38 N/A N/A
VIDEO_FPS_RENDERED 50 2.88 N/A N/A
VIDEO_END_TO_END_LATENCY 142 2078506374505629.75 N/A N/A
MAX_VIDEO_END_TO_END_LATENCY 5 18446744073708232.00 N/A N/A
VIDEO_INTER_FRAME_QP 1436 28.18 N/A N/A
MAX_VIDEO_INTER_FRAME_QP 9 40.00 N/A N/A
VIDEO_INTRA_FRAME_QP 10 31.80 N/A N/A
MAX_VIDEO_INTRA_FRAME_QP 7 46.00 N/A N/A
CLIENT METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
HANDSHAKE_CONNECT_TO_SERVER_TIME 6 1.043 N/A N/A
AUDIO_RECEIVE_TIME 315 0.001 N/A N/A
MAX_AUDIO_RECEIVE_TIME 5 0.026 N/A N/A
AUDIO_FPS_SKIPPED 50 0.380 N/A N/A
READ_PACKET_TIME_TCP 1534 3.435 N/A N/A
MAX_READ_PACKET_TIME_TCP 5 11.901 N/A N/A
READ_PACKET_TIME_UDP 7187 5.218 N/A N/A
MAX_READ_PACKET_TIME_UDP 5 11.507 N/A N/A
HANDLE_SERVER_MESSAGE_TIME_TCP 5 0.018 N/A N/A
MAX_HANDLE_SERVER_MESSAGE_TIME_TCP 5 0.027 N/A N/A
CURSOR_UPDATE_TIME 8 0.539 N/A N/A
MAX_CURSOR_UPDATE_TIME 5 3.832 N/A N/A
VIDEO_DECODE_SEND_PACKET_TIME 144 7.580 N/A N/A
MAX_VIDEO_DECODE_SEND_PACKET_TIME 5 21.358 N/A N/A
VIDEO_DECODE_GET_FRAME_TIME 100327 0.001 N/A N/A
MAX_VIDEO_DECODE_GET_FRAME_TIME 5 0.139 N/A N/A
VIDEO_FPS_RENDERED 50 2.880 N/A N/A
VIDEO_END_TO_END_LATENCY 142 2078506374505629.750 N/A N/A
MAX_VIDEO_END_TO_END_LATENCY 5 18446744073708232.000 N/A N/A
MIN_VIDEO_END_TO_END_LATENCY 5 447.000 N/A N/A
VIDEO_RECEIVE_TIME 144 0.000 N/A N/A
MAX_VIDEO_RECEIVE_TIME 5 0.001 N/A N/A
VIDEO_RENDER_TIME 470 1.020 N/A N/A
MAX_VIDEO_RENDER_TIME 5 28.931 N/A N/A
VIDEO_TIME_BETWEEN_FRAMES 141 498.202 N/A N/A
MAX_VIDEO_TIME_BETWEEN_FRAMES 5 10852.144 N/A N/A
NOTIFICATIONS_RECEIVED 50 0.000 N/A N/A
CLIENT_CPU_USAGE 29 32.928 N/A N/A
SERVER METRICS:
Metric Entries (this branch) Average (this branch) Average (mark/use-sanitisers-on-macos) Delta
AUDIO_ENCODE_TIME 12971 0.181 N/A N/A
MAX_AUDIO_ENCODE_TIME 13 1.813 N/A N/A
HANDLE_USERINPUT_TIME 1 0.017 N/A N/A
MAX_HANDLE_USERINPUT_TIME 1 0.017 N/A N/A
THROTTLED_PACKET_DELAY 33468 0.956 N/A N/A
MAX_THROTTLED_PACKET_DELAY 9 6.898 N/A N/A
THROTTLED_PACKET_DELAY_RATE 33468 0.002 N/A N/A
MAX_THROTTLED_PACKET_DELAY_RATE 9 1.239 N/A N/A
VIDEO_CAPTURE_CREATE_TIME 1 306.570 N/A N/A
MAX_VIDEO_CAPTURE_CREATE_TIME 1 306.570 N/A N/A
VIDEO_CAPTURE_UPDATE_TIME 2 48.969 N/A N/A
MAX_VIDEO_CAPTURE_UPDATE_TIME 1 97.922 N/A N/A
VIDEO_CAPTURE_SCREEN_TIME 927 3.738 N/A N/A
MAX_VIDEO_CAPTURE_SCREEN_TIME 9 54.810 N/A N/A
VIDEO_CAPTURE_TRANSFER_TIME 1446 3.183 N/A N/A
MAX_VIDEO_CAPTURE_TRANSFER_TIME 9 24.842 N/A N/A
VIDEO_ENCODER_UPDATE_TIME 14 5.887 N/A N/A
MAX_VIDEO_ENCODER_UPDATE_TIME 6 80.416 N/A N/A
VIDEO_ENCODE_TIME 1446 10.501 N/A N/A
MAX_VIDEO_ENCODE_TIME 9 16.093 N/A N/A
VIDEO_FPS_SENT 130 11.115 N/A N/A
VIDEO_FPS_SKIPPED_IN_CAPTURE 130 0.569 N/A N/A
VIDEO_FRAME_SIZE 1445 8905.194 N/A N/A
MAX_VIDEO_FRAME_SIZE 9 97756.000 N/A N/A
VIDEO_FRAME_PROCESSING_TIME 1445 20.063 N/A N/A
MAX_VIDEO_FRAME_PROCESSING_TIME 9 84.392 N/A N/A
GET_CURSOR_TIME 1446 0.696 N/A N/A
MAX_GET_CURSOR_TIME 9 11.880 N/A N/A
VIDEO_INTER_FRAME_QP 1436 28.180 N/A N/A
MAX_VIDEO_INTER_FRAME_QP 9 40.000 N/A N/A
VIDEO_INTRA_FRAME_QP 10 31.800 N/A N/A
MAX_VIDEO_INTRA_FRAME_QP 7 46.000 N/A N/A
VIDEO_SEND_TIME 1444 16.729 N/A N/A
MAX_VIDEO_SEND_TIME 9 85.192 N/A N/A
DBUS_MSGS_RECEIVED 130 0.962 N/A N/A
SERVER_CPU_USAGE 31 53.955 N/A N/A

@philippemnoel philippemnoel merged commit 90ca634 into dev Mar 10, 2022
@philippemnoel philippemnoel deleted the mark/use-sanitisers-on-macos branch March 10, 2022 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📁 Repo: protocol This PR/Issue modifies /protocol code 📁 Repo: workflows This PR/Issue modifies /workflows code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants