Flaky test_send_metrics_to_scheduler

[fjetter]

This test is a little too specific and is flaking

I do see various different errors like

E         At index 13 diff: ('get-data', 'network', 'seconds') != ('execute', None, 'x', 'memory-read', 'count')
E         Full diff:
E           [
E            ('execute', None, 'x', 'deserialize', 'seconds'),
E            ('execute', None, 'x', 'thread-cpu', 'seconds'),
E            ('execute', None, 'x', 'thread-noncpu', 'seconds'),
E            ('execute', None, 'x', 'executor', 'seconds'),
E            ('execute', None, 'x', 'other', 'seconds'),
E            ('get-data', 'memory-read', 'count'),
E            ('get-data', 'memory-read', 'bytes'),
E            ('get-data', 'serialize', 'seconds'),
E            ('get-data', 'compress', 'seconds'),
E            ('gather-dep', 'decompress', 'seconds'),
E            ('gather-dep', 'deserialize', 'seconds'),
E            ('gather-dep', 'network', 'seconds'),
E            ('gather-dep', 'other', 'seconds'),
E         +  ('get-data', 'network', 'seconds'),
E            ('execute', None, 'x', 'memory-read', 'count'),
E            ('execute', None, 'x', 'memory-read', 'bytes'),
E         -  ('get-data', 'network', 'seconds'),
E           ]

I do not think these tests should assert every single message.

The test report does not show any failure but I can reproduce this on main by running it many times concurrently.

cc @crusaderky

[fjetter]

This #8049 seems to make things worse