tests: do not exclude pipewire tests on Whonix 17

[marmarek]

Whonix 17 is not Debian 11

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.39%. Comparing base (a044cf5) to head (9817c5c).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #600   +/-   ##
=======================================
  Coverage   68.39%   68.39%           
=======================================
  Files          56       56           
  Lines       11236    11236           
=======================================
  Hits         7685     7685           
  Misses       3551     3551           

Flag	Coverage Δ
unittests	68.39% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[qubesos-bot]

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2024061618-4.3&flavor=pull-requests

Test run included the following:

• Fix guivm-vnc service name for sys-gui-vnc qubes-mgmt-salt-dom0-virtual-machines#62 (QubesOS/qubes-mgmt-salt-dom0-virtual-machines@2cbaa94)
• update to new device API qubes-desktop-linux-manager#184 (QubesOS/qubes-desktop-linux-manager@6c57e91)
• Trigger build (don't merge) qubes-linux-utils#115 (QubesOS/qubes-linux-utils@659280e)
• Test "unsafe" file copy #596 (0aa1382)
• tests: do not exclude pipewire tests on Whonix 17 #600 (9817c5c)
• Fix VNC GUIVM qubes-gui-agent-linux#213 (QubesOS/qubes-gui-agent-linux@4b71d95)
• new device API qubes-app-linux-usb-proxy#32 (QubesOS/qubes-app-linux-usb-proxy@227ec98)
• Add basic SELinux policy qubes-gui-daemon#143 (QubesOS/qubes-gui-daemon@e585798)
• Allow unsafe characters if invoked as qubes.UnsafeFileCopy qubes-core-agent-linux#497 (QubesOS/qubes-core-agent-linux@ef3f478)

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2024052808-4.3&flavor=update

• system_tests_guivm_vnc_gui_interactive

  • clipboard_and_web: unnamed test (unknown)
  • clipboard_and_web: Failed (test died)
    # Test died: no candidate needle with tag(s) 'clipboard-copy-notifi...

• system_tests_basic_vm_qrexec_gui_xfs

  • TC_20_AudioVM_PipeWire_whonix-workstation-17-pool: test_251_audio_playback_audiovm_pipewire_late_start (failure)
    AssertionError: too short audio, expected 10s, got 9.26326530612244...

Failed tests

12 failures

• system_tests_pvgrub_salt_storage

  • [unstable] TC_41_HVMGrub_debian-12-xfce: test_000_standalone_vm (error)
    qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

  • [unstable] TC_41_HVMGrub_debian-12-xfce: test_010_template_based_vm (error)
    qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

  • TC_41_HVMGrub_fedora-40-xfce: test_000_standalone_vm (error)
    qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

  • TC_41_HVMGrub_fedora-40-xfce: test_010_template_based_vm (error)
    qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

• system_tests_splitgpg

  • TC_10_Thunderbird_whonix-workstation-17: test_020_send_receive_inline_with_attachment (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

• system_tests_extra

  • [unstable] TC_00_QVCTest_fedora-40-xfce: test_020_webcam (failure + cleanup)
    AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

  • [unstable] TC_00_QVCTest_whonix-workstation-17: test_020_webcam (failure + cleanup)
    self.assertNotEqual(vm.run('test -e /dev/vid... AssertionError: 0 == 0

• system_tests_guivm_vnc_gui_interactive

  • clipboard_and_web: unnamed test (unknown)
  • clipboard_and_web: Failed (test died)
    # Test died: no candidate needle with tag(s) 'clipboard-copy-notifi...

• system_tests_basic_vm_qrexec_gui_btrfs

  • [unstable] TC_20_AudioVM_Pulse_fedora-40-xfce-pool: test_225_audio_rec_unmuted_hvm (failure)
    AssertionError: too short audio, expected 10s, got 7.30310657596371...

• system_tests_basic_vm_qrexec_gui_ext4

  • [unstable] TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: too short audio, expected 10s, got 7.46013605442176...

• system_tests_basic_vm_qrexec_gui_xfs

  • TC_20_AudioVM_PipeWire_whonix-workstation-17-pool: test_251_audio_playback_audiovm_pipewire_late_start (failure)
    AssertionError: too short audio, expected 10s, got 9.26326530612244...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/101100#dependencies

39 fixed

• system_tests_basic_vm_qrexec_gui

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_220_audio_play_pulseaudio (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_222_audio_rec_unmuted_pulseaudio (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_223_audio_play_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_pvgrub_salt_storage

  • StorageFile: test_001_non_volatile (error)
    subprocess.CalledProcessError: Command '/usr/lib/qubes/destroy-snap...

• system_tests_extra

  • TC_00_QVCTest_debian-12-xfce: test_020_webcam (failure)
    AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

  • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
    self.assertNotEqual(vm.run('test -e /dev/vid... AssertionError: 0 == 0

• system_tests_usbproxy

  • TC_20_USBProxy_core3_whonix-gateway-17: test_070_attach_not_installed_front (error)
    qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...

• system_tests_network_updates

  • TC_00_Dom0Upgrade_whonix-gateway-17: test_006_update_flag_clear (failure)
    Error: Failed to download metadata for repo 'test': Cannot download...

  • TC_10_QvmTemplate_debian-12-xfce: test_010_template_install (failure)
    AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • TC_10_QvmTemplate_fedora-40-xfce: test_010_template_install (failure)
    AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • TC_10_QvmTemplate_whonix-gateway-17: test_010_template_install (failure)
    AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • TC_11_QvmTemplateMgmtVM_debian-12-xfce: test_010_template_install (failure)
    AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • TC_11_QvmTemplateMgmtVM_fedora-40-xfce: test_010_template_install (failure)
    AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • TC_11_QvmTemplateMgmtVM_whonix-gateway-17: test_010_template_install (failure)
    AssertionError: qvm-template failed: Downloading 'qubes-template-de...

• system_tests_basic_vm_qrexec_gui_zfs

  • TC_00_Basic: test_120_start_standalone_with_cdrom_dom0 (failure)
    AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop6 device to...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_220_audio_play_pulseaudio (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_222_audio_rec_unmuted_pulseaudio (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_223_audio_play_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_basic_vm_qrexec_gui_btrfs

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_220_audio_play_pulseaudio (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_222_audio_rec_unmuted_pulseaudio (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_223_audio_play_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_basic_vm_qrexec_gui_ext4

  • TC_00_Basic: test_141_libvirt_objects_reconnect (failure)
    AssertionError: libvirt event impl drain timeout

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_220_audio_play_pulseaudio (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_222_audio_rec_unmuted_pulseaudio (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_223_audio_play_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_basic_vm_qrexec_gui_xfs

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_220_audio_play_pulseaudio (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_222_audio_rec_unmuted_pulseaudio (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_223_audio_play_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17-pool: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_basic_vm_qrexec_gui@hw1

  • TC_00_Basic: test_204_udev_block_exclude_custom_file (error)
    subprocess.CalledProcessError: Command '/usr/lib/qubes/destroy-snap...

  • TC_20_AudioVM_Pulse_debian-12-xfce: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_220_audio_play_pulseaudio (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_222_audio_rec_unmuted_pulseaudio (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_223_audio_play_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_guivm_gui_interactive

  • update_guivm: Failed (test died)
    # Test died: command '(set -o pipefail; qubesctl --all --show-outpu...

Unstable tests

• system_tests_suspend

  suspend/ (2/4 times with errors)

  • job 102431 None
  • job 102443 None
  suspend/Failed (2/4 times with errors)

  • job 102431 # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  • job 102443 # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  suspend/wait_serial (2/4 times with errors)

  • job 102431 # wait_serial expected: qr/2E8vz-\d+-/...
  • job 102443 # wait_serial expected: qr/2E8vz-\d+-/...

• system_tests_basic_vm_qrexec_gui

  TC_20_AudioVM_Pulse_whonix-workstation-17/test_220_audio_play_pulseaudio (2/2 times with errors)

  • job 101109 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  • job 101758 AssertionError: too short audio, expected 10s, got 8.73532879818594...
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_222_audio_rec_unmuted_pulseaudio (2/2 times with errors)

  • job 101109 AssertionError: only silence detected, no useful audio data
  • job 101758 AssertionError: too short audio, expected 10s, got 9.36446712018140...
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_223_audio_play_hvm (1/2 times with errors)

  • job 101109 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_252_audio_playback_audiovm_switch_hvm (1/2 times with errors)

  • job 101109 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_pvgrub_salt_storage

  TC_41_HVMGrub_debian-12-xfce/test_000_standalone_vm (1/2 times with errors)

  • job 101773 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  StorageFile/test_001_non_volatile (1/2 times with errors)

  • job 101124 subprocess.CalledProcessError: Command '/usr/lib/qubes/destroy-snap...
  TC_41_HVMGrub_debian-12-xfce/test_010_template_based_vm (1/2 times with errors)

  • job 101773 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

• system_tests_extra

  TC_00_QVCTest_whonix-workstation-17/test_010_screenshare (1/2 times with errors)

  • job 101116 self.assertNotEqual(vm.run('test -e /dev/vid... AssertionError: 0 == 0
  TC_00_QVCTest_debian-12-xfce/test_020_webcam (1/2 times with errors)

  • job 101116 AssertionError: 'qubes-video-companion webcam' exited early (0): b'...
  TC_00_QVCTest_fedora-40-xfce/test_020_webcam (1/2 times with errors)

  • job 101765 AssertionError: 'qubes-video-companion webcam' exited early (0): b'...
  TC_00_QVCTest_whonix-workstation-17/test_020_webcam (1/2 times with errors)

  • job 101116 AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

• system_tests_usbproxy

  TC_20_USBProxy_core3_whonix-gateway-17/test_070_attach_not_installed_front (1/2 times with errors)

  • job 101101 qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...

• system_tests_network_updates

  TC_00_Dom0Upgrade_whonix-gateway-17/test_006_update_flag_clear (1/2 times with errors)

  • job 101123 Error: Failed to download metadata for repo 'test': Cannot download...
  TC_10_QvmTemplate_debian-12-xfce/test_010_template_install (1/2 times with errors)

  • job 101123 AssertionError: qvm-template failed: Downloading 'qubes-template-de...
  TC_10_QvmTemplate_fedora-40-xfce/test_010_template_install (1/2 times with errors)

  • job 101123 AssertionError: qvm-template failed: Downloading 'qubes-template-de...
  TC_10_QvmTemplate_whonix-gateway-17/test_010_template_install (1/2 times with errors)

  • job 101123 AssertionError: qvm-template failed: Downloading 'qubes-template-de...
  TC_11_QvmTemplateMgmtVM_debian-12-xfce/test_010_template_install (1/2 times with errors)

  • job 101123 AssertionError: qvm-template failed: Downloading 'qubes-template-de...
  TC_11_QvmTemplateMgmtVM_fedora-40-xfce/test_010_template_install (1/2 times with errors)

  • job 101123 AssertionError: qvm-template failed: Downloading 'qubes-template-de...
  TC_11_QvmTemplateMgmtVM_whonix-gateway-17/test_010_template_install (1/2 times with errors)

  • job 101123 AssertionError: qvm-template failed: Downloading 'qubes-template-de...

• system_tests_dispvm

  TC_20_DispVM_fedora-40-xfce/test_100_open_in_dispvm (1/2 times with errors)

  • job 101764 AssertionError: './open-file test.txt' failed with ./open-file test...

• system_tests_basic_vm_qrexec_gui_btrfs

  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (1/2 times with errors)

  • job 101110 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (1/2 times with errors)

  • job 101110 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_223_audio_play_hvm (1/2 times with errors)

  • job 101110 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  TC_20_AudioVM_Pulse_fedora-40-xfce-pool/test_225_audio_rec_unmuted_hvm (1/2 times with errors)

  • job 101782 AssertionError: too short audio, expected 10s, got 6.28875283446712...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_252_audio_playback_audiovm_switch_hvm (2/2 times with errors)

  • job 101110 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  • job 101782 AssertionError: pacat for test-inst-vm1 (xid 90) running(False) in ...

• system_tests_basic_vm_qrexec_gui_ext4

  TC_00_Basic/test_141_libvirt_objects_reconnect (1/2 times with errors)

  • job 101111 AssertionError: libvirt event impl drain timeout
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (1/2 times with errors)

  • job 101111 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (1/2 times with errors)

  • job 101111 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_223_audio_play_hvm (1/2 times with errors)

  • job 101111 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_252_audio_playback_audiovm_switch_hvm (1/2 times with errors)

  • job 101111 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_basic_vm_qrexec_gui_xfs

  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (2/2 times with errors)

  • job 101112 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  • job 101781 AssertionError: too short audio, expected 10s, got 8.33947845804988...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (2/2 times with errors)

  • job 101112 AssertionError: only silence detected, no useful audio data
  • job 101781 AssertionError: too short audio, expected 10s, got 8.17784580498866...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_223_audio_play_hvm (2/2 times with errors)

  • job 101112 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  • job 101781 AssertionError: too short audio, expected 10s, got 7.71551020408163...
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_252_audio_playback_audiovm_switch_hvm (1/2 times with errors)

  • job 101112 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_basic_vm_qrexec_gui@hw1

  TC_20_AudioVM_Pulse_whonix-workstation-17/test_220_audio_play_pulseaudio (2/2 times with errors)

  • job 101109 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  • job 101758 AssertionError: too short audio, expected 10s, got 8.73532879818594...
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_222_audio_rec_unmuted_pulseaudio (2/2 times with errors)

  • job 101109 AssertionError: only silence detected, no useful audio data
  • job 101758 AssertionError: too short audio, expected 10s, got 9.36446712018140...
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_223_audio_play_hvm (1/2 times with errors)

  • job 101109 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_252_audio_playback_audiovm_switch_hvm (1/2 times with errors)

  • job 101109 AssertionError: Command 'timeout 20s paplay --format=float32le --ra...

• system_tests_suspend@hw1

  suspend/ (2/4 times with errors)

  • job 102431 None
  • job 102443 None
  suspend/Failed (2/4 times with errors)

  • job 102431 # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  • job 102443 # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  suspend/wait_serial (2/4 times with errors)

  • job 102431 # wait_serial expected: qr/2E8vz-\d+-/...
  • job 102443 # wait_serial expected: qr/2E8vz-\d+-/...