Flaky test: ui_tests/gui/test_main_window.py::test_right_click_plot_button_opens_external_plotter

[berland]

FAILED ui_tests/gui/test_main_window.py::test_right_click_plot_button_opens_external_plotter - RuntimeError: wrapped C/C++ object of type QWindow has been deleted

Failed in https://github.com/equinor/komodo-releases/actions/runs/12591094039/job/35094141266

=================================== FAILURES ===================================
_____________ test_right_click_plot_button_opens_external_plotter ______________

qtbot = <pytestqt.qtbot.QtBot object at 0x7f1b6281dc10>
storage = <ert.storage.local_storage.LocalStorage object at 0x7f1b6281d390>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f1b6281e410>

    @pytest.mark.usefixtures("use_tmpdir", "set_site_config")
    def test_right_click_plot_button_opens_external_plotter(qtbot, storage, monkeypatch):
        monkeypatch.setattr(PlotApi, "get_all_ensembles", lambda _: [])
        config_file = "minimal_config.ert"
        with open(config_file, "w", encoding="utf-8") as f:
            f.write("NUM_REALIZATIONS 1")
        args_mock = Mock()
        args_mock.config = config_file
        ert_config = ErtConfig.from_file(config_file)
        with StorageService.init_service(
            project=os.path.abspath(ert_config.ens_path),
        ):
            gui = _setup_main_window(ert_config, args_mock, GUILogHandler(), storage)
            qtbot.addWidget(gui)
    
            button_plot_tool = gui.findChild(SidebarToolButton, "button_Create_plot")
            assert button_plot_tool
    
            def top_level_plotter_windows() -> list[QWindow]:
                top_level_plot_windows = []
                top_level_windows = QApplication.topLevelWindows()
                for win in top_level_windows:
                    if "Plotting" in win.title() and win.isVisible():
                        top_level_plot_windows.append(win)
                return top_level_plot_windows
    
            def right_click_plotter_button() -> None:
                top_level_windows = len(top_level_plotter_windows())
                qtbot.mouseClick(button_plot_tool, Qt.RightButton)
                qtbot.wait_until(
                    lambda: len(top_level_plotter_windows()) > top_level_windows,
                    timeout=5000,
                )
    
>           right_click_plotter_button()

/tmp/f_scout_ci/actions-runner-02/_temp/test_root/tests/ert/ui_tests/gui/test_main_window.py:632: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/f_scout_ci/actions-runner-02/_temp/test_root/tests/ert/ui_tests/gui/test_main_window.py:625: in right_click_plotter_button
    top_level_windows = len(top_level_plotter_windows())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def top_level_plotter_windows() -> list[QWindow]:
        top_level_plot_windows = []
        top_level_windows = QApplication.topLevelWindows()
        for win in top_level_windows:
>           if "Plotting" in win.title() and win.isVisible():
E           RuntimeError: wrapped C/C++ object of type QWindow has been deleted

/tmp/f_scout_ci/actions-runner-02/_temp/test_root/tests/ert/ui_tests/gui/test_main_window.py:620: RuntimeError

[eivindjahren]

The problem seems to be that QApplication.topLevelWindows() may return a window that has been opened but not yet completely closed by a different test, which is then closed and so destroyed between getting it and testing whether it is a plotter window.

Ideally, other tests should not keep windows open and we could create an autouse fixture for that. Also, this test should probably not use this method to get the plotting windows.

This failure seems to happen very rarely, so giving it a low criticality for now.