Skip to content

Add mechanism to set a multiplier for test timeouts #4022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hololeap opened this issue Jan 28, 2024 · 1 comment
Closed

Add mechanism to set a multiplier for test timeouts #4022

hololeap opened this issue Jan 28, 2024 · 1 comment
Labels
type: enhancement New feature or request

Comments

@hololeap
Copy link
Contributor

hololeap commented Jan 28, 2024

Is your enhancement request related to a problem? Please describe.

I have been handling packaging haskell-language-server for Gentoo Linux, and as such I try to run the test suite bundled with each plugin. While there have been some other issues with the test suites that I am working on solutions for, one problem that occasionally bites me is that it seems that my computer is too slow and will sometimes trigger a timeout failure:

    :info! reports type, constructors and unfiltered instances for multiple types:                    FAIL
      Exception: Timed out waiting to receive a message from the server.
      Last message received:
      {
          "jsonrpc": "2.0",
          "method": "textDocument/publishDiagnostics",
          "params": {
              "diagnostics": [],
              "uri": "file:///tmp/extra-dir-1657892564956/TInfo.hs"
          }
      }
      Use -p '/:info! reports type, constructors and unfiltered instances for multiple types/' to rerun this test only.

Note that this test passed on a previous run, probably because the system load was lower.

(Full test log)

Describe the solution you'd like

I was told that Meson has a solution for problems like this; namely they added a timeout multiplier for tests that you can set on the command line:

  -t TIMEOUT_MULTIPLIER, --timeout-multiplier TIMEOUT_MULTIPLIER
                                        Define a multiplier for test timeout,
                                        for example when running tests in
                                        particular conditions they might take
                                        more time to execute. (<= 0 to disable
                                        timeout)

(Source)

I think that something like this should be implemented for HLS so that those with slower computers can run the test suites without running into these false positives. It could be implemented so that the multiplier is passed via an environment variable or as a flag passed via Cabal's --test-option flag.

@fendor
Copy link
Collaborator

fendor commented Mar 6, 2024

The test is flaky and would timeout even if you waited for 10 hours. We need to fix that!
Tracking ticket #3736.

Closing, as the proposal will not solve the issue.

@fendor fendor closed this as completed Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants