Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suppress a hypothesis HealthCheck on test_split_datasets that fails in nightly CI. #5790

Conversation

bdice
Copy link
Contributor

@bdice bdice commented Mar 1, 2024

Fixes a HealthCheck error in nightly CI on test_split_datasets:

_____________________________ test_split_datasets ______________________________
[gw0] linux -- Python 3.9.18 /opt/conda/envs/test/bin/python3.9

    @given(split_datasets(standard_datasets()))
>   @settings(suppress_health_check=[HealthCheck.too_slow])
E   hypothesis.errors.FailedHealthCheck: Examples routinely exceeded the max allowable size. (20 examples overran while generating 2 valid ones). Generating examples this large will usually lead to bad results. You could try setting max_size parameters on your collections and turning max_leaves down on recursive() calls.
E   See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.data_too_large to the suppress_health_check settings for this test.

test_strategies.py:101: FailedHealthCheck

Also, from a different run:

FAILED test_strategies.py::test_split_datasets - hypothesis.errors.FailedHealthCheck: Examples routinely exceeded the max allowable size. (20 examples overran while generating 3 valid ones). Generating examples this large will usually lead to bad results. You could try setting max_size parameters on your collections and turning max_leaves down on recursive() calls.

@bdice bdice requested a review from a team as a code owner March 1, 2024 17:33
@github-actions github-actions bot added the Cython / Python Cython or Python issue label Mar 1, 2024
@bdice bdice self-assigned this Mar 1, 2024
@bdice bdice added bug Something isn't working non-breaking Non-breaking change labels Mar 1, 2024
@dantegd
Copy link
Member

dantegd commented Mar 2, 2024

/merge

@rapids-bot rapids-bot bot merged commit b0ba340 into rapidsai:branch-24.04 Mar 2, 2024
62 checks passed
rapids-bot bot pushed a commit that referenced this pull request Mar 5, 2024
Follow-up to #5790. We suppressed `HealthCheck.too_slow` and `HealthCheck.data_too_large`, but another failure appeared with `HealthCheck.filter_too_much`. Instead, we should just skip all health checks for this test.

```
=========================== short test summary info ============================
FAILED test_strategies.py::test_split_datasets - hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data. Health check found 50 filtered examples but only 4 good ones. This will make your tests much slower, and also will probably distort the data generation quite a lot. You should adapt your strategy to filter less. This can also be caused by a low max_leaves parameter in recursive() calls
See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.filter_too_much to the suppress_health_check settings for this test.
= 1 failed, 13454 passed, 5886 skipped, 632 xfailed, 54 xpassed, 10155 warnings in 1659.57s (0:27:39) =
```

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Dante Gama Dessavre (https://github.com/dantegd)

URL: #5791
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Cython / Python Cython or Python issue non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants