Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

events for more mark_invalid() cases #3654

Merged
merged 5 commits into from
May 26, 2023
Merged

Conversation

carljm
Copy link
Contributor

@carljm carljm commented May 25, 2023

This PR adds a why argument to ConjectureData.mark_invalid() (and many.reject() which can call through to it), which can optionally take a string description of the reason for the data to be marked as invalid, and will record an event.

I didn't try to ensure that every single callsite now provides these arguments (there can always be follow-up PRs to add more where it seems useful), but I covered the ones where I felt able to provide something (hopefully) sensible.

The specific case I wanted to cover was MAX_DEPTH invalidation, since I have a use case where I'm getting >99% invalidation due to that, and prior to this PR it was invisible in the statistics, making me dive into internals to understand what was happening. With this PR I now see:

    - Events:
      * 99.29%, max depth exceeded

I think ideally this message would give more info on what the user can do to avoid this, but at the moment I don't know enough to say more than maybe "use less recursive strategies."

See #3643 for discussion.

Copy link
Member

@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @carljm, this looks great 🙏

@Zac-HD Zac-HD merged commit 2044e7b into HypothesisWorks:master May 26, 2023
@carljm carljm deleted the invalidwhy branch May 26, 2023 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants