Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allow skipping validation when constructing workspaces #1710

Merged

Conversation

alexander-held
Copy link
Member

@alexander-held alexander-held commented Nov 22, 2021

Description

This adds a validate kwarg to the Workspace constructor and to Workspace.build, replicating the existing functionality in Model to skip validation against the JSON schema. For complex workspaces / models, where validation can take an appreciable amount of time (see #1699), this can be a useful option if users are sure that their inputs follow the required format.

There are two things in here that are optional, I tried to split them across commits to easily take out anything you may not want:

  • added typing for the validate kwargs,
  • moved the log info about reference schema to only trigger if validation is actually performed.

There was previously no documentation for validate in the Model constructor, so this has also been added.

A test for workspace construction without validation is also included, for both construction from a specification and from model + data. This uses the same simplemodels setup also used elsewhere in the workspace tests, so this could be refactored if #1709 is merged first (or otherwise could be refactored in #1709). Since #1709 also adds a test at the end, merging one probably results in a conflict with the other PR as well. -> #1709 was merged before, and the implementation here was changed to re-use the fixture added in that PR.

resolves #1699

Checklist Before Requesting Reviewer

  • Tests are passing
  • "WIP" removed from the title of the pull request
  • Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

  • Summarize commit messages into a comprehensive review of the PR
* Add validate kwarg to pyhf.workspace.Workspace to allow skipping validation
if the user desires.
   - The motivation for this comes from some workspace validations taking
     multiple seconds. Power users might want the ability to turn off
     validation in these cases to save time. This behavior is discouraged
     in general, but should be allowed.
* Add docstring for pyhf.workspace.Workspace and add validate kwarg to
pyhf.pdf.Model docstring.
* Add tests for workspace validate kwarg.

src/pyhf/workspace.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Nov 22, 2021

Codecov Report

Merging #1710 (e506094) into master (43c1567) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1710   +/-   ##
=======================================
  Coverage   98.12%   98.12%           
=======================================
  Files          64       64           
  Lines        4269     4270    +1     
  Branches      682      683    +1     
=======================================
+ Hits         4189     4190    +1     
  Misses         46       46           
  Partials       34       34           
Flag Coverage Δ
contrib 26.25% <28.57%> (-0.01%) ⬇️
doctest 60.58% <71.42%> (-0.02%) ⬇️
unittests 96.18% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/pyhf/pdf.py 97.80% <100.00%> (ø)
src/pyhf/workspace.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 43c1567...e506094. Read the comment docs.

@matthewfeickert
Copy link
Member

Relevant RTD builds:

Copy link
Member

@matthewfeickert matthewfeickert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @alexander-held. This all is looking good to me and your plan for docstrings sounds great. 👍

src/pyhf/pdf.py Outdated Show resolved Hide resolved
src/pyhf/workspace.py Outdated Show resolved Hide resolved
tests/test_workspace.py Outdated Show resolved Hide resolved
@matthewfeickert
Copy link
Member

Okay this one can get rebased and refactored now. 👍

@kratsg kratsg merged commit eb45d60 into scikit-hep:master Dec 9, 2021
@kratsg
Copy link
Contributor

kratsg commented Dec 9, 2021

Congrats on the 1k'th commit @alexander-held :)

@alexander-held alexander-held deleted the feat/skip-workspace-validation branch December 9, 2021 20:02
matthewfeickert added a commit that referenced this pull request Dec 23, 2021
* Add milestone to README for 1000 project commits.
   - 1000th commit came from Alexander Held in PR #1710
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation related feat/enhancement New feature or request tests pytest
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Option to skip workspace validation
3 participants