Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Truncate floating point docstring examples to 8 decimal places #1726

Merged

Conversation

matthewfeickert
Copy link
Member

@matthewfeickert matthewfeickert commented Dec 7, 2021

Description

Resolves #1629

To harden the docs and doctests against floating point noise due to changes in backend libraries or operating systems, truncate all floating point examples in the docstring examples to 8 decimal places using the doctest ELLIPSIS directive.

This should be relatively uncontroversial. The only place where this might not be desirable (in terms of communication with the user) is for the CLI API in src/pyhf/cli/infer.py.

Checklist Before Requesting Reviewer

  • Tests are passing
  • "WIP" removed from the title of the pull request
  • Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

  • Summarize commit messages into a comprehensive review of the PR
* Truncate all floating point values in docstring examples to 8 decimal
places to harden the docstrings against small changes in backends, operating
systems, and code that could cause fluctuations beyond a reasonable level
of interest for final results. The choice of 8 decimal places is to match
the default number of decimal places for reprs for most array libraries.
   - Use doctest's ELLIPSIS option to truncate floating point numbers
   - c.f. https://docs.python.org/3/library/doctest.html#doctest.ELLIPSIS

@matthewfeickert matthewfeickert added docs Documentation related tests pytest labels Dec 7, 2021
@matthewfeickert matthewfeickert self-assigned this Dec 7, 2021
@matthewfeickert matthewfeickert changed the title Docs/update floating point precision in all examples docs: Truncate floating point docstring examples to 8 decimal places Dec 7, 2021
@@ -69,14 +69,14 @@ def fit(
{
"mle_parameters": {
"mu": [
0.00017298628839781602
0.000172986...
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These truncations might be debatable of use. I'm open to changes and suggestions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These aren't actually being tested in the doctest, so we can ignore them and revert the changes

$ pytest -r sx src/
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.9.6, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
Matplotlib: 3.5.0
Freetype: 2.6.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/feickert/Code/GitHub/pyhf, configfile: pyproject.toml
plugins: mock-3.6.1, console-scripts-1.2.1, mpl-0.13, requests-mock-1.9.3, benchmark-3.4.1, cov-3.0.0, anyio-3.3.3
collected 123 items                                                                                                                                                                                       

src/pyhf/compat.py ..                                                                                                                                                                               [  1%]
src/pyhf/events.py ..                                                                                                                                                                               [  3%]
src/pyhf/patchset.py .                                                                                                                                                                              [  4%]
src/pyhf/pdf.py .......                                                                                                                                                                             [  9%]
src/pyhf/probability.py .......                                                                                                                                                                     [ 15%]
src/pyhf/simplemodels.py ..                                                                                                                                                                         [ 17%]
src/pyhf/utils.py ..                                                                                                                                                                                [ 18%]
src/pyhf/contrib/utils.py .                                                                                                                                                                         [ 19%]
src/pyhf/contrib/viz/brazil.py ...                                                                                                                                                                  [ 21%]
src/pyhf/infer/__init__.py .                                                                                                                                                                        [ 22%]
src/pyhf/infer/calculators.py ..............                                                                                                                                                        [ 34%]
src/pyhf/infer/intervals.py .                                                                                                                                                                       [ 34%]
src/pyhf/infer/mle.py ...                                                                                                                                                                           [ 37%]
src/pyhf/infer/test_statistics.py .....                                                                                                                                                             [ 41%]
src/pyhf/infer/utils.py ..                                                                                                                                                                          [ 43%]
src/pyhf/tensor/jax_backend.py ................                                                                                                                                                     [ 56%]
src/pyhf/tensor/manager.py ..                                                                                                                                                                       [ 57%]
src/pyhf/tensor/numpy_backend.py ................                                                                                                                                                   [ 70%]
src/pyhf/tensor/pytorch_backend.py .................                                                                                                                                                [ 84%]
src/pyhf/tensor/tensorflow_backend.py ...................                                                                                                                                           [100%]

============================================================================================ warnings summary =============================================================================================

@codecov
Copy link

codecov bot commented Dec 7, 2021

Codecov Report

Merging #1726 (6df812d) into master (1bc425f) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1726   +/-   ##
=======================================
  Coverage   98.12%   98.12%           
=======================================
  Files          64       64           
  Lines        4270     4270           
  Branches      683      683           
=======================================
  Hits         4190     4190           
  Misses         46       46           
  Partials       34       34           
Flag Coverage Δ
contrib 26.25% <ø> (ø)
doctest 60.58% <ø> (ø)
unittests 96.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/pyhf/infer/calculators.py 100.00% <ø> (ø)
src/pyhf/probability.py 100.00% <ø> (ø)
src/pyhf/tensor/numpy_backend.py 98.51% <ø> (ø)
src/pyhf/tensor/tensorflow_backend.py 97.29% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1bc425f...6df812d. Read the comment docs.

@matthewfeickert matthewfeickert force-pushed the docs/update-floating-point-precision-in-all-examples branch from f819a23 to 6df812d Compare December 10, 2021 19:29
@matthewfeickert matthewfeickert merged commit ce70574 into master Dec 10, 2021
@matthewfeickert matthewfeickert deleted the docs/update-floating-point-precision-in-all-examples branch December 10, 2021 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation related tests pytest
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use trailing ... to avoid having to deal with floating point differences in doctest
2 participants