Rt with infection feedback #123

gvegayon · 2024-05-17T21:39:32Z

This PR:

Creates the InfectionsWithFeedback latent module.
Adds the new module pyrenew.datautils for common data management operations.
Adds the tutorial extending_pyrenew.qmd
Corrects the order of outputs from the sample_infections_with_feedback function.
Adds tests modules for InfectionsWithFeedback and pyrenew.datautils.

Extra:

Uses jax.numpy.atleast_1d() for DeterministicRV.sample() (so scalars are coerced into 1d arrays)
Uses jax.numpy.atleast_1d() for DistributionalRV.sample() (so scalars are coerced into 1d arrays)
Fixes the reminder of the ghost outbreak.

codecov · 2024-05-17T21:41:58Z

Codecov Report

Attention: Patch coverage is 94.20290% with 4 lines in your changes missing coverage. Please review.

Project coverage is 93.90%. Comparing base (f9c057a) to head (6f70084).

Files	Patch %	Lines
model/src/pyrenew/latent/infectionswithfeedback.py	89.74%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #123      +/-   ##
==========================================
+ Coverage   92.06%   93.90%   +1.84%     
==========================================
  Files          34       36       +2     
  Lines         643      706      +63     
==========================================
+ Hits          592      663      +71     
+ Misses         51       43       -8

Flag	Coverage Δ
unittests	`93.90% <94.20%> (+1.84%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…ted to fail)

model/src/test/test_infectionsrtfeedback.py

model/src/pyrenew/latent/infection_functions.py

…tions with zero-strength feedback, and returns the R(t) timeseries correctly

…compare in tests

…entation

…ot occur

… implementation. This will break other tests but is a worthwhile breaking change

gvegayon · 2024-05-23T17:28:13Z

@dylanhmorris, 219e710 and 6db0fb7 fixed the failing tests. I had to introduce three changes:

Make sure the output from latent.Infections.sample() is at least 1 dimension (link).
Pad the sampled I0 within model.RtInfectionsRenewalModel.sample() to match the generation interval (link).
Still add some padding for the force of infection, otherwise jax.lax.scan breaks as it needs the xs argument needs the arrays to have the same shape (link).

This introduced a couple of other changes:

It seems it solved a bug in the "ghost outbreak." In the case of the infections function, before your changes, the conv scanner was using jnp.dot() to multiply an array with a scalar, which resulted in inflated infections at the beginning of the sequence. Compare these two images, this is what we have in main:

versus the PR

Because of this change, I had to update the reference figure generated during pytest. I'm creating an issue to include a note on the developers' documentation discussing the pytest involving matplotlib.

I think we are good to go!

model/docs/extending_pyrenew.qmd

model/Makefile

model/pyproject.toml

model/src/pyrenew/latent/infectionswithfeedback.py

model/src/pyrenew/transform.py

damonbayer

~~One very minor change requested.~~ I made the change myself.

damonbayer · 2024-06-03T21:27:17Z

@dylanhmorris Needs your final review.

model/docs/extending_pyrenew.qmd

@damonbayer

@damonbayer made the requested changes

dylanhmorris · 2024-06-03T21:44:22Z

Update: needs conflicts resolved @damonbayer. Can you do and request re-review from me?

dylanhmorris

LGTM

commit 879c665 Author: George G. Vega Yon <g.vegayon@gmail.com> Date: Wed Jun 5 11:29:09 2024 -0600 Weekly Rt with autoregressive difference (#131) Co-authored-by: Damon Bayer <xum8@cdc.gov> Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com> commit f0a35ee Author: George G. Vega Yon <g.vegayon@gmail.com> Date: Wed Jun 5 10:32:48 2024 -0600 Renaming datautils to arrayutils (#154) commit 394a03d Author: Damon Bayer <xum8@cdc.gov> Date: Tue Jun 4 16:49:10 2024 -0500 Record deterministic `RandomVariables` by default (#148) commit a2c1307 Author: Damon Bayer <xum8@cdc.gov> Date: Tue Jun 4 16:02:07 2024 -0500 read date columns as dates (#153) commit f11ce38 Author: George G. Vega Yon <g.vegayon@gmail.com> Date: Mon Jun 3 16:20:08 2024 -0600 Rt with infection feedback (#123) --------- Co-authored-by: Dylan H. Morris <dzl1@cdc.gov> Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com> Co-authored-by: Damon Bayer <xum8@cdc.gov> commit f9c057a Author: George G. Vega Yon <g.vegayon@gmail.com> Date: Fri May 31 13:18:58 2024 -0600 Refactor transformation module to wrap `numpyro.distributions.transforms` (#140) Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>

gvegayon added 3 commits May 16, 2024 11:45

Working on Rt docs

27b65d2

Starting off module

1c306bd

Adding test and starting tutorial

f9951b6

gvegayon linked an issue May 17, 2024 that may be closed by this pull request

Infection feedback in R(t) #9

Closed

4 tasks

Removing extra file

26fb259

gvegayon added 4 commits May 20, 2024 16:39

Adding a test and extending documentation

8ed3eca

Adding test for datautils

d8b55f5

Adding more content to the tutorial

0beb654

Adding a test checking for the calculations of the double conv (expec…

0665b63

…ted to fail)

gvegayon commented May 21, 2024

View reviewed changes

model/src/test/test_infectionsrtfeedback.py Outdated Show resolved Hide resolved

Update model/src/test/test_infectionsrtfeedback.py

49d803d

gvegayon commented May 21, 2024

View reviewed changes

model/src/test/test_infectionsrtfeedback.py Outdated Show resolved Hide resolved

gvegayon marked this pull request as ready for review May 21, 2024 17:53

gvegayon mentioned this pull request May 21, 2024

State-level unadjusted R(t) #8

Closed

4 tasks

dylanhmorris reviewed May 21, 2024

View reviewed changes

model/src/pyrenew/latent/infection_functions.py Outdated Show resolved Hide resolved

dylanhmorris and others added 12 commits May 21, 2024 17:37

Clarify variable names in sample_infections_with_feedback()

1865216

Sign consistency and documentation

38e7028

Add basic test that infections with feedback reduces to regular infec…

880d4c6

…tions with zero-strength feedback, and returns the R(t) timeseries correctly

Add pytest-mpl to pyproject.toml given use of @pytest.mark.mpl_image_…

fafe2cf

…compare in tests

Run precommit

0119e92

Clarify pmf input format for sample_infections_with_feedback in docum…

7b96e4a

…entation

Update lockfile, style infections.py

da6ff39

clarify use of reversed PMFs, remove padding operations that should n…

a814f4a

…ot occur

Harmonize indexing and remove autopadding; fix manual renewal process…

52bfdb6

… implementation. This will break other tests but is a worthwhile breaking change

Update latent admissions test

b63acbe

Fixing broken tests

219e710

Merge branch 'main' into 9-infection-feedback-in-rt

98de6a5

Adding needed padding for convo

6db0fb7

damonbayer reviewed May 29, 2024

View reviewed changes

model/docs/extending_pyrenew.qmd Show resolved Hide resolved

AFg6K7h4fhy2 reviewed May 30, 2024

View reviewed changes

model/Makefile Show resolved Hide resolved

gvegayon added 3 commits May 30, 2024 14:26

Addressing reviewer's comments

5eb31cc

Minor change in docstring for infections test

7ea53df

Clarifying tutorial extending pyrenew and making sure models match

b603eb5

gvegayon mentioned this pull request May 30, 2024

Process returns tuple with array equal to duration (patch to PR 123) #143

Closed

AFg6K7h4fhy2 reviewed May 31, 2024

View reviewed changes

model/pyproject.toml Outdated Show resolved Hide resolved

AFg6K7h4fhy2 reviewed May 31, 2024

View reviewed changes

model/src/pyrenew/latent/infectionswithfeedback.py Outdated Show resolved Hide resolved

AFg6K7h4fhy2 reviewed May 31, 2024

View reviewed changes

model/src/pyrenew/transform.py Outdated Show resolved Hide resolved

Removing duplicated docstring

b0f72cf

gvegayon requested review from dylanhmorris and damonbayer May 31, 2024 17:53

gvegayon mentioned this pull request May 31, 2024

Rework Initial Infections and Support Exponential Growth Initializations #124

Merged

damonbayer previously requested changes Jun 3, 2024

View reviewed changes

add docs/extending_pyrenew.md to makefile

5973703

damonbayer self-requested a review June 3, 2024 21:26

dylanhmorris previously approved these changes Jun 3, 2024

View reviewed changes

model/docs/extending_pyrenew.qmd Outdated Show resolved Hide resolved

damonbayer added 2 commits June 3, 2024 17:02

grammar fix

3cded51

Merge branch 'main' into 9-infection-feedback-in-rt

6f70084

damonbayer requested a review from dylanhmorris June 3, 2024 22:13

dylanhmorris approved these changes Jun 3, 2024

View reviewed changes

dylanhmorris merged commit f11ce38 into main Jun 3, 2024
8 checks passed

dylanhmorris deleted the 9-infection-feedback-in-rt branch June 3, 2024 22:20

This was referenced Jun 3, 2024

Rename datautils module to arrayutils #149

Closed

Re-remove transform.py #157

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rt with infection feedback #123

Rt with infection feedback #123

gvegayon commented May 17, 2024 •

edited

Loading

codecov bot commented May 17, 2024 •

edited

Loading

gvegayon commented May 23, 2024 •

edited

Loading

damonbayer left a comment •

edited

Loading

damonbayer commented Jun 3, 2024

dylanhmorris commented Jun 3, 2024

dylanhmorris left a comment

Rt with infection feedback #123

Rt with infection feedback #123

Conversation

gvegayon commented May 17, 2024 • edited Loading

codecov bot commented May 17, 2024 • edited Loading

Codecov Report

gvegayon commented May 23, 2024 • edited Loading

damonbayer left a comment • edited Loading

Choose a reason for hiding this comment

damonbayer commented Jun 3, 2024

dylanhmorris commented Jun 3, 2024

dylanhmorris left a comment

Choose a reason for hiding this comment

gvegayon commented May 17, 2024 •

edited

Loading

codecov bot commented May 17, 2024 •

edited

Loading

gvegayon commented May 23, 2024 •

edited

Loading

damonbayer left a comment •

edited

Loading