Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

~~Test~~ Demo using data from WW R package #55

Conversation

gvegayon
Copy link
Member

@gvegayon gvegayon commented Apr 2, 2024

This PR does the following:

  • It introduces the datasets submodule that includes test data generated by the WW team (see here)
  • It includes a test asserting that the wastewater data is properly loaded. It uses polars to load the data.
    - [ ] It uses HospitalizationsModel to fit the data and compares it with the estimates obtained from the R package.
  • It adds a demo using a polars dataset (with the WW data).
  • Adds an example creating a weekday effect.
  • Adds a padding argument to the hospital admissions and basic renewal model. The padding assumes the first padding observations are nan.

image

From the 🏰 Gargoyleosaurus sprint (April 12)

Part of the PR initially wanted to deal with adding a test comparing the results of the WW hospitalization model in R. Still, the plans changed as the number of codebase modifications grew substantially. Things that need to be addressed during the next sprint:

  • Check the initialization of the model: The first half of NUTS estimated hospitalizations doesn't cover the data properly:

image

cc @dylanhmorris, @SamuelBrand1

@gvegayon
Copy link
Member Author

gvegayon commented Apr 3, 2024

I'm holding making progress on this one until #60 is merged.

Copy link

codecov bot commented Apr 4, 2024

Codecov Report

Attention: Patch coverage is 96.77419% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 92.09%. Comparing base (fffbee0) to head (e72c000).

Files Patch % Lines
model/src/pyrenew/deterministic/deterministic.py 80.00% 1 Missing ⚠️
model/src/pyrenew/mcmcutils.py 94.73% 1 Missing ⚠️
model/src/pyrenew/model/hospitalizations.py 92.30% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #55      +/-   ##
==========================================
+ Coverage   90.57%   92.09%   +1.51%     
==========================================
  Files          28       33       +5     
  Lines         552      620      +68     
==========================================
+ Hits          500      571      +71     
+ Misses         52       49       -3     
Flag Coverage Δ
unittests 92.09% <96.77%> (+1.51%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gvegayon gvegayon changed the title Test comparing WW R package estimation with pyrenew ~~Test~~ Demo using WW R package estimation with pyrenew Apr 10, 2024
@gvegayon gvegayon changed the title ~~Test~~ Demo using WW R package estimation with pyrenew ~~Test~~ Demo using data from WW R package Apr 10, 2024
@gvegayon gvegayon marked this pull request as ready for review April 10, 2024 20:38
@gvegayon
Copy link
Member Author

@dylanhmorris and @AFg6K7h4fhy2, this PR is ready for review. The only problem is a test that checks the Model.plot_posterior function is able to reproduce an image. I had the good intention to check reproducibility (relevant to #81 (comment),) but the test fails. I'm not sure why. Any ideas?

@SamuelBrand1
Copy link
Collaborator

Some context: @gvegayon and I just re-ran with the

$$\log \mathcal{R}_t - \log \mathcal{R}_{t-1} \sim N(0., \sigma)$$

$\sigma$ value set at $\sigma = 0.1$ (large!) as opposed to $ \sigma = 0.025$, and that repairs a chunk of the posterior predictive fit; as expected by @dylanhmorris .

Obviously, that can't fix the whole problem because of how the infection -> hosp obs. delay works.

@gvegayon
Copy link
Member Author

OK, @dylanhmorris, this is ready for review.

model/src/pyrenew/model/hospitalizations.py Outdated Show resolved Hide resolved
model/src/pyrenew/model/rtinfectionsrenewal.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@dylanhmorris dylanhmorris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few key questions, mostly looks ready @gvegayon

gvegayon and others added 5 commits April 19, 2024 13:08
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
…al.py

Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
@gvegayon gvegayon merged commit c7dc7da into main Apr 23, 2024
8 checks passed
@gvegayon gvegayon deleted the 3-replicate-model-2-from-cdcgovwastewater-informed-covid-forecasting-testing branch April 23, 2024 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replicate Model 2 from cdcgov/wastewater-informed-covid-forecasting
4 participants