Refactor doctor visits to use geo utils package #401
Conversation
"Get up-to-date utils"
Diff uploads usafacts
EMR: standard signal names
[GHT] try until pull the new data successfully
Propagate fix for #242 to main
* Add updated encrypted credentials * Add wip_signal to production params template * Add newline * Run on 12 cores * Add shell script to run the indicator
* Add updated encrypted credentials
* Add wip_signal to production params template
* Add newline
* Run on 12 cores
* Add shell script to run the indicator
* Update run script
- Use production ingestion dir
- Remove hard fail to work around an issue with `cp`.
- `cp` fails when files don't exist. I thought we could squash the
error by sending stderr to /dev/null, but `set -eo` still catches
and fails the script. Even worse, it happens silently. There should
be a better way to handle this so will add it to a future task.
Merge the SafeGraph deployment branch updates to main
Sanity check for linting
Propagate NYC and pylint hotfix for JHU
Add `percent_of_expected_deaths` signal and dry-run mode to NCHS mortality data pipeline
Extend geocode utility to actually support the state to state mappings
cdc_covidnet: change states to lowercase to align with geo_util default
Add specific exceptions to bare excepts
Gate pulling/syncing data from AWS in safegraph behind a flag in params
Update safegraph patterns docstrings to pass pydocstyle
CHC SFTP Downloads
Update nchs docstrings to pass pydocstyle
| .sum() | ||
| .reset_index() | ||
| ) | ||
| megacounty_df["ToExclude"] = False |
There was a problem hiding this comment.
I think the values where ToExclue=True were meant to be filtered out but weren't here.
There was a problem hiding this comment.
I'm not sure. In line 185, we subset by the ["ToExclude"] column, which should have removed all the cases we wanted to filter out. The line at 190 may just be redundant?
|
|
||
| def test_county_to_megacounty(self): | ||
|
|
||
| out, name = GM.county_to_megacounty(DATA, 10, 10) | ||
| out, name = GM.county_to_megacounty(DATA, 100000, 10) |
There was a problem hiding this comment.
Previous threshold led to no megacounties being made
|
@chinandrew Thanks for the breakdown. Are any state-level changes expected? Most of the values are the same, but I do see some differences (I noticed Alaska). Also, we have more HRRs than before. I don't think these changes are the result of this PR, but maybe the geo utils themselves? |
For HRR's, @dshemetov and I discussed how the mapping files were updated and some very minor changes were to be expected (e.g. fips 01011 got converted to 146,0.003024 and 7,0.996976 (HRR,weight) in the new mapping, while the original gets converted to 7,1.0000). Not sure about the state, it's possible that mapping file was updated? how large is the difference? |
Refactor doctor visits to use the new geoutils package. Closes #385
Summary of changes:
I branched this off dv-package but had to merge main in to get the most up to date geo utils, so the changes are...unreadable. Could remedy this by merging main into dv-package. Pretty sure there are only 2 files I touched, doctor_visits/delphi_doctor_visits/geo_maps.py and doctor_visits/tests/test_geomap.py, both of which I've left a comment on below so they can be found easily.