Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor usafacts to use geo utils #316

Merged
merged 11 commits into from
Oct 30, 2020
Merged

Refactor usafacts to use geo utils #316

merged 11 commits into from
Oct 30, 2020

Conversation

sgsmob
Copy link
Contributor

@sgsmob sgsmob commented Oct 15, 2020

The USAFacts indicator no longer uses localized libraries that have been superseded by delphi_utils.GeoMapper.

Fixes #338, which itself addresses part of #306. Synced to and #309 and #314, so should be submitted after them.

Comment on lines 14 to 17
for fname in listdir("../receiving"):
if fname[0] == ".":
continue
remove(join("../receiving", fname))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be made into a one liner with

[remove(fname) for fname in glob.glob('receiving/*')]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops realized this is part of #314

Comment on lines 132 to 133
print(expected)
print(actual)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these debug statements?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, removed sorry for having the multi sync workspace here.

df = df.copy().sort_values(["fips", "timestamp"])
for col in COLS:
for col in cols:
# Get values from the aggregated county:
vals = df.loc[df["fips"] == pooled_fips, col].values / len(fips_list)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nitpick, but would be good to standardize single vs double quotes within the file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Replace modular arithmetic with string suffix checking.

Co-authored-by: chinandrew <chinandrew96@gmail.com>
@chinandrew
Copy link
Contributor

Looks good to me, just left a few comments. Also FYI aside from initially skimming the whole thing, I mainly just lookeat at usafacts/delphi_usafacts/geo.py for this PR, and not the changes introduced by #309 and #314

@krivard
Copy link
Contributor

krivard commented Oct 17, 2020

Blocked on #325.

@krivard krivard added the blocked This task is waiting for completion of another task label Oct 17, 2020
@dshemetov
Copy link
Contributor

dshemetov commented Oct 20, 2020

@sgsmob Just noticed: we can replace the static population file here by using

gmpr = GeoMapper()
pop_df = gmpr._load_crosswalk("fips", "pop").rename(columns={"pop": "population"})

@dshemetov
Copy link
Contributor

@sgsmob better yet, we can remove that pop_df argument entirely and use GeoMapper's add_population_column here.

@bweaver-work bweaver-work linked an issue Oct 20, 2020 that may be closed by this pull request
@dshemetov
Copy link
Contributor

Lgtm!

@krivard krivard merged commit 8a0a2c0 into cmu-delphi:main Oct 30, 2020
@krivard krivard mentioned this pull request Oct 30, 2020
3 tasks
@chinandrew
Copy link
Contributor

I just realized there theres still static files that are used in USAFacts, sorry for missing in review. Figure we'll fix it when we add hhs + nation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked This task is waiting for completion of another task
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor usafacts to use geo utils
4 participants