Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MSA and HRR levels to SafeGraph data #114

Closed
capnrefsmmat opened this issue Jun 29, 2020 · 4 comments · Fixed by #573
Closed

Add MSA and HRR levels to SafeGraph data #114

capnrefsmmat opened this issue Jun 29, 2020 · 4 comments · Fixed by #573
Assignees

Comments

@capnrefsmmat
Copy link
Contributor

Though we do not ourselves use MSA-level data for forecasting, it may be of general interest to journalists and researchers interested in studying outbreaks in specific cities. Currently the SafeGraph pipeline does not do any aggregation to MSAs, only to states.

We should also add HRRs for completeness, since anyone using the HRR-level data from other sources may want to compare against HRR-level data from SafeGraph.

@jsharpna, maybe this is something that would be easy for you to do as you refactor the Python pipelines to use common geographic aggregation functions?

@jsharpna jsharpna self-assigned this Jun 29, 2020
@capnrefsmmat
Copy link
Contributor Author

@jsharpna FYI, adding MSA to SafeGraph is now more important -- the upcoming release candidate is going to feature all graphs simultaneously, so when you switch to the MSA view, you constantly see "No data available" for the SafeGraph data.

I assume this is part of the geo refactoring and will get fixed as it happens?

@krivard
Copy link
Contributor

krivard commented Aug 26, 2020

Upping the priority of Safegraph refactor to use the geo utility and tracking in #248

@capnrefsmmat
Copy link
Contributor Author

Ping. It appears safegraph was refactored to use the geo utility, but I still don't see MSA and HRR in the API.

@chinandrew How hard is this to add to the SafeGraph codebase now that it uses the new geomapper?

@chinandrew
Copy link
Contributor

Ping. It appears safegraph was refactored to use the geo utility, but I still don't see MSA and HRR in the API.

@chinandrew How hard is this to add to the SafeGraph codebase now that it uses the new geomapper?

It doesn't look too complex. This is the current code, so I think it'd be adding the two additional conditionals for msa and hrr. @krivard what does the process look to approve + publish these signals once the indicator code is able to create additional resolutions?

    # Prepare geo resolution
    if geo_resolution == 'county':
        geo_transformed_df = df.copy()
        geo_transformed_df['geo_id'] = df['county_fips']
    elif geo_resolution == 'state':
        gmpr = GeoMapper()
        geo_transformed_df = gmpr.add_geocode(df,
                              from_col='county_fips',
                              from_code='fips',
                              new_code='state_id',
                              new_col='geo_id',
                              dropna=False)
    else:
        raise ValueError(
            f'`geo_resolution` must be one of {GEO_RESOLUTIONS}.')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants