Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for geojson downloads #43

Open
tomschenkjr opened this issue May 22, 2015 · 6 comments
Open

Add support for geojson downloads #43

tomschenkjr opened this issue May 22, 2015 · 6 comments

Comments

@tomschenkjr
Copy link
Contributor

Some Socrata datasets now support geojson. An example is Chicago's crime data set.

This feature would support the download of geojson files through read.socrata or similar function. This would convert to a map-able output with minimal manipulation.

Some potential issues would be rgdal dependencies, which can be complicated on Linux.

@dmpe
Copy link
Contributor

dmpe commented Jul 25, 2015

Currently we use mime:: because of type guessing. For super-optimal support of GeoJSON, we will need to think about it, because mime has no current support for it, even in sudo nano /etc/mime.types. I have already opened an issue:
yihui/mime#3


My approach to providing a geojson support:

Given that there is geojsonio::geojson_read() method the user would input
read.socrata(url= " some .geojson URL", options for that method) and we would return a data frame (or a list; see method's options) .

Something like this in getContentAsDataFrame :

"application/vnd.geo+json" =  # use geojson_read directly through its response link
    geojsonio::geojson_read(response$url, method = "local", parse = FALSE, what = "list")

See beginning here: dmpe@d167e8e

@tomschenkjr
Copy link
Contributor Author

There is some nuance with this feature:

  • Socrata's GeoJSON file obeys existing SoDA API rules: that is, returns 1,000 records unless set otherwise by the $limit parameters (max 50,000 records)
  • Therefore, we must recursively build retrieve the data using $limit and $offset parameters
  • RSocrata has been recursively building using data frames, however, mapping in R can take the form through data frames, lists, or sp data types.
  • Therefore, we have something to decide. Should we aim to return a data frame (as we have), a list, or sp?
    • data frame - would be doable using existing code structure, but data frame is not the most common way to make a map, though possible with the broom package
    • lists - it's an output format available in geojsonio, but doesn't seem to be easy to make maps from it.
    • sp - is the de facto format for mapping in R and an output option from geojsonio, however, it's unclear to me how you can recursively create an output for this.

I think sp makes most sense, but would need to research on how to iteratively build it.

Thoughts?

@dmpe
Copy link
Contributor

dmpe commented Feb 15, 2016

https://github.com/dmpe/RSocrata/blob/dev/R/returnData.R#L155

Thoughts?

Mostly none. Wont contribute any code (PR) anymore, not interested. Plus the code is out there, just copy & edit and paste.

Socrata's GeoJSON file obeys existing SoDA API rules: that is, returns 1,000 records unless set otherwise by the $limit parameters (max 50,000 records)
Therefore, we must recursively build retrieve the data using $limit and $offset parameters

(Stupid) Ok.

I think sp makes most sense, but would need to research on how to iteratively build it.

Makes sense.

@tomschenkjr
Copy link
Contributor Author

Per my comments, that code doesn't fully work. It'll return the 1,000 records, which can be slightly modified, but at most, will only return 50,000 records. So will need to recursively build it through multiple API calls using $limit and $offset parameters.

@dmpe
Copy link
Contributor

dmpe commented Feb 16, 2016

Well, I wrote very clearly that I wont contribute any code (PR) anymore,(...) and if you need & want, just copy it & edit (it yourself) and paste.

@barryrowlingson
Copy link

Reading a geoJSON resource via RSocrata cropped up on SO here:

https://stackoverflow.com/questions/44769441/converting-geojson-into-a-simple-feature-in-r/44769906#44769906

The new sf classes in R are a lot more like data frames, so it might be that the existing RSocrata methods for building data frames from batched responses will work out of the box.

thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants