Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hstore support in postgis provider #71

Closed
ARolek opened this issue Aug 16, 2016 · 3 comments
Closed

hstore support in postgis provider #71

ARolek opened this issue Aug 16, 2016 · 3 comments
Milestone

Comments

@ARolek
Copy link
Member

ARolek commented Aug 16, 2016

When working with an OSM export translated using osmosis some key / val pairs are being stored in an hstore datatype in PostGIS. These values are currently being encoded as a single string. We should decode the hstore and add the values alongside the feature's other tags.

@ARolek
Copy link
Member Author

ARolek commented Jan 13, 2017

@gdey do you recall why we have this block of code? While implementing hstore support this line is causing a bug where the tag is being encoded but is empty. A couple things to note:

  • Hstore's are key value pairs, and I'm simply adding all of the k/v pairs in the hstore to the feature tags, but the original hstore column that was referenced is empty. This is kind of obscure behavior. For example, OSM dumps have a column called "other_tags" which is an hstore. Once I encode all the k/v pairs in the "other_tags" column, the aforementioned block still encodes an empty "other_tags" tag in the feature.
  • Since we support a "fields" array in our config and filter columns by that array, if the user references an hstore should we encode everything in the hstore? Alternatively we could look for hstores, decode them and sniff for values in them per the provided "fields" array. I'm leaning towards the former.

@ARolek
Copy link
Member Author

ARolek commented Jan 13, 2017

While testing the hstore implementation it appears that all hstore values are of type string. This may cause problems when trying to use values for say "building_height" to extrude buildings. I think trying to parse every string into a float / int would be unnecessary overhead. An alternative solution is to allow the user to configure how to handle an hstore field. This would also allow for filtering out unnecessary values. A possible config:

[[providers]]
name = "bonn_osm"
type = "postgis"
host = ""
port = 5432
database = "" 
user = ""
password = ""

    [[providers.layers]]
    name = "building"
    tablename = "buildings_3857"
    geometry_fieldname = "wkb_geometry"
    id_fieldname = "ogc_fid"
    fields = [ "name", "amenity", "shop", "other_tags"]

		[[providers.layers.other_tags]]
		fields = [ "name", "amenity", "shop", "other_tags"] # field filter

			[[providers.layers.other_tags.building_height]]
			name = "foo" # for mapping the name to another value?
			data_type = "int32" # datatype to parse. this needs to be thought out more.
			default = 0 # default value if the parsing fails. is this necessary?

@ARolek
Copy link
Member Author

ARolek commented Jan 16, 2017

Digging into this a bit more a better solution might be to suggest the user better prepare their import if they need a specific attribute from the dump. This is discussed on StackOverflow here.

The advantage to this approach is we don't need to add additional parsing to the hstore data and therefore don't need to expand our config file options as previously defined. Hstores are of type string:string and would stay this way.

@ARolek ARolek modified the milestone: v0.4.0 Apr 25, 2017
@ARolek ARolek closed this as completed Apr 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant