map GeoNames feature codes to the Pelias categories taxonomy (#20) #22

sevko · 2015-05-07T16:55:11Z

Map the extensive geonames feature codes onto the Pelias taxonomy.

add a category mappings file, metadata/category_mapping.json (mostly uses the same keys as openstreetmap, but there are some new ones like admin, industry, academic, natural:water)
set Document categories in task/import.js

fixes #20

taxonomy.yaml -Add a first iteration of the geonames-to-pelias taxonomy mapping. -Currently a YAML file for human-writable convenience, will be converted to JSON later on. -Needs another look-over to revise some mappings/fill in missing ones.

taxonomy.yaml -Perform a second pass over the GeoNames taxonomy, tweaking some mappings added in 4f47082 and filling in empty ones. Roughly done, now.

taxonomy.yaml -Remove any GeoNames categories without corresponding categories in the Pelias taxonomy.

taxonomy.yaml -> metadata/taxonomy_mapping.json -YAML was initially used because it's much easier to handwrite than JSON. Now that the taxonomy mapping is complete, convert it to JSON.

"taxonomy" is less clear than "category".

task/import.js -Use the `category_mapping.json` file added in 4663087 to map geonames records' feature-codes to Pelias categories, and set them on `Document` objects.

test_cases/categories.json -Add a bunch of test-cases for the addition of categories to GeoNames data (see [0]). -They aren't actually usable until the API can return categories, so must wait until [1] gets merged. 0: pelias/geonames#22 1: pelias/api#122

missinglink · 2015-05-11T14:22:01Z

what do you see as the semantic difference between academic and education?
in the osm mapper natural:water maps to recreation https://github.com/pelias/openstreetmap/blob/master/config/category_map.js#L270

let's discuss further on a unified strategy.

sevko · 2015-05-11T14:29:30Z

Looks like I missed education when looking that file over, my fault. academic sounds like a better fit since it's more general (ie, a lab might be classified as academic but not necessarily educational), but seeing as it's already used in OSM I'm fine with leaving it as is, and changing all these occurrences of academic to education.

I departed from the OSM schema in some places, introducing categories like natural and natural:water, because what we currently have is a bit restrictive. It makes more sense to categorize a marsh, for instance, as simply a natural, watery area rather than a recreational one. I also added tags for industry, industry:agriculture, etc.

missinglink · 2015-05-11T14:55:54Z

The idea is to provide a unified taxonomy, so things like https://github.com/pelias/openstreetmap/blob/master/config/category_map.js#L264 need to both map to the same pelias category.

In order to be consistent we need to either change the pelias/openstreetmap categories if they are wrong or change these ones if they are wrong, but we cannot have the same concept mapping to different categories.

sevko · 2015-05-11T15:05:31Z

Totally agree, we need to discuss the new categories before this gets merged.

sevko · 2015-05-11T15:29:57Z

As per these changes to the taxonomy, we've decided to add top-level keys industry and natural. We'll update the pelias/openstreetmap mapping accordingly. There are also several issues in this PR that I'll need to address:

there's a mix of agriculture and industry:agriculture categories; only use industry:agriculture
use the finance category where it makes sense
use education instead of academic

metadata/category_mapping.json -`agriculture` was mistakenly used as a top-level category in some places; as per #22 (comment), replace all such occurrences with `industry:agriculture`.

metadata/category_mapping.json -Some of the Geonames categories also fall under the category of `finance`.

metadata/category_mapping.json -As per #22 (comment), use `education` instead of `academic`.

metadata/category_mapping.json -Some category sets with the `industry:agriculture` category were missing the superset `industry` category. Add it.

missinglink · 2015-05-11T15:55:13Z

👍

metadata/category_mapping.json -Typo in a category... oops.

hkrishna · 2015-05-12T18:21:41Z

looks good 👍

dianashk · 2015-05-12T18:21:43Z

task/import.js

+        peliasCategories.forEach( function ( category ){
+          try {
+            record.addCategory( category );
+          } catch ( ex ) {}


should probably log this error

@dianashk

task/import.js -Log `addCategory()` errors. -Use `hasOwnProperty()` for greater readability. -cc @dianashk

dianashk · 2015-05-12T18:33:21Z

👍

sevko added 5 commits May 6, 2015 16:21

Add a prototype taxonomy mapping.

4f47082

taxonomy.yaml -Add a first iteration of the geonames-to-pelias taxonomy mapping. -Currently a YAML file for human-writable convenience, will be converted to JSON later on. -Needs another look-over to revise some mappings/fill in missing ones.

Tweak the taxonomy mapping.

4328ea3

taxonomy.yaml -Perform a second pass over the GeoNames taxonomy, tweaking some mappings added in 4f47082 and filling in empty ones. Roughly done, now.

Remove unmapped categories.

520b2cd

taxonomy.yaml -Remove any GeoNames categories without corresponding categories in the Pelias taxonomy.

Convert taxonomy file from YAML to JSON.

fff7cfd

taxonomy.yaml -> metadata/taxonomy_mapping.json -YAML was initially used because it's much easier to handwrite than JSON. Now that the taxonomy mapping is complete, convert it to JSON.

Rename taxonomy_mapping.json to category_mapping.json

4663087

"taxonomy" is less clear than "category".

missinglink added the in progress label May 7, 2015

Set model categories in the importer.

aa03387

task/import.js -Use the `category_mapping.json` file added in 4663087 to map geonames records' feature-codes to Pelias categories, and set them on `Document` objects.

sevko force-pushed the feature-taxonomy#20 branch from 62121a8 to aa03387 Compare May 7, 2015 18:02

sevko mentioned this pull request May 8, 2015

Add test-cases for geonames categories. pelias/acceptance-tests#46

Merged

sevko added 4 commits May 11, 2015 11:35

Category map: agriculture -> industry:agriculture

0881b1e

metadata/category_mapping.json -`agriculture` was mistakenly used as a top-level category in some places; as per #22 (comment), replace all such occurrences with `industry:agriculture`.

Add the finance category to some category mappings.

1eb1d1f

metadata/category_mapping.json -Some of the Geonames categories also fall under the category of `finance`.

Category map: academic -> education

7219cce

metadata/category_mapping.json -As per #22 (comment), use `education` instead of `academic`.

Add industry to category sets with industry:agriculture

4b61ad0

metadata/category_mapping.json -Some category sets with the `industry:agriculture` category were missing the superset `industry` category. Add it.

sevko added 2 commits May 11, 2015 11:56

Add the nightlife category to CSNO (casinos).

45a5661

accomodation -> accommodation

362877a

metadata/category_mapping.json -Typo in a category... oops.

dianashk reviewed May 12, 2015
View reviewed changes

Address minor pull-request comments.

eb1f980

task/import.js -Log `addCategory()` errors. -Use `hasOwnProperty()` for greater readability. -cc @dianashk

sevko merged commit eb1f980 into master May 12, 2015

sevko removed the in progress label May 12, 2015

orangejulius deleted the feature-taxonomy#20 branch February 5, 2016 15:56

missinglink mentioned this pull request Feb 17, 2016

Create mapping from Geonames feature codes to layers #31

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

map GeoNames feature codes to the Pelias categories taxonomy (#20) #22

map GeoNames feature codes to the Pelias categories taxonomy (#20) #22

sevko commented May 7, 2015

missinglink commented May 11, 2015

sevko commented May 11, 2015

missinglink commented May 11, 2015

sevko commented May 11, 2015

sevko commented May 11, 2015

missinglink commented May 11, 2015

hkrishna commented May 12, 2015

dianashk May 12, 2015

dianashk commented May 12, 2015

map GeoNames feature codes to the Pelias categories taxonomy (#20) #22

map GeoNames feature codes to the Pelias categories taxonomy (#20) #22

Conversation

sevko commented May 7, 2015

missinglink commented May 11, 2015

sevko commented May 11, 2015

missinglink commented May 11, 2015

sevko commented May 11, 2015

sevko commented May 11, 2015

missinglink commented May 11, 2015

hkrishna commented May 12, 2015

dianashk May 12, 2015

Choose a reason for hiding this comment

dianashk commented May 12, 2015