Verify plausability of special phrase terms

[lonvia]

The Special Phrase pages in the wiki get the occasional entry for tag combinations that don't really exist. Two things: a) make sure the key list is restricted to tags actually imported by osm2pgsql. b) Taginfo could be used to verify that a key/value combination actually exists.

[AntoJvlt]

It this issue still revelant @lonvia?

If I get your point well, during the import of the special phrases we should check if the key is present in the generic_keys array of osm2pgsql/style.lua.

Then if it is present we could call the TagInfo API to check if the given value match with the tag.

For example if we get aeroway=aerodrome, we would call :
https://taginfo.openstreetmap.org/api/4/tag/combinations?key=aeroway&value=aerodrome&format=json&rp=1&page=1

And then check that total field is not equal to 0.

Off course for every keys and key/value pairs we would cache the result. This cache could be in memory during the import process or by creating a file to cache results inside, so that if we do another special phrases import we can use and update this cache.

I don't know if the second option is overkill and if there is a risk that a given key (or a key/value) may not be supported anymore or removed.

Please let me know what you think and how you imagined that, thanks.

[lonvia]

Yes, that's still relevant although for the moment I would keep to a). The taginfo idea would be more relevant if we wanted to start supplying precomputed special-terms lists instead of downloading the ones from the wiki, i.e. in that case, the actual computation of the special phrases would happen rather rarely.

By osm2pgsql config, Nominatim's style config is meant, e.g this one. get_import_style_file() gets you the configuration file in use. There is a description of the file format in the documentation. Essentially you would be interested in the key/value combinations with a 'main' property. Those combinations are the only ones that can be searched with the special terms later.

[lonvia]

Some updates on the issue as the code has moved a bit: The special phrase importer is now part of the Python code. Implementing a) requires something like this:

1. After the index on classes and types has been created query the database and get a list of combinations of class and type and their number of occurrences. Restrict the list to combinations with more than 100 occurrences.
2. Remove all class/type combinations from the list of phrases, which have not been found in the database.

[kitsiosvas]

@lonvia Can I have a bit more information about the problem? As far as I understand, you propose to create a new index on a database table? If so, which table in the data base are you referring to (also information about the table would be helpful, like column names).

Also, are we talking here about new functionality in the tools/special_phrases/mp_importer.py file? Should this be a new function in it self?