NIP-3166: Country code tag based on ISO-3166 #763

steliosrammos · 2023-09-05T02:03:20Z

Draft NIP for introducing a new country-code tag that can be added to any event. The tag makes use of the ISO-3166 standard country codes and would allow easier filtering and indexing of events based on their geographic location.

It differs from the current g tag presented in NIP-52 in that it is much less granular than a geohash, which is not ideal for indexing.

The same difference is true for the proposed Geospatial Types #136.

Possibly a third parameter could be added (eg: ["g", "", "country"]), in conjunction to the existing g tag, as presented in #230 to support multiple g tag types. But I'm not sure that's preferable for indexing. (removed to focus on country code only, as per @mattn's feedback).

Open to feedback!

mattn · 2023-09-05T04:10:40Z

What do you expect from this country code? Language? Or current location? lat/lng can be specified?

steliosrammos · 2023-09-05T04:47:28Z

What do you expect from this country code? Language? Or current location? lat/lng can be specified?

Simple country location. Other NIPs can take care of more granular geo-location.

The direct use-case is for letting profiles show their country of origin, and allow them to be easily searched by it.

mattn · 2023-09-05T04:53:20Z

Then, it should be just to be "Country code" not "GEO Location" ?

steliosrammos · 2023-09-05T05:13:11Z

Then, it should be just to be "Country code" not "GEO Location" ?

Good feedback, updated the NIP and PR description.

AsaiToshiya · 2023-09-05T05:41:56Z

What about the idea to merge with #632?

steliosrammos · 2023-09-05T13:51:36Z

#632 is more focused on language than location. The two don't always map, and they are different concerns imo.

arthurfranca · 2023-09-05T15:04:20Z

Maybe this PR should be converted to an "Using Labels to Locate Events" NIP-32 section. Because if we start reserving letters for city, state and so on we may have no more letters left.

How about using NIP-32 like this:

  // ...
  "tags": [
    ["L", "country"]
    ["L", "state"]
    ["L", "city"]
    ["l", "US", "country"],
    ["l", "US-LA", "state"]
    ["l", "US-LA-New Orleans", "city"]
  ]

staab · 2023-09-05T15:06:34Z

How about using NIP-32 like this:

This was one of the use cases considered in the original NIP 32 conversation, so yeah I would go that direction.

steliosrammos · 2023-09-05T15:58:55Z

How about using NIP-32 like this:

This was one of the use cases considered in the original NIP 32 conversation, so yeah I would go that direction.

Just discovering NIP-32, seems to be a good fit at first sight.

My main concern is that it would come at the expense of better indexing, as the l tag index would include all sorts of namespaces, and get worse over time. As far as I can tell, relays are not sub-indexing on the "L" tag right?

That said, covering more location granularities like city, state, region etc. makes sense. Perhaps a similar scheme can be applied here with a "c" label and "C" namespace, dedicated to locations to keep good indexing.

The reason I'm suggesting that is because a location tag might be generic/popular enough to justify its own tag.

staab · 2023-09-05T16:05:12Z

L tags are single-letter, so based on the spec they would be indexed. In practice, they probably aren't, but if more people use them they will be.

steliosrammos · 2023-09-05T16:52:15Z

L tags are single-letter, so based on the spec they would be indexed. In practice, they probably aren't, but if more people use them they will be.

Although they would be indexed, wouldn't it be less efficient than having an index dedicated to location? I could be misunderstanding how relays index values.

staab · 2023-09-05T17:22:57Z

It depends, but it probably would have an impact. You could also do something like [["L", "country"], ["l", "country:GB"]] and only filter based on the qualified l tag

steliosrammos · 2023-09-05T22:07:16Z

What's the best way forward here? Open a new PR that extends NIP-32 with a new section detailing the location use case?

vitorpamplona · 2023-09-05T22:41:08Z

The best way forward is almost always implementing it inside your preferred use case to see if it is actually better.

I am not sure if any of you coded this, but I learned a lot when implementing the geohash tag. And I am not saying that that is better than this. It's just always good to test things out.

Semisol · 2023-09-05T23:10:12Z

I think we should have a use case for this and/or an implementation before making this an official NIP. Otherwise, good idea.

Semisol · 2023-09-05T23:10:37Z

It depends, but it probably would have an impact. You could also do something like [["L", "country"], ["l", "country:GB"]] and only filter based on the qualified l tag

Anything a-zA-Z must be indexed

steliosrammos · 2023-09-06T04:22:59Z

For sure, not rushing for this to be merged, just getting feedback on the idea. The use case is displaying country/region on profiles such that they can be browsed by country/region. Client implementation will follow, still a few weeks away.

steliosrammos · 2023-09-06T04:28:11Z

Anything a-zA-Z must be indexed

At this point I'm starting to think that the indexing strategy needs to be discussed separately. But my take-away thus far is:

Should a location tag be indexable? -> yes
Is the current NIP-32 l tag good for this use case? -> pretty much, yes
Will the l tag indexes start to grow too large because they are generic, impacting the performance of query filtering on location? -> probably yes, but too soon to worry about (?)

dskvr · 2023-09-06T10:41:36Z

Then, it should be just to be "Country code" not "GEO Location" ?

@mattn Why limit geocoordinates to just the country code? Why not allow developers to discern the precision of a geotag as opposed to creating a limitation from the start?

lat/lng can be specified?

geohash is much more reliable than lat/lon since lat/lon has varying precision.

From #230.

    ["g","ww8p1r4t8","geohash"],
    ["g","Amsterdam","city"],
    ["g","NL","countryCode"],
    ["g","EU","continent"],
    ["g","Earth","planet"],

As implemented above, index-case has no conflicts and the read-case doesn't require the client to discern what the datapoint represents, since it is expressed in tag[2]

fiatjaf · 2023-09-06T12:55:28Z

I agree with the sandwich.

arthurfranca · 2023-09-06T14:01:51Z

@dskvr Good idea. But imo the tag[1] should be enough to filter out (relay side) most of the things you don't want. The values are different enough from each other that it probably works withough tag[2] (except for continent code clashing with country code or state) . Also geohashes may have many entries to allow for broader filtering maybe down to 4 digits (city-level) up to 9 (building level) depending on what your app needs.

So I would turn your example into this:

  ["g","ww8p1r"]
  ["g","ww8p1"],
  ["g","ww8p"], // city precision
  ["g","ww8"], // small state precision
  ["g","ww"], // big state precision
  ["g","US-LA-New Orleans"], // city; or LA-New Orleans
  ["g","US-LA"], // state; or just LA because country used the 3-letter code
  ["g","US"], // country. or USA
  ["g","continent>NA"], // unusual
  ["g","Earth"] // unusual

edit: to cover city right (by using smaller pieces) when searching it probably needs higher precision than 4 digits

arthurfranca · 2023-09-06T14:18:08Z

@fernandolguevara why do you disagree? xD

If you search for Republic of Namibia country (NA), relay would send you all North America (NA continent) entries and probably some other country entries that has NA state =P.

fiatjaf · 2023-09-06T23:06:26Z

I just realized https://github.com/nostr-protocol/nips/blob/master/32.md could already be used for this.

fernandolguevara · 2023-09-06T23:33:02Z

arthurfranca · 2023-09-07T02:24:18Z

@fernandolguevara oh ok but right now relays can't use the tag array's third item (second value) on searches. The third item would be just something you could use to filter client-side, after receiving relay response.

dskvr · 2023-09-07T12:00:50Z

@fiatjaf That is true, may not need a new tag, but it would still be beneficial for geodata to be consistent.

@arthurfranca The third key doesn't need to be filtered against. If you are filtering against AN (Antarctica) you don't need to specify you are filtering a continent. ~~There are no clashes between Continent Codes and Country Codes.~~ Was wrong, you mentioned Republic of Nambia and North America collision. To resolve this the Alpha‑3 code of ISO-3166 could be used instead of the Alpha-2 code, which could resolve stateRegion conflicts as well.

arthurfranca · 2023-09-07T15:43:27Z

@dskvr Ok, so if keeping the third item to account client-side for an unexpected clash you would want something like this:

  ["g","New Orleans", "city"],
  ["g","LA", "stateCode"],
  ["g","USA", "countryCode"],
  ["g","North America", "continent"],
  ["g","Earth", "planet"]

Turned continent code into the full name to not clash with any state code.
But if searching by city, instead of filtering like { #g: ["New Orleans"] } it would have to be { #g: ["USA", "LA", "New Orleans"] } because there exists equal city names in different states of the same or not country. And there are equal state codes at different countries.

BUT from NIP-01 At least one of the arrays' values must match the relevant field in an event for the condition to be considered a match. so this filter { #g: ["USA", "LA", "New Orleans"] } is an OR clause which wouldn't work.

So the correct would probably be something like this:

  ["g","US-LA-New Orleans", "city"],
  ["g","US-LA", "stateCode"],
  ["g","US", "countryCode"],
  ["g","North America", "continent"],
  ["g","Earth", "planet"]

Or using three letter code for countries and codes for continents (though I prefer the previous one):

  ["g","USA-LA-New Orleans", "city"],
  ["g","USA-LA", "stateCode"],
  ["g","USA", "countryCode"],
  ["g","NA", "continentCode"],
  ["g","Earth", "planet"]

It ended up not being that simple, so @steliosrammos now indeed a NIP for g tag may be useful for interoperability.

Now the city filter works as expected: { #g: ["US-LA-New Orleans"] }

vitorpamplona · 2023-09-07T17:51:24Z

You might want to consider using colon (i.e. USA:LA:New Orleans) since there are city names with a dash like Port-au-Prince, Haiti.

dskvr · 2023-09-07T18:40:17Z

@arthurfranca I definitely overlooked the or limitation of filters.

US-LA-New Orleans diverges ISO-3166-2 subdivisions.

If using subdivisions, might as well represent the key correctly.

["g","US-LA", "ISO-1366-2"],
["g","US", "ISO-1366-1:Alpha-2"],
["g","USA", "ISO-1366-1:Alpha-3"],
["g","840", "ISO-1366-1:Numeric"],

But there is still the issue with cities, which breaks out of ISO-1366, so some alternatives...

Used by shipping companies: https://unece.org/trade/uncefact/unlocode
IATA: https://en.wikipedia.org/wiki/IATA_airport_code
GeoNames Project: http://www.geonames.org/

You might want to consider using colon (i.e. USA:LA:New Orleans) since there are city names with a dash like Port-au-Prince, Haiti.

This would be better USA-LA:New Orleans because USA-LA is a ISO-3166-2 subdivision whereas USA:LA:New Orleans would kind of require a brand new specification.

arthurfranca · 2023-09-07T20:06:06Z

@vitorpamplona Yeah HT-OU-Port-au-Prince would be a problem when splitting =o.

@dskvr Regarding Cities:
unlocode: According to a comment on stackoverflow, the DB is incomplete
iata: depends on the city having airport, so incomplete too
geonames: good one with integer codes, free and widely used. (negative is people can input fake data and it depends on the DB existence)

Geonames though can also be used for country and state
HT (3723988) - https://www.geonames.org/3723988/haiti.html
OU (3719432) - https://www.geonames.org/3719432/departement-de-l-ouest.html
Port-au-Prince (3718426) - https://www.geonames.org/3718426/port-au-prince.html

If we use geonames just for city and other smaller subdivisions it becomes this:

["g","4335045", "GeoNames"], // city
["g","US-LA", "ISO-1366-2"],
["g","US", "ISO-1366-1:Alpha-2"],
["g","North America", "continent"],
["g","Earth", "planet"]

So, USA-LA:New Orleans or 4335045 or both?

s3x-jay · 2023-09-21T18:10:38Z

I would encourage you to use NIP-32. While you may want ISO-3166, others may want some other coding system. NIP-32 handles all of them. But to clarify some of the questions about it…

To put things simply (to make sure everyone is on the same page), only the first parameter after a single letter tag is indexed. AND there's (currently) no support for starts with queries in Nostr. That means both l and L are needed so you can query on all values matching a particular string OR all the uses of a particular namespace / coding system.

When we were coming up with NIP-32 there was some argument about how values should be structured. I was mostly thinking in terms of structured vocabularies (like ISO-3166) and wanted the namespace / vocabulary / coding system as a prefix on the value. Others wanted to query for values across multiple coding systems and wanted the value to have no prefix.

As a result, most of the examples in the NIP follow the no-prefix example like this…

["L", "GeoNames"],
["L", "ISO-3166-2"],
["l", "3173435", "GeoNames", "{\"confidence\":1,\"quality\":1}"],
["l", "IT-MI", "ISO-3166-2", "{\"confidence\":1,\"quality\":1}"],

But to use that data you have to query on both L and l and you may get results that aren't want you're looking for if another namespace is used in the event that has the value you're looking for.

If you notice the last paragraph of the NIP mentions…

Vocabularies MAY choose to fully qualify all labels within a namespace (for example, ["l", "com.example.vocabulary:my-label"]. This may be preferred when defining more formal vocabularies that should not be confused with another namespace when querying without an L tag.

Using that approach you can query just on l and get a clean set of results returned to you. NIP-32 does not state what delimiter should be used between the namespace and the value. Personally I always favored using '>' since it has few conflicts and implies a parent-child relationship. [You should query on L to see what others are using when working with a defined coding system like ISO-whatever…] If you take this approach your tags would look like this…

["L", "GeoNames"],
["L", "ISO-3166-2"],
["l", "GeoNames>3173435", "GeoNames", "{\"confidence\":1,\"quality\":1}"],
["l", "ISO-3166-2>IT-MI", "ISO-3166-2", "{\"confidence\":1,\"quality\":1}"],

Let me know if any of that raises questions…

dskvr · 2023-12-13T15:28:14Z

@s3x-jay

Let me know if any of that raises questions…

Not really any questions, but NIP-32 solution doesn't work well with filters. Since NIP-32 is another event kind and filters do not presently (and will probably never) support joins and the association is created separate from the event that needs to be labeled, it cannot be filtered on easily.

missed this bit:

"Publishers can self-label by adding l tags to their own non-1985 events..."

dskvr · 2023-12-15T17:21:33Z

I had to unblock myself on NIP-66 so I made this https://github.com/sandwichfarm/nostr-geotags, might be useful for someone here.

steliosrammos added 2 commits September 4, 2023 19:50

first draft for NIP-3166

52b8167

Merge branch 'nostr-protocol:master' into nip-3166-geo-location-tag

479f394

steliosrammos changed the title ~~first draft for NIP-3166~~ NIP-3166: Geo-location tag based on ISO-3166 Sep 5, 2023

steliosrammos added 2 commits September 5, 2023 00:09

Update to Country Code instead of Geo-location

c7e6bb3

replace geo-location for country code in description

48a3697

steliosrammos changed the title ~~NIP-3166: Geo-location tag based on ISO-3166~~ NIP-3166: Country code tag based on ISO-3166 Sep 5, 2023

MerryOscar mentioned this pull request Sep 7, 2023

Referencing Podcasts in Nostr Events #760

Open

dskvr mentioned this pull request Dec 27, 2023

NIP-115 Yet Another Geo Tag (ISO-3166-1/2/3) #952

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NIP-3166: Country code tag based on ISO-3166 #763

NIP-3166: Country code tag based on ISO-3166 #763

steliosrammos commented Sep 5, 2023 •

edited

Loading

mattn commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

mattn commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

AsaiToshiya commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

arthurfranca commented Sep 5, 2023

staab commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

staab commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

staab commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

vitorpamplona commented Sep 5, 2023

Semisol commented Sep 5, 2023

Semisol commented Sep 5, 2023

steliosrammos commented Sep 6, 2023

steliosrammos commented Sep 6, 2023

dskvr commented Sep 6, 2023 •

edited

Loading

fiatjaf commented Sep 6, 2023

arthurfranca commented Sep 6, 2023 •

edited

Loading

arthurfranca commented Sep 6, 2023

fiatjaf commented Sep 6, 2023

fernandolguevara commented Sep 6, 2023 •

edited

Loading

arthurfranca commented Sep 7, 2023

dskvr commented Sep 7, 2023 •

edited

Loading

arthurfranca commented Sep 7, 2023

vitorpamplona commented Sep 7, 2023

dskvr commented Sep 7, 2023 •

edited

Loading

arthurfranca commented Sep 7, 2023 •

edited

Loading

s3x-jay commented Sep 21, 2023

dskvr commented Dec 13, 2023 •

edited

Loading

dskvr commented Dec 15, 2023

NIP-3166: Country code tag based on ISO-3166 #763

Are you sure you want to change the base?

NIP-3166: Country code tag based on ISO-3166 #763

Conversation

steliosrammos commented Sep 5, 2023 • edited Loading

mattn commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

mattn commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

AsaiToshiya commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

arthurfranca commented Sep 5, 2023

staab commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

staab commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

staab commented Sep 5, 2023

steliosrammos commented Sep 5, 2023

vitorpamplona commented Sep 5, 2023

Semisol commented Sep 5, 2023

Semisol commented Sep 5, 2023

steliosrammos commented Sep 6, 2023

steliosrammos commented Sep 6, 2023

dskvr commented Sep 6, 2023 • edited Loading

fiatjaf commented Sep 6, 2023

arthurfranca commented Sep 6, 2023 • edited Loading

arthurfranca commented Sep 6, 2023

fiatjaf commented Sep 6, 2023

fernandolguevara commented Sep 6, 2023 • edited Loading

arthurfranca commented Sep 7, 2023

dskvr commented Sep 7, 2023 • edited Loading

arthurfranca commented Sep 7, 2023

vitorpamplona commented Sep 7, 2023

dskvr commented Sep 7, 2023 • edited Loading

arthurfranca commented Sep 7, 2023 • edited Loading

s3x-jay commented Sep 21, 2023

dskvr commented Dec 13, 2023 • edited Loading

dskvr commented Dec 15, 2023

steliosrammos commented Sep 5, 2023 •

edited

Loading

dskvr commented Sep 6, 2023 •

edited

Loading

arthurfranca commented Sep 6, 2023 •

edited

Loading

fernandolguevara commented Sep 6, 2023 •

edited

Loading

dskvr commented Sep 7, 2023 •

edited

Loading

dskvr commented Sep 7, 2023 •

edited

Loading

arthurfranca commented Sep 7, 2023 •

edited

Loading

dskvr commented Dec 13, 2023 •

edited

Loading