[WIP] Render maritime borders differently

[matthijsmelissen]

This renders maritime borders less prominently (blue instead of purple).

In order to implement this, we render admin boundaries from lines
instead of from polygons. This has the additional advantage that
it becomes possible in the future to render admin boundaries
thinner than currently.

Before:

After:

This fixes #621.

[matthijsmelissen]

As the admin_level is currently missing from some administrative boundaries (in particularly in Poland), this requires some work on the mapping side.

[kocio-pl]

Look again at the discussion about relations and potential data doubling on the Polish forum:

https://forum.openstreetmap.org/viewtopic.php?pid=673850#p673850

[imagico]

To make sure everyone is aware of this early on - this change (in a similar way as #2873+#2952) would move this style further away from the goal of mapper support (i.e. supporting mappers in consistent mapping and tagging but leaving the decisions on how to map things to the mappers) towards active mapper steering, i.e. trying to steer/nudge mappers to mapping things in a different way that is more convenient for how developers here decide they would like to render them.

That the way this is implemented (in contrast to the idea to render maritime boundaries differently) is cartographically a dead end and would lead to all kinds of problems is a different matter (which of course has been discussed in depth in the past).

[matthijsmelissen]

To make sure everyone is aware of this early on - this change (in a similar way as #2873+#2952) would move this style further away from the goal of mapper support (i.e. supporting mappers in consistent mapping and tagging but leaving the decisions on how to map things to the mappers) towards active mapper steering, i.e. trying to steer/nudge mappers to mapping things in a different way that is more convenient for how developers here decide they would like to render them.

Sorry, I'm afraid I don't follow you. What are you referring to?

[imagico]

Sorry, I'm afraid I don't follow you. What are you referring to?

Are your serious? Less than a day ago you wrote:

As the admin_level is currently missing from some administrative boundaries (in particularly in Poland), this requires some work on the mapping side.

Where you essentially admit that this change requires mappers to change the way they map boundaries (and pointing out Poland as a special case is misleading here - it is only from a narrow central European viewpoint).

See also #3074 (comment).

[matthijsmelissen]

Are your serious?

@imagico This is not the way we communicate here, please adapt your communication style.

Where you essentially admit that this change requires mappers to change the way they map boundaries (and pointing out Poland as a special case is misleading here - it is only from a narrow central European viewpoint).

Given that most of the world has admin_level tags on ways, I would consider the missing admin_level tags in Poland a data error. Having more complete data will help other data consumers as well. So I don't agree with your point of view.

[imagico]

Given that most of the world has admin_level tags on ways, I would consider the missing admin_level tags in Poland a data error. Having more complete data will help other data consumers as well. So I don't agree with your point of view.

Interesting. When you write more complete data - what data is currently missing in Poland? It cannot be the data what admin_level a certain boundary represents because that data already exists in form of the boundary relations.

What you seem to be saying is exactly what i said, namely that you want this style to change its goal from mapper support to mapper steering, in this case steering mappers to add redundant tags to ways - tags mappers evidently do not consistently add themselves because they are unnecessary since the information in them already exists in the boundary relations they have to map anyway. Yes, you can of course justify this by saying there are other map designers for whom it would be equally convenient to have admin_level tags on boundary ways but that is incredibly short sighted and a slap in the face of all the mappers who spend countless hours in maintaining the boundary relations and of all the data users who rely on correct boundary relation data.

In any case - you seem to have made up your mind and i don't expect you to change it. But don't tell me later i have not raised my objections early enough.

[kocio-pl]

@imagico This is not the way we communicate here, please adapt your communication style.

👍 - really, please, Christoph; I don't like to feel like personal confrontation instead of just discussing merits, even if details can be tedious and frustrating sometimes.

But don't tell me later i have not raised my objections early enough.

It's a valid objection for me and it's a general question which can be decided case by case - "when the data manipulations are acceptable for us?"

My rule of thumb is when we don't lie about the objects, it's on the allowed side, since this is not "tagging for rendering" rules violation. Data duplication is OK for me then, because we just tell something in two different ways. Notable examples:

1. We typically add lines for roads to make it easier for routing, while the ground truth should be tagging street areas and the routing lines can be just computed in theory. In practice we don't remove lines after the areas are drawn.
2. It's a popular practice that bus stops are tagged in the old (highway=bus_stop) and new schema (public transport) in parallel.

So in this case I accept the change.

[imagico]

when the data manipulations are acceptable for us?

No, this is a completely different subject here. Since every maintainer can merge changes without consent from the others there is no us here. If @matthijsmelissen decides to merge this change he is responsible for all the consequences. My objection concerns that this change constitutes a change in the goals of this style - If this change in goals is acceptable or not is not the issue here (because consensus is not required on that either) - and frankly it is ultimately not up to the maintainers here to decide as far as the map on osm.org is concerned.

As a general suggestion: In case of decisions about merging changes it would probably be a good idea to focus less on justifying individual changes and more on the technical debt you are amassing with those changes.

[kocio-pl]

If @matthijsmelissen decides to merge this change he is responsible for all the consequences.

Sure, but that was always true (for whoever merged a piece of code or prepared it) and it doesn't mean we don't try to reach the consensus.

As a general suggestion: In case of decisions about merging changes it would probably be a good idea to focus less on justifying individual changes and more on the technical debt you are amassing with those changes.

So what technical debt on the osm-carto side do you see with this code?

[imagico]

it doesn't mean we don't try to reach the consensus.

Trying to reach the consensus means trying to convince dissenting opinions to change their mind through arguments. When consensus was still required this happened frequently simply out of necessity. After we changed the procedure this became very rare - which is natural due to the lack of incentive. This is a problem i pointed out back when we made this decision.

So what technical debt on the osm-carto side do you see with this code?

I hope you have seen that my suggestion was to focus less on justifying individual changes.

If i look at the changes from the past year or so i can see quite a massive increase in technical dept - compared to the level of the years ago. You are probably unaware of most of it because there are no open issues documenting this - either because previously created issues have been closed without having been fixed or because no one bothered to actually create an issue. If i look at my mental list of the big challenges of this style design wise lets say right after the database reload most of them would be much harder to solve (no matter how) based on the current style compared to back then.

The particular problem of this change (like #2873+#2952) is the quite large amount of interest mappers and OSM data users are pressed to pay for the technical dept acquired by not actually solving the problem but - to stay within the image restructuring the depth at their expense.

[kocio-pl]

I hope you have seen that my suggestion was to focus less on justifying individual changes.

I can spent some time discussing also general issues:

When consensus was still required this happened frequently simply out of necessity.

But that meant that a lot of time was just wasted without any conclusion or action. Consensus was required, but solving problems was optional, so to speak. That is a broken model for me, since nobody took the responsibility for lack of action, so it lacked the balance.

The cases of the discussions you recalled are good illustration - they fell into oblivion and now I didn't even remember I was taking part in it. Now you have to make another effort to reread this, rethink if this is still valid (after 3 years people can change their mind or loose interest or the time) - and that happens rarely.

If i look at the changes from the past year or so i can see quite a massive increase in technical dept - compared to the level of the years ago. You are probably unaware of most of it because there are no open issues documenting this - either because previously created issues have been closed without having been fixed or because no one bothered to actually create an issue.

I'm aware of it, but it depends on how do you count it. I tend to look at PRs merged instead of open tickets only. Of course every merge might be both fixing some problem (decreasing debt) and creating another one (increasing debt), but this is when you have to check all the sides and decide. In my experience this is the hardest part, since this is not easy to count and someone might always say "I calculate it in a different way, here disadvantages are bigger than gains".

Adding almost any feature means more code to look after, but the ultimate solution for this problem is to just remove everything - "no features, no cry". This is a corner case of technical debt - it doesn't make sense alone, only in the context. And this context can be both having a rich map as a goal and resolving issues like "please render key=value".

The particular problem of this change (like #2873+#2952) is the quite large amount of interest mappers and OSM data users are pressed to pay for the technical dept acquired by not actually solving the problem but - to stay within the image restructuring the depth at their expense.

Yes, for me this is the practical problem if the community is willing to add tagging and this is my main concern with this code.

[imagico]

The cases of the discussions you recalled are good illustration - they fell into oblivion and now I didn't even remember I was taking part in it. Now you have to make another effort to reread this, rethink if this is still valid (after 3 years people can change their mind or loose interest or the time) - and that happens rarely.

I don't want to judge here - people have different ability to remember past lessons learned just as people have different abilities to comprehend complex problems in the first place. And it also varies with age obviously. But ultimately it boils down to: Those who cannot remember the past are condemned to repeat it. I would be careful dismissing the long term usefulness of arguments and discussion just because you did not get any persistent insights from them.

As far as the matter of technical debt is concerned - i don't think this discussion is going to lead anywhere. Technical debt results from people either being unaware of it or from people justifying it by more pressing needs. I don't want to argue if a certain technical debt can be justified. Even if it is repaying the debt in a timely manner would be a necessity for long term viability. My suggestion was to make yourself aware of the accumulation of technical debt happening through the changes you are responsible for.

Yes, for me this is the practical problem if the community is willing to add tagging and this is my main concern with this code.

Once again confirming my original remark about the change in goals of this style - obviously this can only work if the community is ultimately willing to yield to the pressure of the standard style - which is still a pretty safe bet of course.

[matthijsmelissen]

I don't want to judge here - people have different ability to remember past lessons learned just as people have different abilities to comprehend complex problems in the first place. And it also varies with age obviously. But ultimately it boils down to: Those who cannot remember the past are condemned to repeat it. I would be careful dismissing the long term usefulness of arguments and discussion just because you did not get any persistent insights from them.

But you are judging. For the last time @imagico: please avoid attacking people and focus on the merits of the ideas proposed.

[imagico]

But you are judging.

No, i am not, i am arguing for having a discourse based on arguments, reasoning and understanding as well as valuing experience and knowledge developed in the past. Making decisions based on the merits of the arguments and reasoning and careful consideration of all the information available.

Anyway - like i already said: I can only try to make you aware of the problems. The rest is up to you.

[kocio-pl]

I would be careful dismissing the long term usefulness of arguments and discussion just because you did not get any persistent insights from them.

I'm here not to get the insights, my aim is to have better map, even if it means doing a lot of rather dull stuff. And the most valuable lesson I've learned from the past (it's been ~20 years when I take part in the open projects) is that unlimited talking, no matter how wise, is bad when not leading to some results, because "long term" might exceed the lifetime of a project.

My suggestion was to make yourself aware of the accumulation of technical debt happening through the changes you are responsible for.

And I have answered that I was already aware of this.

As of me, I would like you to know that in my opinion you are responsible not only for the technical debt, analysis and picking errors, but for the whole project - including (but not limited to):

• how do you interact with other people,
• merging and releasing stuff (only 8 persons have such rights) or not doing this,
• not fixing the bugs you are aware of etc.

Whatever you choose to do or abstain from doing, you always take the responsibility for the consequences. At least this is what I think.

obviously this can only work if the community is ultimately willing to yield to the pressure of the standard style - which is still a pretty safe bet of course.

You tend to talk about the pressure only in the context of people wanting some features. This is not how I feel about it - the pressure is stressing some things. For me it's a natural thing that people want some things and that people disagree. This is not the pressure. The pressure for me is "we MUST have this feature" and "I have ALREADY told you to not do it".

Whether community will agree or not is an open question, but I'm pretty sure that Matthijs has enough skills to just discuss it without pressure.

As far as the matter of technical debt is concerned - i don't think this discussion is going to lead anywhere.

That's why I don't want to spend too much time discussing big things and I'm happy to get back to work.

[polarbearing]

In order to implement this, we render admin boundaries from lines
instead of from polygons.

I am strongly against this move as it completely breaks the data model of boundary relations.

most of the world has admin_level tags on ways

I just checked a level 4 boundary nearby. It is member of 10 (ten) boundary relation of entities on different levels. The 'name' on this way is more a note for the mapper to remember which line this is, not a name suitable to be rendered. If it was used for rendering, we would need to add all the 10 entities into the name.

[DaveF63]

My rule of thumb is when we don't lie about the objects, it's on the allowed side, since this is not "tagging for rendering" rules violation. Data duplication is OK for me then, because we just tell something in two different ways. Notable examples:

Duplicating purely for the convenience of the renderer/router is not acceptable. When boundaries change it's means double checking, double the work for those doing the mapping.

We typically add lines for roads to make it easier for routing, while the ground truth should be tagging street areas and the routing lines can be just computed in theory. In practice we don't remove lines after the areas are drawn.

This is not a valid example. I've previously claimed this is laziness by routers. If they can navigate a roundabout then they can navigate the circumference of an area.

It's a popular practice that bus stops are tagged in the old (highway=bus_stop) and new schema (public transport) in parallel.

Is it? Are you sure it's not because it's in transition from one scheme to another?

[yvecai]

Basically, I find it wrong to ask for retagging objects otherwise considered as valid to ease rendering.

[matthijsmelissen]

The map at maps.wikimedia.org seems to handle this issue nicely. Do we know how they accomplish their rendering? Where can we find their source code?

[matthijsmelissen]

Discussion on the tagging mailing list:

https://lists.openstreetmap.org/pipermail/tagging/2018-March/035347.html

[kocio-pl]

Where can we find their source code?

I guess this is it:

https://github.com/kartotherian/brighmed

[matthijsmelissen]

Good find, can @pnorman confirm?

[DaveF63]

@matthijsmelissen How does the tag maritime=yes on ways not provide a solution for your proposal?

[matthijsmelissen]

How does the tag maritime=yes on ways not provide a solution for your proposal?

@DaveF63 We don't have the admin_level tag on all ways (and in some cases not even the boundary=administrative tag). I'm not aware of a way to derive the admin_level tag of a way based on the relations that way is a member with the tools that we have.

An addition issue: I'm also not sure what the semantics would be of a maritime=yes tag on a way if the corresponding boundary=administrative tag is missing on that way.

[ElminsterAU]

@matthijsmelissen It would seem to me that osm2pgsql-dev/osm2pgsql#230 would seem to solve that problem.

[pnorman]

@pnorman Can we assume that the _rels table exist in locations where this style will be deployed?

We'd have to change our import instructions.

This requires --slim import, right?

Yes.

I am against requiring --slim import, and in general, relying on the slim tables is a bad idea. They're an internal osm2pgsql detail, and have changed between releases.

From a practical matter, I don't recommend querying the rels table because it's not practical do so.

I am 100% in favour of rendering admin borders from the way tags.

[matthijsmelissen]

I am 100% in favour of rendering admin borders from the way tags.

@pnorman Do you support duplication of admin data in the database (both on relations and on ways)? Or do you prefer tagging on ways over tagging on relations?

How mature is https://github.com/pnorman/osmborder?

[pnorman]

@pnorman Do you support duplication of admin data in the database (both on relations and on ways)? Or do you prefer tagging on ways over tagging on relations?

I don't consider it duplication

How mature is https://github.com/pnorman/osmborder?

It's stable, but I'm not sure if I'd call it mature.

The issues we're facing are common to many styles, and the practice of the admin_level of a way being the lowest of its parent relations is a long-standing one, dating back to before I joined OSM. We shouldn't be using exotic solutions like complicated joins with the slim tables, or preprocessing which isn't necessary.

[DaveF63]

I don't consider it duplication

If there are two identical tags, they've been duplicated.
It's the dictionary definition! https://en.wiktionary.org/wiki/duplicate
It's been agreed the tag on the way is redundant.

[matthijsmelissen]

I don't consider it duplication

@pnorman Why not? The main point made on the tagging list was that we can derive the tags for lines from the relation tags, given the right tools. Do you think this process is not possible? Because if so, I am missing something.

[pnorman]

@pnorman Why not? The main point made on the tagging list was that we can derive the tags for lines from the relation tags, given the right tools.

This ignores past practices, and the fact that people don't do what's proposed.

Do you think this process is not possible? Because if so, I am missing something.

It's technically possible, but won't happen. I'm aware of two pieces of software that do it - one is long-abandoned, and the other is what I wrote, and not really maintained. The fact that there are no other options written this long this long after #344 tells you how practical it is, and how likely it is to happen.

It's wroth reviewing #344, in particular where @cquest said

The way to solve this I've explored, is to postprocess the boundary polygons to recreate linestrings.
I've worked on this for another purpose (create a topologically correct simplified boundary set for french municipalities), but the results my be reused. It could be maintained up-to-date using a trigger (not tested).

This kind of pre/post processing is a nice thing to do on renderings that aim the best graphical output despite data inconsistency, but I doubt this should be the goal of the OSM default rendering that needs to be showing data as the are and not as they should be.

[matthijsmelissen]

Just to be sure, we currently render on the basis of relation tags. And so do most (but not all) other renderers I checked.

[imagico]

It's technically possible, but won't happen. I'm aware of two pieces of software that do it - one is long-abandoned, and the other is what I wrote, and not really maintained. The fact that there are no other options written this long this long after #344 tells you how practical it is, and how likely it is to happen.

Based on this argument we would among other things not have

• a hstore database
• regularly updated coastlines
• rendering of the Antarctic ice
• rendering of unpaved roads (well - we still don't but we have a working solution for them)

True innovation typically does not happen over night. There has to be sufficient need (not smothered by a 'good enough' superficial solution) and commitment for it in the project as a whole.

It's wroth reviewing #344, in particular where @cquest said

The way to solve this I've explored, is to postprocess the boundary polygons to recreate linestrings.
I've worked on this for another purpose (create a topologically correct simplified boundary set for french municipalities), but the results my be reused. It could be maintained up-to-date using a trigger (not tested).

Indeed. And @cquest is not the only person in the world who has faced and dealt with this problem in the past. What has not happened so far is that someone has developed a sufficiently generic, robust and scalable solution.

This kind of pre/post processing is a nice thing to do on renderings that aim the best graphical output despite data inconsistency, but I doubt this should be the goal of the OSM default rendering that needs to be showing data as the are and not as they should be.

Since this can be misunderstood easily - this is not a statement against using boundary relations for rendering, it is against having a process that attempts to fix data inconsistencies (like by combining ways and relations) to create a more complete rendering from incomplete and inconsistent data.

[matthijsmelissen]

@pnorman To be honest, I don't find your points very convincing - however I might not understand them sufficiently. The proposal here is to switch from polygon rendering to line rendering in order to accomplish a small rendering improvement. The tagging community asks us not to do this because the polygon tagging is conceptually better than the line tagging. A technical solution to apply tags from relations to linestrings seems a long way off, so this is not really an option for us, however not changing anything on our side is a possibility too.

[yvecai]

Having both ways and relations to map the same kind of objects, but stick to ways for the simpler cases is the way things are mapped in OSM for a very long time, not only for boundaries. I don't think OSM-carto should avoid dealing with this complexity, as the main display of our data. Agreed this is not only a cartocss issue. Yves

[pnorman]

The proposal here is to switch from polygon rendering to line rendering in order to accomplish a small rendering improvement. The tagging community asks us not to do this because the polygon tagging is conceptually better than the line tagging. A technical solution to apply tags from relations to linestrings seems a long way off, so this is not really an option for us, however not changing anything on our side is a possibility too.

People seem to think that using the relations tags only is practical, and I am saying it is not. It is possible but quite difficult, and we have a long-established way of tagging that deals with the problem: using the tags on ways. This is different from @imagico's examples of

a hstore database
regularly updated coastlines
rendering of the Antarctic ice
rendering of unpaved roads (well - we still don't but we have a working solution for them)

People have used hstore with their databases back to at least 2012 as far as I know, and probably earlier. There are multiple renderings that show paved and unpaved road information. Antarctic ice is specialized enough that I'm not aware of its history. Coastlines is in some ways comparable, except that we've never had any alternative way of modelling them that was easier to handle.

[yvecai]

I don't think it's a matter of rendering from ways OR relations, but from both ! from the wiki : "The boundary=administrative tag is used on ways. It may also (or instead) be used on a relation grouping several ways. " This is simply the way those are mapped. Yves

[matthijsmelissen]

People seem to think that using the relations tags only is practical, and I am saying it is not. It is possible but quite difficult,

@pnorman I'm still not sure if I follow are you. Are you saying that what we currently doing is difficult?

[imagico]

@pnorman - what i think you need to ask yourself is if you think tagging both ways and relations (which i agree with @DaveF63 to be duplication) is desirable because of convenience for some data users or if you really think there are advantages for mappers in that. And would your view change if there was an established tool for generating consistent boundary line topologies from the boundary relations?

But this discussion should really move to the tagging ML - it has nothing specifically to do with this style.

As for the examples i gave - a bit of history on those:

• The current coastline processing dates back to 2012 (see https://blog.jochentopf.com/2012-03-07-osm-coastlines.html and https://blog.jochentopf.com/2012-05-23-launching-openstreetmapdata-com.html) - before the coastline was manually processed using coastcheck - maybe twice a year so no useful mapper feedback in that and it could only produce land polygons which severely limits styling options. In Tiles@home there were other localized approaches to coastline rendering - see: https://wiki.openstreetmap.org/wiki/Tiles@home/Dev/Interim_Coastline_Support. There is no change in mapping/tagging conventions involved in that.
• The Antarctic ice problem was caused by a change in mapping conventions in 2013 (see https://wiki.openstreetmap.org/wiki/Antarctica/Tagging) that was introduced because it became obvious that the previous approach (one giant MP for the whole continent) was going to be unsustainable in the long term. It took 2.5 years for this change to be taken into account by this style (adding antarctic icesheet rendering based on preprocessed shapefiles #1540 (comment)). While it was difficult to get this through it was never a question though if this style might insist on maintaining the previous mapping conventions (i.e. only render explicitly mapped ice).
• The unpaved roads and how they can be rendered in this style is something that has been discussed at least since 2013 in Render paved/unpaved #110 - it has nothing to do with tagging but it is a great example how technical constraints make solving design problems more difficult and how thinking outside the box and not accepting defeat and concluding no one has come up with a good solution in years so we just have to go with a bad solution now is essential to overcoming these constraints.

[matthijsmelissen]

The map at maps.wikimedia.org seems to handle this issue nicely. Do we know how they accomplish their rendering? Where can we find their source code?

For the record: the wikimedia maps use https://github.com/pnorman/osmborder.

[jeisenbe]

Is it possible for this style to use https://github.com/pnorman/osmborder ?

[jeisenbe]

I'm assuming we cannot use https://github.com/pnorman/osmborder to obtain the correct data?

While checking the alt-color style, I notice that @imagico renders maritime admin boundaries from the ways, drawn on top of the other admin boundaries, which are still rendered from the relations:

  - id: admin-low-zoom
    geometry: linestring
    <<: *extents
    Datasource:
      <<: *osm2pgsql
      table: |-
        (SELECT
            way,
            admin_level,
            tags->'maritime' as maritime
          FROM planet_osm_roads
          WHERE boundary = 'administrative'
            AND admin_level IN ('0', '1', '2', '3', '4')
            AND (osm_id < 0 OR (osm_id > 0 AND tags @> 'maritime=>yes'))
          ORDER BY admin_level DESC,
            CASE
              WHEN tags @> 'maritime=>yes' THEN 1
              ELSE 0
            END ASC
        ) AS admin_low_zoom

However, I suppose this would still be considered a problem, because it would encourage mappers to add "boundary=administrative", "admin_level=*" and "maritime=yes" to the relevant OSM ways, and we should develop a novel technical solution that creates new line strings from the relations plus intersection with the coastline?

[imagico]

This way of maritime boundary rendering would be incompatible to the use of line simplification employed now (which is not really working otherwise, especially also for maritime boundaries - see #3666 (comment), but that's a different story). Apart from that tagging of level 2 maritime boundaries on ways is applied very consistently by mappers so this is not in principle a problem to use.

Different rendering of maritime boundaries is also possible from just the relations when using ocean polygons obviously.

[jeisenbe]

@matthijsmelissen do you want to close this PR? Since oceans are now rendered separately, there are many options for rendering maritime borders differently, as in #3102 (comment) or #3854 (comment).

[pnorman]

Closing as stale