Rendering errors (symbol clipping) at vector tile edges

[chriszrc]

I'm seeing the circle markers getting cut off at the tile edges:

My understanding is the the tiles themselves are fine, since you can't clip a point. It looks like it's a problem with the translation of the point to a CircleMarker. Is there a fix for this?

[chriszrc]

Regular icon markers have the same problem:

[chriszrc]

It looks like this was a known issue with the geojson-vt lib

mapbox/mapbox-gl-js#1271

And the fix was to change the buffer size for geojsonvt. From the source code of VectorGrid.Protobuf.js, it looks like geojsonvt was at one point used, but not anymore. Is there some other place we can easily adjust the buffering of tiles so we don't get symbol clipping?

initialize: function(url, options) {
		// Inherits options from geojson-vt!
                // this._slicer = geojsonvt(geojson, options);
		this._url = url;
		L.VectorGrid.prototype.initialize.call(this, options);
	},

[drzhouq]

@chriszrc We also suffered from this issue. Hopefully, our solution will help you. It has nothing to do with the VectorGrid. It turns out the issue can be addressed through the mbtiles file generated using the tippecanoe tool. By specifying the base zoom and buffer distance when generating the mbtiles file, we are able to have duplicated points in different tiles to cover the same location. After rendering by VectorGrid, the two or more clipped symbols(circle) becomes one, gives the impression that there is no clipping. The command is something like this:
tippecanoe -z20 -s EPSG:3857 -b 10 -B14 -rf30 -o points.mbtiles points.json
The -z20 means to generate tiles to maximum 20; -b 10 means using a buffer 10px (if any point feature falls in the buffer, it is duplicated); -B14 means the at zoom level 14, I want all feature be present (if this is not specified, it will use default maximum scale and won't generate duplicates less than the maximum, which in this example is 20. That means no duplicates under zoom scale 20, which won't solve your clipping issue). And -rf30 means, any zoom scale level smaller than 14 (13, 12,11...9), only 30 point features are allowed, this is to avoid too much points show up when map is zoomed out.

[chriszrc]

I'm using postgis to create the mvt tiles, and it had a "buffer" argument as well, it was one of the first things I played with, but it never changed the result. But reading your reply, I decided to dig a little deeper and did a few more experiments, like actually changing the size of the "bounds" for the tile, and sure enough, I was able to get rid of the clipping!

Now however, I'm left with a different problem, in that it looks like the postgis function is not really doing what it's supposed to be doing for points :(

[chriszrc]

Hahaha, I found the problem, here's what the postgres query looked like before:

SELECT ST_AsMVT(q1, 'layername', 4096, 'geom') as mvt
FROM (
       SELECT layername_id as i, category_code as c,
                ST_AsMVTGeom(t.web_geom, TileBBox($5::int, $6, $7), 4096, 256, false) geom
       FROM layername t,
       (SELECT ST_MakeEnvelope($1, $2, $3, $4, 3857) as geom) as env
      WHERE ST_Intersects(t.web_geom,  env.geom)
)q1
 -- PARAMETERS: [-8296780.798186171,4774562.53480525,-8257645.039704161,4813698.293287256,10,300,389]

ST_AsMVTGeom is the function for creating the mvt geom, and it takes the param "256" as the buffer size (in number of pixels) to expand the contents of the tile beyond the bounds. However, in this case, it had no effect, because in order to make the query efficient, the point features are already being filtered by the "WHERE ST_Intersects()" so there never were any points outside the bounds to optionally include the first place.

Doh.

I'm not actually sure what the best way to expand the envelope is, given that the size of the buffer changes with the zoom level, but mapbox published the TileBBox function, as well as some other helpful functions, one of them being the ZRes function (https://github.com/mapbox/postgis-vt-util#zres) which is useful to help simplify geometries according to how much detail can realistically be seen at a given zoom level. Seems like it almost fits the bill using it like:

SELECT ST_AsMVT(q1, 'layername', 4096, 'geom') as mvt
FROM (
       SELECT layername_id as i, category_code as c,
                ST_AsMVTGeom(t.web_geom, TileBBox($5::int, $6, $7), 4096, 256, false) geom
       FROM layername t,
       (SELECT ST_Expand(ST_MakeEnvelope($1, $2, $3, $4, 3857),ZRes($5)*$5) as geom) as env
      WHERE ST_Intersects(t.web_geom,  env.geom)
)q1
 -- PARAMETERS: [-8296780.798186171,4774562.53480525,-8257645.039704161,4813698.293287256,10,300,389]

[drzhouq]

@chriszrc Many thanks for sharing your findings, Chris. Your findings and solution will save me a lot of pains because I am planning to use the postgis to dynamically generate the tiles in the future. The only reason I haven't made the transition is that AWS RDS currently doesn't have the necessary protobuf library to support the function of ST_AsMVT.

[chriszrc]

@drzhouq No problem :)
I would not however wait for RDS to include protobuf support, it's a very customized build of postgis, and I doubt if they'll ever do it. We're using postgres on ec2, and it's really not so bad to install and maintain yourself-

[drzhouq]

cool. Thanks, @chriszrc
How does postgis handle the dropping the points when you zoom out? You don't want to display so many points when you are zoomed out. I didn't read the source code for the "ST_AsMVTGeom", but from the description, they vaguely mentioned about this (...might collapse geometry into a lower dimension in the process...)

[chriszrc]

@drzhouq Postgis does nothing on this front. Thankfully, mapbox open sourced their set of tools for dealing with mvtiles in postgis. To deal with filtering points based on zoom levels, they have "LabelGrid":

https://github.com/mapbox/postgis-vt-util#labelgrid

Which does the process that's outlined here (carto did not release their tools....)

https://carto.com/blog/inside/tile-aggregation/

It works well-

[drzhouq]

Fantastic. Many thanks for the pointers, @chriszrc . It seems the carto's approach supports the aggregation of attributes while the lebelgrid just drops the points. I believe Carto shares their codes on github (most likely in their windshaft repo). They also did a performance comparison of generating 10K points tiles usinng postgis and mapnik. https://carto.com/blog/inside/MVT-mapnik-vs-postgis/

[chriszrc]

Yeah, I read that article too. Well hey, if you find the code carto is using, pass it along! :)

…

On Tue, Apr 17, 2018 at 10:46 AM, John Zhou ***@***.***> wrote: Fantastic. Many thanks for the pointers, @chriszrc <https://github.com/chriszrc> . It seems the carto's approach supports the aggregation of attributes while the lebelgrid just drops the points. I believe Carto shares their codes on github (most likely in their windshaft repo). They also did a performance comparison of generating 10K points tiles usinng postgis and mapnik. https://carto.com/blog/inside/ MVT-mapnik-vs-postgis/ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#149 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYgWKQzjPdsHjkwT28M5LDmPUOFztfqZks5tpgBWgaJpZM4Skhle> .

-- -- Chris Marx ZevRoss - Know Your Data Data Science & Spatial Analysis New website: www.zevross.com Ithaca, NY

[drzhouq]

@chriszrc The LabelGrid can be used to aggregate attributes, therefore, there is no need to tap into carto code. Just out of my curiosity, I located their aggregation code at at https://github.com/CartoDB/Windshaft-cartodb/blob/master/lib/cartodb/models/aggregation/aggregation-query.js

[chriszrc]

@drzhouq Nice! thanks for finding that, still interesting to see their code, I can see why they didn't release this, it's all in JS and wrapped up in their app-

[drzhouq]

@chriszrc Now I am going to implement the vector tile in postgis. Reading your code, I am not sure why you need to have the parameter for $1,$2,$3,$4? In order to use your function, one has to pass seven parameters in? z, x, y (5,6,7), but why 1,2,3,4? wouldn't you be able to get the 1,2,3,4 from the z,x,y? Thanks for your help.

[chriszrc]

Hi,

Yes, I do get the bbox coordinates from the zxy, but I get them in node and
then pass them to postgres. If there's some way to do that in postgres let
me know!

import * as SphericalMercator from '@mapbox/sphericalmercator';

this.mercator = new SphericalMercator();
let bbox = this.mercator.bbox(ctd.x, ctd.y, ctd.z, false, "900913");

//and in the query
[bbox[0], bbox[1], bbox[2], bbox[3].... etc]

[drzhouq]

@chriszrc That's why I am puzzled. The TileBox gives you the same thing. You can confirm that with select st_xmin((st_makeenvelope(-8296780.798186171,4774562.53480525,-8257645.039704161,4813698.293287256,3869))), st_xmin(Tilebbox(10,300,389));
You still need to perform the intersect, but you don't need to pass in those 4 parameters.

[chriszrc]

Ah, You're totally right, yes these queries were assembled slowly from both the postgis docs and online sources. And often, the postgis layers would be in srid 4326 rather than the webmercator, so the bounding box for the intersect wouldn't be the same as the tilebbox (although if you look at the source for tilebbox, there is an option to do the transform to other projections in there). That seemed inefficient to me, so I added web_geom fields to my tables, which was just the geom already transformed into the webmercator projection, so I didn't have to do any transforms on the fly. But then I never got around to removing the st_makeevelope, which yes, is totally doing the exact same thing as the tilebbox-

…

On Fri, Jun 15, 2018 at 9:05 PM, John Zhou ***@***.***> wrote: @chriszrc <https://github.com/chriszrc> That's why I am puzzled. The TileBox gives you the same thing. You can confirm that with select st_xmin((st_makeenvelope(-8296780.798186171,4774562. 53480525,-8257645.039704161,4813698.293287256,3869))), st_xmin(Tilebbox(10,300,389)); You still need to perform the intersect, but you don't need to pass in those 4 parameters. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#149 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYgWKSQKIyv8Jx1KK1UvDW9JjwiBfk8Tks5t9FnugaJpZM4Skhle> .

-- -- Chris Marx ZevRoss - Know Your Data Data Science & Spatial Analysis New website: www.zevross.com Ithaca, NY

[mngyng]

Struggled for a while and the almighty Google got me here to this pain reliever. Thanks @chriszrc for the wonderful hint!

For those who follows the example on the main page, here's my simplified solution for you to get the idea:

SELECT your_id as your_id, your_var_1 as your_var_1, ST_AsGeoJSON(geom) as the_geom_geojson
 FROM your_layer_name lyr, (SELECT ST_Expand(!bbox_4326!,0.02) as xbbox) as envlp
 WHERE ST_Intersects(lyr.geom, envlp.xbbox)

The "lyr" and "envlp" stands for the point vector layer and the envelope with extended bounding box in the query. The alias is temporarily used, so you don't need to name it like that to make it work. Same thing goes with all the "your_" variables. That "your_id" needs to be consistent with what's called by getFeatureId, though.

I extended the bounding box by 0.02 degrees on the WGS84 coordinate, which is around 200 meters in the applied area of my map. I set the point tiles to shows up only at level 13 and when further zoomed-in, so a buffer of 200 meters works quite well in my case.

I'll work on the buffer zone at the different zoom levels when I have time. So far I'm happy with my outcome.

[dbauszus-glx]

@chriszrc

Just came across this as I was looking into way to expand the envelope for the ST_AsMVTGeom input.

I use ST_DWithin in the where statement and pass in r halved. Where r is the param used to generate the tile at a specified zoom level in the first place.

let m = 20037508.34;

let r = (m * 2) / (Math.pow(2, z));


        `SELECT
          ${id} id,
          ST_AsMVTGeom(
            ${geom_3857},
            ST_MakeEnvelope(
              ${-m + (x * r)},
              ${ m - (y * r)},
              ${-m + (x * r) + r},
              ${ m - (y * r) - r},
              3857
            ),
            4096,
            256,
            true
          ) geom

        FROM ${table}

        WHERE
          ST_DWithin(
            ST_MakeEnvelope(
              ${-m + (x * r)},
              ${ m - (y * r)},
              ${-m + (x * r) + r},
              ${ m - (y * r) - r},
              3857
            ),
            ${geom_3857},
            ${r/2}
          )`

I am aware that this is quite an old post. But AWS implemented protobuffer support a while ago.

https://forums.aws.amazon.com/thread.jspa?threadID=277371

This should now also be supported via aurora.