Make query methods return raster features/sources

[samanpwbb]

It would be useful if raster layers were included in featuresAt. Especially for raster layers with sources with limited bounds.

I see here that this is a noop:

mapbox-gl-js/js/source/raster_tile_source.js

Line 98 in 2a2aec4

callback(null, []);

[scothis]

Looks to already be working.

[samanpwbb]

@scothis in that example, the raster layers are hacked in to show up across the whole map. Ideally the raster layer only appears in the popup if it exists where user clicked.

[lbud]

At this time we don't return raster sources or features from map.query*Features at all. We should do this.

This will be dependent on #3186: querying searches for features in a tile, but since we shoehorn features into a tile, querying a point on a visible raster layer may search for points past a tile (the source cache's tile)'s "extent."

The equivalent of the noop mentioned in #1404 (comment) is now query_features.js#L12: raster layers don't have feature indices. querySourceFeatures also only queries geojson and vector data.

This was brought up in #3745. Indeed for something like a canvas-based raster layer it would be helpful to return an [x,y] coordinate mapped back to the original raster data.

[mollymerp]

This could be particularly useful for raster layers like the terrain rgb where users could query the pixel values and calculate elevation at a point based on the raster data 💭

[jucor]

Any luck on this? This would very useful.

[jfirebaugh]

#5916 notes that raster layers with a canvas source should be included with this feature.

[shawnd]

This could be particularly useful for raster layers like the terrain rgb where users could query the pixel values and calculate elevation at a point based on the raster data 💭

Exactly the use case I have. Layer with rgb data that I want to be able to query with a click event point.

[felix-ht]

Any updates on this?

[ansarikhurshid786]

any updates or solution ?

[dongmingwang2198881]

any updates or solution ?

[gagecarto]

Oh gosh.. This would be so helpful especially for querying via RGB values.. fingers crossed on this one

[nextstopsun]

Any news here? I'd like to query RGB values of underlying raster pixel too.

[mprove]

While this is not working, you can add a note to the documentation that it does not work for raster layers. This would reduce confusion.
https://docs.mapbox.com/mapbox-gl-js/api/map/#map#on

[nm2501]

I'm interested in a solution. Adding my upvote here...
👍

[jbeuckm]

Yes, and it would be great to get a distribution of raster color values within a given polygon.

[jbeuckm]

Well, I worked a new method through to the tile.js layer here . The idea is that you could ask for the value from a raster source at a given location:

map.on("mousemove", (event) => {
    map.queryRasterSource(<my raster sourceId>, event.point);
});

But this still needs the function to pull a pixel value out of the raster source:

    // Find raster value at a given location
    queryRasterValues(
        layers: { [_: string]: StyleLayer },
        serializedLayers: { [string]: Object },
        sourceFeatureState: SourceFeatureState,
        tileResult: TilespaceQueryGeometry,
        params: {
            filter: FilterSpecification,
            layers: Array<string>,
            availableImages: Array<string>,
        },
        transform: Transform,
        pixelPosMatrix: Float32Array,
        visualizeQueryGeometry: boolean
    ): Color {
        // const texturePos = pixelPosMatrix.getTexturePos()
        // const color = new ArrayBuffer()
        // tileResult.tile.texture.readPixels(texturePos.x, texturePos.y, 1, 1, type, &color)
        // return color
    }

It might be that we need to use the Painter to render the source/layer for a viewport that is one pixel square? That seems like a lot of code for one pixel. Maybe there is a way to query the screen x,y for the already-rendered layer? Would a new painter function work?

painter.renderLayerPixel(layerId, mousePos) {
        this.context.viewport.set([mouseX, mouseY, 1, 1]);
        this.renderLayer(this, sourceCache, layer, coords);
        // how to get the pixel value???
}

I'm hoping someone who knows the internals well can jump in.

[jbeuckm]

Here is something that is starting to kind of work. I started with the Painter.render() function and tried to pare down to rendering just the requested layer. Then I read the requested pixel value. When I try this with multi-layer maps, it interferes with the main map render and the result is not stable. Someone who understands the gl buffers and how mapbox uses them should be able to fix that and maybe improve this to just render the pixel rather than the entire viewport.

    colorForLayerPoint(layerId: string, point: PointLike) {

        const layer = this.style._layers[layerId];
        const sourceCache = this.style._getLayerSourceCache(layer);

        sourceCache.prepare(this.context);

        const coordsAscending: Array<OverscaledTileID> = sourceCache.getVisibleCoordinates();
        const coordsDescending: Array<OverscaledTileID> = coordsAscending.slice().reverse();

        // Rebind the main framebuffer now that all offscreen layers have been rendered:
        // this.context.bindFramebuffer.set(null);
        this.context.viewport.set([0, 0, this.width, this.height]);

        this.context.clear({
            color: 'rgba(0,0,0,0)', //options.showOverdrawInspector ? Color.black : clearColor,
            depth: 1,
        });
        this.clearStencil();

        this._showOverdrawInspector = false; //options.showOverdrawInspector;

        this.renderPass = "translucent";


        // For symbol layers in the translucent pass, we add extra tiles to the renderable set
        // for cross-tile symbol fading. Symbol layers don't use tile clipping, so no need to render
        // separate clipping masks
        const coords = sourceCache
            ? coordsDescending
            : undefined;

        this._renderTileClippingMasks(
            layer,
            sourceCache,
            sourceCache ? coordsAscending : undefined
        );
        this.renderLayer(this, sourceCache, layer, coords);

        const gl = this.context.gl;
        var pixel = new Uint8Array(4);
        gl.readPixels(
            point.x * window.devicePixelRatio,
            this.height - point.y * window.devicePixelRatio,
            1,
            1,
            gl.RGBA,
            gl.UNSIGNED_BYTE,
            pixel
        );

        return pixel;
    }

[Carlos-Carreno-Berlanga]

This would be very useful, for generating interactive tooltip information for raster sources. My use case is generating a tooltip for a weather map.

[mkaskov]

any updates?

[tredpath]

You can force the map to repaint after querying the color and it will render correctly. I didn't see any flickering or stuttering but it's not great performance wise when dragging the cursor over the map. For my use it was good enough to only query the color when the cursor has stopped moving to improve that.

let timeout
map.on("mousemove", e -> {
    clearTimeout(timeout)
    timeout = setTimeout(() => {
        //query the color for a layer created elsewhere
        const clr = map.colorForLayerPoint("layer_id", e.point)
        //draw the map to fix anything getting the color broke
        map.triggerRepaint()
        //use clr to do something interesting
        useColor(clr)
    }, 150)
})

If you want to use the color as a lookup as I did you also need to be careful how you add the layer to the map. Raster layers need resampling set to nearest and you need the latest version with the fix from #12577 to disable interpolation as best as possible.

const layer = {
    id: "layer_id",
    type: "raster",
    source: "source_id",
    paint: {
        "raster-resampling": "nearest"
    }
}