Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Experimental Layer: TerrainLayer #3984

Merged
merged 18 commits into from
Feb 5, 2020
Merged

Add Experimental Layer: TerrainLayer #3984

merged 18 commits into from
Feb 5, 2020

Conversation

chrisgervang
Copy link
Collaborator

@chrisgervang chrisgervang commented Dec 9, 2019

Background

Many transportation projects, such as UAM network planning projects, need to be able to represent terrain and the real world. This layer is the first demonstration of loading global elevation data with deck.gl. It opens the door to more exploration, such as offsetting any layer by terrain. Eventually, it'd be amazing to correctly rendering entire cities, the buildings within them, and the aviation layers defined above them in AGL (above ground level) units.

This TerrainLayer is able to produce textured terrain meshes out of any rectangular image with elevation encoded as RGB values. It produces an optimal mesh using the Martini library. It asynchronously loads the terrain texture and will render an colored mesh by default, unless a surface texture is supplied.

Terrain Layer Props
// Image that encodes height data
terrainImage: {type: 'object', value: null, async: true},
// Image to use as texture
surfaceImage: {type: 'object', value: null, async: true},
// Martini error tolerance in meters, smaller number -> more detailed mesh
meshMaxError: {type: 'number', value: 4.0},
// Bounding box of the terrain image, [minX, minY, maxX, maxY] in world coordinates
bounds: {type: 'object', value: [0, 0, 1, 1]},
// Color to use if surfaceImage is unavailable
color: {type: 'color', value: [255, 255, 255]},
// Function to decode height data, from (r, g, b) to height in meters
getElevation: {type: 'accessor', value: (r, g, b) => r}

For demonstration purposes, we've implemented an example application to load global terrain and satellite imagery provided by Mapbox.

73803292-28d4c580-4775-11ea-987a-e64315930eac

Change List

  • Adding TerrainLayer.
  • Adding experimental terrain-layer example.

@tgorkin tgorkin added this to the 8.1 milestone Dec 11, 2019
@kylebarron
Copy link
Collaborator

I tested the example and got these errors in the console:

webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/shader.js?:123 luma: GLSL compilation errors in vertex shader ground-level-layer-vertex-shader-1
463: 
464:   vec4 position_worldspace = vec4(curr, 1.0);
465:   gl_Position = project_position_to_clipspace(position_worldspace);
^^^ ERROR: 'assign' : cannot convert from 'const mediump float' to 'Position highp 4-component vector of float'
Uncaught (in promise) Error: GLSL compilation errors in vertex shader ground-level-layer-vertex-shader-1
    at VertexShader._compile (webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/shader.js?:125)
    at VertexShader.initialize (webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/shader.js?:73)
    at VertexShader.Shader (webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/shader.js?:58)
    at new VertexShader (webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/shader.js?:157)
    at Program.initialize (webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/program.js?:78)
    at new Program (webpack://App/./node_modules/@luma.gl/webgl/dist/esm/classes/program.js?:58)
    at ProgramManager.get (webpack://App/./node_modules/@luma.gl/core/dist/esm/resource-management/program-manager.js?:197)
    at Model._checkProgram (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/base-model.js?:305)
    at Model.initialize (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/base-model.js?:89)
    at Model.initialize (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/model.js?:47)
    at Model.BaseModel (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/base-model.js?:34)
    at new Model (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/model.js?:39)
    at TerrainLayer._getModel (webpack://App/./src/terrain-layer/TerrainLayer.js?:221)
    at TerrainLayer.updateState (webpack://App/./src/terrain-layer/TerrainLayer.js?:91)
    at TerrainLayer._updateState (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer.js?:811)
    at TerrainLayer._initialize (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer.js?:780)
    at LayerManager._initializeLayer (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer-manager.js?:461)
    at LayerManager._updateSublayersRecursively (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer-manager.js?:392)
    at LayerManager._updateSublayersRecursively (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer-manager.js?:413)
    at LayerManager._updateLayers (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer-manager.js?:345)
    at LayerManager.setLayers (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer-manager.js?:205)
    at LayerManager.setProps (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/layer-manager.js?:164)
    at Deck.setProps (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/deck.js?:244)
    at Deck._setGLContext (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/deck.js?:682)
    at Deck._onRendererInitialized (webpack://App/./node_modules/@deck.gl/core/dist/esm/lib/deck.js?:716)
    at AnimationLoop.onInitialize (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/animation-loop.js?:281)
    at eval (webpack://App/./node_modules/@luma.gl/core/dist/esm/lib/animation-loop.js?:159)

@chrisgervang
Copy link
Collaborator Author

Thanks for posting the error, that’s the same runtime error I’m getting right now too. We’re going to try to fix it soon (seems to have come from some recent updates)

@kylebarron
Copy link
Collaborator

kylebarron commented Dec 12, 2019

Regarding Martini, I think the main difference would be that you'd need to pass the image data to the geometry generator. It looks like right now, you're calculating the grid geometry just from the x and y resolutions and the bounding box. For its optimizations, Martini needs the elevation data when creating the geometry.

I think a basic martini-geometry.js would be something like

import GL from "@luma.gl/constants";
import { Geometry } from "@luma.gl/core";
import Martini from '@mapbox/martini'

export default class MartiniGeometry extends Geometry {
  constructor(opts = {}) {
    const { id = "martini-geometry" } = opts;

    const mesh = runMartini(opts);
    const indices = mesh.triangles
    const positions = mesh.vertices

    super(
      Object.assign({}, opts, {
        id,
        drawMode: GL.TRIANGLES,
        attributes: {
          // No size/type information is needed for known vertex names
          indices,
          positions
        },
        vertexCount: indices.length
      })
    );
  }
}

// From Martini demo
// https://observablehq.com/@mourner/martin-real-time-rtin-terrain-mesh
function runMartini({ pngData, xResolution }) {
  const tileSize = xResolution;
  const gridSize = tileSize + 1
  const terrain = new Float32Array(gridSize * gridSize);

  let R, G, B, height;
  for (let x = 0; x < tileSize; x++) {
    for (let y = 0; y < tileSize; y++) {
      // .get method is from ndarray, which is how I had queried the png files
      // https://github.com/scijs/ndarray
      R = pngData.get(x, y, 0)
      G = pngData.get(x, y, 1)
      B = pngData.get(x, y, 2)
      height = -10000 + ((R * 256 * 256 + G * 256 + B) * 0.1)
      terrain[y * gridSize + x] = height;
    }
  }

  // backfill right and bottom borders
  for (let x = 0; x < gridSize - 1; x++) {
    terrain[gridSize * (gridSize - 1) + x] = terrain[gridSize * (gridSize - 2) + x];
  }
  for (let y = 0; y < gridSize; y++) {
    terrain[gridSize * y + gridSize - 1] = terrain[gridSize * y + gridSize - 2];
  }

  const martini = new Martini(gridSize);
  const tile = martini.createTile(terrain);
  const mesh = tile.getMesh(10);
  return mesh;
}

I'm also curious if you've tested this with vector layers as well as raster layers. Anyways, this is really exciting stuff.

@chrisgervang
Copy link
Collaborator Author

I did some research into using Mapbox vector map tiles, rendering them with the GeoJsonLayer without any terrain awareness. My primary goal was to customize the styles, and when I unable to get the result I was looking for I abandoned the idea. Off topic of terrain, but it seems like a part I’m missing there is a parser/mapper function between a mapbox style JSON, and the vector file data (the mapping isn’t intuitive).

I guess that’s a long way of saying I didn’t test terrain mesh with vector tiles yet.

Is a raster layer different from satellite imagery? This layer renders satellite map tiles or any bitmap tile on the terrain mesh (This PR also demos rendering Aviation navigation charts on the mesh).

In the future I’d like to add a LayerExtension to bring “terrain awareness” to any layer. We have some ideas on how that might be done.

@kylebarron
Copy link
Collaborator

parser/mapper function between a mapbox style JSON, and the vector file data

In a style JSON, the sources key maps individual sources to keys that are later referenced when styled. So for example in this osm-liberty style, two separate map layers are imported, and then the keys of that object are referenced later for each piece of the style. So this rule takes the openmaptiles source, finds all the geometries in the waterway layer within the vector tile, and then styles the ones that meet the filter criteria.

You can have as many sources as you want. For example, with the map I'm currently developing, right now I have 5.

Raster is the same thing as satellite imagery. Or more specifically, satellite is a type of raster data.

You do set mapbox-dark as the baselayer for the StaticMap, which is a vector layer, but then it's presumably hidden by the layers provided in TileLayer. What I'm curious about is: if the only layer provided here is the Terrain RGB layer, and you don't also load OSM/Satellite imagery, is the StaticMap basemap, which could be a vector style, projected onto the terrain?

Another reason I asked about the vector layer is that tiles conforming to Mapbox's Terrain RGB spec can also be used for on-device hillshading. If the terrain-rgb tiles are used both for hillshading and for the TerrainLayer, it would be nice if they weren't loaded twice.

@ibgreen
Copy link
Collaborator

ibgreen commented Dec 12, 2019

@chrisgervang Would love to see a screenshot...

@infospark
Copy link

Instead of
gl_Position = project_position_to_clipspace(position_worldspace);

This seems to work well:
gl_Position = project_position_to_clipspace(positions, positions64xyLow, vec3(0.0,0.0,curr.z), geometry.position);

@kylebarron
Copy link
Collaborator

As someone who doesn't know WebGL, is there anything I can do to help move this along?

@chrisgervang
Copy link
Collaborator Author

gl_Position = project_position_to_clipspace(positions, positions64xyLow, vec3(0.0,0.0,curr.z), geometry.position);

@infospark I tried this out and got this error:

504:   gl_Position = project_position_to_clipspace(positions, positions64xyLow, vec3(0.0,0.0,curr.z), geometry.position);
^^^ ERROR: 'assign' : cannot convert from 'const mediump float' to 'Position highp 4-component vector of float'

Is there something else I need to change?

@infospark
Copy link

infospark commented Jan 27, 2020

I created a fork with the changes I made here. https://github.com/infospark/deck.gl/tree/terrain-layer.
I should note that it's working in Chrome and Firefox but I get WebGL: ERROR: 0:225: '27' : memory exhausted in Safari. Not yet tried to diagnose that.

The fork is now also upgraded to deck.gl 8 (uses positions64Low). Still doesn't work in Safari unfortunately.

@Pessimistress
Copy link
Collaborator

gl_Position = project_position_to_clipspace(positions, positions64xyLow, vec3(0.0,0.0,curr.z), geometry.position);

positions64xyLow (vec2) was changed to positions64Low (vec3) from 7.3 to 8.0.

@Pessimistress
Copy link
Collaborator

image

image

@coveralls
Copy link

coveralls commented Feb 5, 2020

Coverage Status

Coverage remained the same at 80.9% when pulling be15154 on terrain-layer into 6526772 on master.

@chrisgervang chrisgervang changed the title TerrainLayer WIP Add Experimental Layer: TerrainLayer Feb 5, 2020
@chrisgervang chrisgervang merged commit af5e949 into master Feb 5, 2020
@chrisgervang chrisgervang deleted the terrain-layer branch February 5, 2020 02:31
@chrisgervang chrisgervang mentioned this pull request Feb 5, 2020
7 tasks
@Pessimistress Pessimistress mentioned this pull request Feb 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants