WebRave Raster Color Ramps

[MattReimer]

NB: This is just a place to collect my thoughts. I'll ask @philipbaileynar nicely to make a video once I know how this is going to work.

We need color ramps (they're not actually csv files) so that webrave tilerizer can hook in and bake the ramps into projects

These ramps need to be compatible with gdaldem because that's what we're using to make the RGBA rasters

From the GDALDem Docs

Note: the syntax of the color configuration file is derived from the one supported by GRASS r.colors utility. ESRI HDR color table files (.clr) also match that syntax. The alpha component and the support of tab and comma as separators are GDAL specific extensions.

Some examples of valid rows are here:

# this named color at absolute value 3500
3500   white

# this is R:G:B color at absolute balue 2500
2500   235:220:175

# This is RGBA(190,185,135,50) so it has some transparency at relative raster value 50% between min and max
50%   190 185 135 50
700    240 250 150
0      50  180  50

# "nv" is the special key that helps us symbolize NODATA values. Here it is set to 0 (perfectly transparent). I'm guessing most our ramps will need this line.
nv     0   0   0   0

We should be able to get compatible files exported from QGIS AND ArcMap but I'm not 100% sure that's going to work yet.
https://desktop.arcgis.com/en/arcmap/10.3/manage-data/raster-and-images/creating-a-color-map-clr-file.htm

QGIS

In QGIS you can export a color ramp to a file. I still need to check if it works with GDALDem

ArcMap

I tried to use the isntructions here for creating color ramps but I kept getting an error

[MattReimer]

@philipbaileynar in the latest kick in the teeth, QGIS exported ramps DON'T WORK with gdaldem...

For each one I had to hand-edit it to get it into the right format and that meant switching from absolute scale to % relative scale and adding a nodata value.

Then I had to test each one with gdaldem manually to make sure I did it right.

The whole thing is a lot more trouble than I had hoped but it does work

[MattReimer]

I did some more investigation and I don't think I can go any further before I get a discussion and signoff from either both @joewheaton and @philipbaileynar

Here are two color ramps that have been exported by QGIS:

DEM

# QGIS Generated Color Map Export File
INTERPOLATION:INTERPOLATED
1259.3421630859375,255,235,176,255,1259
1670.69324584960941138,38,115,0,255,1671
2082.04432861328132276,115,77,0,255,2082
2505.860595703125,255,255,255,255,2506

Fair market value

# QGIS Generated Color Map Export File
INTERPOLATION:DISCRETE
1000,0,97,0,255,< $1,000
5000,51,129,0,255,$1,000 to $5,000
10000,102,160,0,255,$5,000 to $10,000
25000,153,192,0,255,$10,000 to $25,000
50000,204,224,0,255,$25,000 to $50,000
100000,255,255,0,255,$50,000 to $100,000
200000,255,211,0,255,$100,000 to $200,000
250000,255,167,0,255,$200,000 to $250,000
500000,255,122,0,255,$250,000 to $500,000
1000000,255,78,0,255,$500,000 to $1,000,000
inf,255,34,0,255,> $1,000,000

The good:

• Dead simple to obtain. One-button export from QGIS
• Contains the interpolation method. i.e. INTERPOLATED vs DISCRETE this needs to feed in as a parameter to GDALDEM as --exact_color_entry and --nearest_color_entry
• Contains the optional label column we can use in the map legend. i.e. $1,000 to $5,000. There is a lot of important context there that we can use to build our legend values in webRAVE

The bad:

• They're not quite CSV files
• The QGIS doesn't escape CSV columns correctly (count the "," on each line to see what I mean)
• The QGIS export doesn't implement any standard I can find. Above are two possible variations. There may be more.
• Not directly usable by GDALDem (the thing we use to build the tiles).
  In order to use it we need to:
  • Strip off the label column, the INTERPOLATION: row and the comment (#) row
  • Add percentages for the interpolated ramp where appropriate
  • Add a nv row fow nodata values.

Here's the translation for what GDALDem can use. As you can see it's pretty different and I worry the translation could be quite error prone and might need a human eye.

DEM

elevation,r,g,b,a
nv,0,0,0,0
0%,255,235,176,255,1306
25%,38,115,0,255,1627
75%,115,77,0,255,1948
100%,255,255,255,255,2279

Fair market value

val,r,g,b,a
nv,0,0,0,0
1000,0,97,0,255
5000,51,129,0,255
10000,102,160,0,255
25000,153,192,0,255
50000,204,224,0,255
100000,255,255,0,255
200000,255,211,0,255
250000,255,167,0,255
500000,255,122,0,255
1000000,255,78,0,255
10000000,255,34,0,255

TL;DR: The Problem

The problem is:

What do we store in RiverscapesXML and who does the translation between the QGIS export what WebRAVE and GDALDem need?

[MattReimer]

Discussed with @joewheaton. We will use color tables as-is from QGIS and I will write a couple of interpreters to build the legend and the gdal file from them

Also I filed a feature enhancement with QGIS: qgis/QGIS#43786

[MattReimer]

I'm re-opening this since we still don't have a good categorical workflow.

[MattReimer]

Closing this in favour of #130