[E & A] Implement colormap configurability #1117

dzole0311 · 2024-08-21T08:27:15Z

Related Ticket: #994

Description of Changes

Implements the color map configuration of E&A layers.

Changes:

Adds curated list of supported color maps taken from the back-end, split into diverging and sequential groups
Supports other valid color map names from the list of color maps: https://openveda.cloud/api/raster/colorMaps
Adds support for reversing the colors
Adds reset all button which currently only resets the "Reverse" option, but will also apply to the "Rescale" functionality which is to be added next
Added a loading skeleton to the colormap in the timeline sidebar when a layer is added. This prevents a visual flash because it might take a second to fetch the color map from the back-end (cc @faustoperez )

The precedence of the color maps are as follows:

Start with what is configured in the sourceParams of the dataset
If it's not defined, check for the dashboard render configuration coming from STAC endpoint
If still undefined, check for asset-specific renders using the sourceParams' assets. This is a special edge case for some datasets that have multiple assets (multiple assets under one collection item)
If all else fails, default to viridis

Please let me know your thoughts on that or if you spot something strange

Notes & Questions About Changes

I additionally pushed a shell script that I used to build the colorMaps.ts file, I hope that's okay. It fetches all the rgba values for each supported colorMap from titiler. We could extend it and integrate it within our Github workflow in the future, so that the colors are fetched on each new release (although it's unlikely they will change that often)

Validation / Testing

Verify that the color map list is correctly split into diverging and sequential groups
Check that reversing the color map works as expected and toggles correctly
Test the "Reset All" button to ensure that it resets the "Reverse" option and works as intended
Check that the precedence of the color maps is followed in the correct order:

Start with sourceParams of the dataset
Fallback to dashboard render configuration from the STAC endpoint
If undefined, check for asset-specific renders using sourceParams' assets
Finally, default to viridis if all else fails

Verify that the changes do not break any existing functionality related to rendering of the tiles on the map (selected color map must match the tile colors)
Test across different datasets with both single and multiple assets to confirm edge cases are handled as described

…ap_name

netlify · 2024-08-21T08:27:30Z

✅ Deploy Preview for veda-ui ready!

Name	Link
🔨 Latest commit	`60559fd`
🔍 Latest deploy log	https://app.netlify.com/sites/veda-ui/deploys/66cf139d23f2670008f3e2cb
😎 Deploy Preview	https://deploy-preview-1117--veda-ui.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

faustoperez · 2024-08-22T12:35:49Z

Added a loading skeleton to the colormap in the timeline sidebar when a layer is added. This prevents a visual flash because it might take a second to fetch the color map from the back-end (cc @faustoperez )

Sounds great @dzole0311 !

hanbyul-here

This needs to be changed if we want to give a priority to manual sourceParams: https://github.com/NASA-IMPACT/veda-ui/pull/1117/files#diff-fc839f8286ca1783e679e0fcddd8a3da452570ebbe9f433e16fda7003b91687eR376 (but hopefully we can move on from manual configurations soon 😬 !)

app/scripts/components/common/map/style-generators/raster-timeseries.tsx

app/scripts/components/exploration/hooks/use-stac-metadata-datasets.ts

app/scripts/components/exploration/components/datasets/data-layer-card.tsx

app/scripts/components/exploration/components/datasets/colormap-options.tsx

app/scripts/components/common/map/layer-legend.tsx

hanbyul-here · 2024-08-22T15:42:29Z

app/scripts/components/common/map/style-generators/raster-timeseries.tsx

@@ -358,7 +374,8 @@ export function RasterTimeseries(props: RasterTimeseriesProps) {
        const tileParams = qs.stringify(
          {
            assets: 'cog_default',
-            ...(sourceParams ?? {})
+            ...(sourceParams ?? {}),
+            colormap_name: colorMap


🤔 probably a nitpick, If colormap is optional anyway, can we do

...colorMap && {colormap_name: colorMap}

and get rid of the https://github.com/NASA-IMPACT/veda-ui/pull/1117/files#diff-b015ae8d170bf7762b2b8ad8c59673574d21cced66fccabdf7cac68e7d30b864R27?

We need this in order to check if the colorMap is part of the url params. Did you mean we could move it one level up or completely get rid of it?

app/scripts/components/exploration/hooks/use-stac-metadata-datasets.ts

hanbyul-here · 2024-08-22T16:31:08Z

One more - do you see any reason not to apply dynamic colormap to other types of raster layers? (cmr and zarr)? - We can separate the issues for these layers since these are not being used on production anyway, but I was wondering if there is any specific reason!

dzole0311 · 2024-08-26T08:41:24Z

Lots of good comments @hanbyul-here, thanks!

One more - do you see any reason not to apply dynamic colormap to other types of raster layers? (cmr and zarr)? - We can separate the issues for these layers since these are not being used on production anyway, but I was wondering if there is any specific reason!

It somehow slipped my mind to pass the colormap to the RasterPaintLayer as well. Should work now for CMR and Zarr layers.

docs/content/frontmatter/layer.md

aboydnw

I think that somehow leading users back to the visualization recommended/configured by the science team will be important to stakeholders. Whether that's the reset button also resetting the colormap selection or providing some sort of visual cue which colormap is the default.

I see in the netlify preview that some of the datasets start with "Default," and some didn't. I assume this has to do with the logic you were talking about @dzole0311 to decide which colormap is the default. Is there any way to ensure that whatever is decided to be the default comes with this tag?

Also, another question, do we think users will want to see both Default AND the colormap name? cc @faustoperez

Example w/o default tag:

There is also an edge case that I discovered. If it's an easy fix, maybe worth addressing. Otherwise we can make a ticket and do it at some point in the future.
Steps to reproduce:

Open exploration tab and select layers
Change the colormap of one of the layers to something other than the Default
Notice that if you close and reopen the colormap bar, the Default tag is still there
Refresh the page and notice that the change in colormap is maintained (I believe this is desirable, so people can share views with others via URL)
Open the colormap bar and notice that the Default tag is no longer there

app/scripts/components/common/map/layer-legend.tsx

sandrahoang686 · 2024-08-27T18:10:49Z

app/scripts/components/exploration/components/datasets/colormap-options.tsx

+  return isReversed ? colors.reduceRight((acc, color) => [...acc, color], []) : colors;
+};
+
+export function ColormapOptions({ colorMap, setColorMap}: ColormapOptionsProps) {


Curious, with Jotai state management, cant we access the same state from the same atom? so instead of passing colorMap state and setter method, can we just pass in the datasetAtom and then do const [colorMap, setColorMap] = useTimelineDatasetColormap(datasetAtom); at this level to avoid prop drilling since datasetAtom was already being previously being passed into DataLayerCard?

**Related Ticket:** #994 ### Description of Changes I made the colormap default to be returned along with other settings values - I think this saves some redundancies of retrieving the value of colormap, and fits well with the existing pattern. I eliminated reconciling functions from data-util (and used the function from data-util-no-faux-module) so there is no confusion about which function is being called.

Co-authored-by: Hanbyul Jo <hi@hanbyul.xyz>

dzole0311 · 2024-08-28T12:16:21Z

As discussed yesterday, I removed the "Reset All" and "Default" label for now.

I see in the netlify preview that some of the datasets start with "Default," and some didn't. I assume this has to do with the logic you were talking about @dzole0311 to decide which colormap is the default. Is there any way to ensure that whatever is decided to be the default comes with this tag?

@aboydnw This and the other issue you spotted are partially related (ticketed here: #1130) and will be handled together within one fix.

hanbyul-here

🙇 Thanks for amazing work!

faustoperez · 2024-08-29T13:29:23Z

Also, another question, do we think users will want to see both Default AND the colormap name? cc @faustoperez

"Default" here refers to the colormap set by the science team. When designing this, my understanding was that they often don't name their colormaps or use readily available ones like the Viridis family, hence my use of "default" to mean the original setting.

Perhaps replacing "Default" with "Initial" would make this clearer?

Also, if we can identify whether the science team's colormap is one already listed, we could set it as the default or highlight it in the colormap panel to avoid duplicates 🤷‍♂️

## 🎉 Features - [E&A] Implement colormap configurability #1117 - [E&A] Feature flag colormap configurability #1147 - Add font md size as 20px for new Design System #1125 ## 🚀 Improvements - [E&A] Return default value of colormap along with other settings #1128 - Create new raster paint layer module and factor out BaseTimeseriesProps #1105 - Consolidate a place where to check linkProps #1121 #1160 - Expose Data Catalog and update child components to pass in routing/nav for library build #1096 #1159 - Set up eslint rule for no trailing spaces #1146 - Replace cmr-stac with titiler-cmr #1131 ## 🐛 Fixes - Update external link in top navigation target #1145 - Update PageHeader/Nav to not throw #1149 - [E&A] Fix to convert the time to usertzdate #1151 - Use sourceparams as it is when it is there #1148 - Fix compare label on block map #1153 - Discard the previous sourceExclusive value to fix the case when the datasets with different sourceExclusive can be selected together #1161 - Show the name of the selected filter on story hub #1161

dzole0311 added 24 commits August 13, 2024 18:13

Core logic for changing the color map

8bd2f18

Remove obsolete prop

7618e24

Use renderParams from STAC

b94e8df

Update LayerLegend to handle cases without steps and only with colorm…

02c8e80

…ap_name

Improve the move listener over the colormap

358eb15

Get rescale from STAC

023f11f

Add defaults

20a717c

Fix types

3836c4a

Check if the colormap should be categorical based on the sourceParams

d1eb323

Get default from config if colormap is not present in the atom

b2d24af

Fix reversed colormaps recognized as unknown

d75d9e1

Styling tweaks

8b367b4

Use USWDS class names

6595370

Few more USWDS classes

edd1938

More classes

5df8580

Safelist the USWDS icon

384bf4a

Update colormap in mdx config

5a1d5e8

Preselect colorMap

ab59663

Remove rescaling of color maps

b25d1f6

Define interpolator for Bwr

4e19296

Obsolete code

9192303

Get colorMaps from veda back-end, include other fixes

b7cba4a

Merge branch 'main' into 994-exploration-colormap

c41c543

Add reset button, add icon

c6a5338

dzole0311 requested review from hanbyul-here and sandrahoang686 August 21, 2024 08:27

dzole0311 requested review from faustoperez and aboydnw August 21, 2024 08:27

Obsolete style

1a424df

TS error

5858224

hanbyul-here requested changes Aug 22, 2024

View reviewed changes

dzole0311 added 5 commits August 23, 2024 13:08

Address few of the review comments

9c6b32a

Make ES lint happy

5bbcaf1

Adjust LoadingSkeleton height

4d27a26

Address leftover review comments

2caa0bf

Make TS happy

16922d0

dzole0311 added 3 commits August 26, 2024 11:51

Explain reasoning for the flattening of the rescale values in a comment

42bc091

Update docs for configuring colormaps in veda-ui

3f23711

Keep the rio-tiler reference

7ff3a00

hanbyul-here reviewed Aug 26, 2024

View reviewed changes

docs/content/frontmatter/layer.md Outdated Show resolved Hide resolved

aboydnw reviewed Aug 26, 2024

View reviewed changes

Return default value of colormap along with other settings

f2707b5

sandrahoang686 reviewed Aug 27, 2024

View reviewed changes

dzole0311 and others added 5 commits August 28, 2024 13:40

Remove the Reset button and the Default label, fix TS errors

2605410

Fix linting errors

585191c

Update docs/content/frontmatter/layer.md

0e55cff

Co-authored-by: Hanbyul Jo <hi@hanbyul.xyz>

Use const

60559fd

hanbyul-here approved these changes Aug 28, 2024

View reviewed changes

dzole0311 merged commit f0a4ace into main Aug 29, 2024
8 checks passed

dzole0311 deleted the 994-exploration-colormap branch August 29, 2024 06:30

sandrahoang686 mentioned this pull request Sep 23, 2024

[RELEASE] Bump minor to v5.8.0 #1167

Merged

aboydnw mentioned this pull request Sep 23, 2024

[E & A] Add range slider for rescaling the min & max range of the color maps #1102

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[E & A] Implement colormap configurability #1117

[E & A] Implement colormap configurability #1117

dzole0311 commented Aug 21, 2024 •

edited

Loading

netlify bot commented Aug 21, 2024 •

edited

Loading

faustoperez commented Aug 22, 2024

hanbyul-here left a comment

hanbyul-here Aug 22, 2024

dzole0311 Aug 26, 2024

hanbyul-here commented Aug 22, 2024

dzole0311 commented Aug 26, 2024

aboydnw left a comment

sandrahoang686 Aug 27, 2024

dzole0311 commented Aug 28, 2024

hanbyul-here left a comment

faustoperez commented Aug 29, 2024

[E & A] Implement colormap configurability #1117

[E & A] Implement colormap configurability #1117

Conversation

dzole0311 commented Aug 21, 2024 • edited Loading

Description of Changes

Notes & Questions About Changes

Validation / Testing

netlify bot commented Aug 21, 2024 • edited Loading

✅ Deploy Preview for veda-ui ready!

faustoperez commented Aug 22, 2024

hanbyul-here left a comment

Choose a reason for hiding this comment

hanbyul-here Aug 22, 2024

Choose a reason for hiding this comment

dzole0311 Aug 26, 2024

Choose a reason for hiding this comment

hanbyul-here commented Aug 22, 2024

dzole0311 commented Aug 26, 2024

aboydnw left a comment

Choose a reason for hiding this comment

sandrahoang686 Aug 27, 2024

Choose a reason for hiding this comment

dzole0311 commented Aug 28, 2024

hanbyul-here left a comment

Choose a reason for hiding this comment

faustoperez commented Aug 29, 2024

dzole0311 commented Aug 21, 2024 •

edited

Loading

netlify bot commented Aug 21, 2024 •

edited

Loading