Skip to content

Latest commit

 

History

History
278 lines (261 loc) · 17 KB

TODO.md

File metadata and controls

278 lines (261 loc) · 17 KB
Raw

TODO

Overview

  • switch data source from GeoMAC to NIFC
  • 2020 data (? - or wait until available as historical data?)
  • data loading strategy
  • design
  • deploy, get feedback
  • narrative
  • polish

Application

geojson loading strategy

  • refine perimeter simplification
    • See simplifyLowRes and acresLowRes in NIFC-fetcher
    • Filter out < 5000 acre fires from low-res
    • remove -low prefix from perimeter filenames in use-all-fires / use-complete-fires hooks
  • even some of these are heavy, simplify geometry more note: increased simplification once, from: .domain([10, 500]).range([30, 3]).exponent(0.25) to: .domain([5, 350]).range([20, 2]).exponent(0.15) but still end up with very heavy (10+MB) merged files. may need to consider other strategies...
  • load low-res at low zoom (on init?), high res when zooming in
  • 2020 data After 2019, Historic_Geomac_Perimeters_ no longer exists. Need to switch, possibly to National Incident Feature Service (https://data-nifc.opendata.arcgis.com/datasets/national-incident-feature-service-archive-2020). How to query by state/year/etc? Couldn't find anything available on NIFC's ArcGIS server. Sent email to BLM_FA_NIFC_Comments@blm.gov Feb 28; if no response, try social media https://twitter.com/NIFC_Fire
  • load (serially?) in the background after app init NOTE: use-complete-fires already does this, only it loads backwards from selected date...didn't i write code once that loaded both forward and backward?
    • TODO NEXT: do this for low-res perimeters
    • load hi-res perimeters in the background on load
  • ohhh wait, each fire folder has multiple perimeters per fire. don't have that in merged fires :/ looks like we are going to have to load all geojsons for each fire, tho we only need to load those near the currentDate...
  • rather than load many files, just load one / one merged per year and set visibility of each feature based on its datestamp 🤦‍♀️
  • however, have to consider bundle size, really want to lazy load...
  • yeah ok, so load each year as single merged file, on-demand
  • debug perimeter display over time
    • load simplified perimeters to improve perf during testing
    • continue with debugging in use-complete-fires and map.js
      • looks like all-fires-for-year is working more or less correctly, only issue is complete-fires not rendering ... Looks request cache is getting popped on every hook call...??
    • restore previous perimeters once debugging complete
  • FEB 2021 DEBUGGING DUPLICATES final perimeters should now be deduped properly, for both H3 and non-. However, there are still duplicate perimeters (same id + name, different geometry / acreage) in allPerimeters, e.g. 2000-CA-014532/Ranch in Year 2000.
    • still seeing same problem when zooming in on 2003: "Unknown GeoJSON type Feature"...just need to run reprocess-all-fires.
    • Found some duplicate features (not H3-specific). e.g.:
      • Morgan, 2000 (all filtered out at low-res) Looks like two totally different fires, with the same name. One is in San Gabriel hills N of LA; the other is close to Clear Lake. Maybe what we need to do here is check for uniquefireidentifier dupes? > Yup, two fires, same name (Morgan), different IDs. Deduping w/ dedupeFinalPerimeters works. DEBUGGING -- FEB 15 PROGRESS -- In the process of abandoning H3; not worth trying to overcome the perf issues as I'd prefer to go with polygons / geojson. Still have to work through data quality issues and tooltip displaying incorrect fire; refactored map.js to DRY out the three layers (prior, previous, current) and to keep H3 around for debugging.
    -- FEB 28 PROGRESS --
    • not sure if tooltip issues yet resolved: Zooming in to LA at end of 2006 results in all fires labeled "J Black" from 2001...not sure wassup. Looks like only priorYears H3HexLayer is accepting mouse input, it's blocking mouse interaction with layers below. Something about H3 layers? Or may be specifically the J Black fire has some bad geometry...?
    • high-res geojson for previousYearRequest.data contains one malformed feature, at index 65: "Ranch" fire has extra level of nesting, Feature instead of Polygon...
      • Looks like I'm flattening dupes incorrectly. I think Ranch is a dupe in 2000 data, but deduping is resulting in a malformed feature.
  • consider hexes at low zoom, polygons at high zoom not sure i want to roll with hexes, but doing this seems to be revealing some issues with data. going to do as much work here as is useful to uncover + fix data issues, then make call on whether to hex or not to hex.
    • Make feature properties available to tooltips?
      • [-] Using H3HexagonLayer causes weirdness with tooltips: hoverInfo is often incorrect, e.g. showing a very small fire (Salt, 2003) instead of the large one hovered on (Simi Incident).
      • [-] However, using H3ClusterLayer does work well with tooltips, but performance is so bad that it causes Canary to crash, and Firefox barely performs. Is there some memoization I need to do?
        • [-] Seems related to bumping deck.gl to 8.4.x...reverted to 8.0.x and appears better. However, tooltip hover perf is really bad (it's bad with HexagonLayer, but even worse with ClusterLayer). Why?? --> It is directly related to number of hexagons -- once zoomed in, perf is substantially worse. Have to abandon H3.

fires layer

  • perf: handle incremental loads with one layer / data payload https://deck.gl/#/documentation/developer-guide/performance-optimization?section=use-updatetriggers#handle-incremental-data-loading
  • only render most recent perimeter for each fire
  • ^^ not sure about best perf strategy; may need to do filtering in getFillColor instead of passing new array every time (generated in extractLatestPerimeters)...
  • memoize properly, not using useState, per comments in map.js
  • check for near-duplicate perimeters. examples:
    • Ranch Fire (2018): two perimeters (ids 'Ranch' and 'RANCH'): - Aug 08 - 17 (329817 acres) - Aug 26 - Sept 23 (410202 acres)
    • Thomas Fire (2017-8) - end date Dec 27, 2017 - end date Jan 06, 2018
  • look into Date comparison optimizations in - map.js::extractLatestPerimeters - use-all-fires-for-year-request.js::sortPerimetersByDate
  • handle requests for years beyond the last gracefully: don't blank out whole fire layer because of null request

slider

  • slider refinement
    • tooltip appears twice / flickers when rolling over tick label
    • tooltip doesn't appear at hover location on bar graph (is this a regression? noticed before adding gradient/annual label bkgd)
    • too difficult to discern summer / winter in timeline. add subtle background gradient
    • slight misalignment between bar chart + underlay and slider (zoom in on start of slider to see this, note also gradient stops not quite aligned with year ticks)
  • align bars w/ slider Problem is within Nivo -- bar/common/getIndexedScale uses d3.scaleBand.rangeRound(). Have local changes to my Nivo fork to add a nice prop to <Bar>, and have verified in fire-map that passing nice=false with that new feature works. So, next steps: - [X] Merge nivo upstream into my fork - [X] Issue PR with these local changes plouc/nivo#1282 - [X] Are there any unit test changes required? - [X] Point fire-map at my nivo fork until PR is merged Ugghhhh...can't figure out how to get @nivo/bar to work with my fork :/ something about it being a scope package / subfolder. May require a postinstall (?) build step to build node_modules/nivo/packages/bar: - cd ./node_modules/nivo && make init; make package-build-bar ...Just stopped work here until now, when my PR has been released (0.66.0)
  • verify data against CALFIRE (e.g. https://www.fire.ca.gov/incidents/2018/)
    • now using final perimeters to determine acres / month; a better solution would be to check last month's perimeters against this month and subtract to calculate differential per perimeter across month boundaries.
  • tooltip: use 🔥 emoji to indicate relative amount of fire, either with number of chars (log scale? one fire per digit) or font size
  • label with notable fires (megafires (> 100k acres), biggest per year, etc)
    • link labels to zoom map + slider to fire (?)
  • consider coloring segments of slider by year depending if data loaded for that year (like buffering/progress bar)
  • debounce according to distance moved; longer lag for fast motion to make lazy-loading work better
  • add bar chart of monthly acres burned
    • how to calculate?
    • write script to pre-calculate and write to a file; load on init
    • 2019 not calculated correctly...same issue in other years?
    • NEXT: fix bar chart / slider interaction:
      • create interaction layer / hit area that sits over both bar chart and slider. this layer is a second nivo chart, with all bars full-height but transparent, and handles interaction events.
        • fix memoization / metadata selector not ref equal, causing tooltip remounting
        • remove debouncing?
        • tooltips use nivo's tooltip handler, but render tooltips in fixed location (per-bar), below slider track display: - month, year - num acres burned
        • interaction handlers hover is handled by nivo; have to add handling for mouseDown/Move/Up to set currentDate.
        • delete slider-tooltip.js and any other cruft
    • just like slider (track / rail), make bars to left of current date solid, bars to right translucent
    • refine slider value tooltip styling
    • finish tooltip / date formatting
      • slider hit area blocks bottom part of bar chart
      • slice-style tooltip hover; don't require hovering directly on bar NOTE: may want to decouple tooltip interaction from nivo, just implement manually

intro / narrative

  • dedication to firefighters
  • instructions
    • press play, sit back and watch (callouts appear at notable moments)
    • explore: scrub manually (callouts appear only over slider)
  • zoom to first megafire in dataset and add callout

design

  • light / dark?
  • mobile layout?
  • fonts
    • title
    • UI / vis labels
    • tooltips, callouts for fires

map

  • basemap
    • dark, w/ terrain?
    • developed area / urban areas
  • hex aggregation
    • reduce perimeters to h3 hexes at low zoom, for perf?
  • [-] Animate viewport from flat/top-down view to oblique view; oblique view (w/ pitch + bearing) is too foreign a perspective to start with

map interactions

  • bug: if fire disappears while hovered, tooltip is not dismissed until next hovered fire

other / ideas

  • encode polygon height to:
    • number of times an area has burned?
    • days-length of fire?
    • Elevation of each starts high, then dies down after fire is contained, like a burning fire. Could even fade from red to grey (ash), tho this won’t work well for additive blending...or maybe it will? Existing greys should push red into...hm, maybe pink. Maybe instead of grey, a greyish yellow?
  • additive blending to highlight overlapping perimeters? Invert basemap colors (maintain terrain on dark tiles?) to enable additive blending? Stack fires w/additive (once showing only most recent perimeter for each fire)
  • is 2019 data available yet?
  • search for fire by name (a la LA Times map)
  • differential bundling for older browser support https://v2.parceljs.org/getting-started/webapp/#differential-serving
  • rain (data layer, not likely a geo layer) diverging bar chart, showing deviation from norm (sim concept to SST)
  • nasa satellite hotspot detection https://earthdata.nasa.gov/earth-observation-data/near-real-time/firms/active-fire-data#tab-content-6

Data

Bugs

  • title + slider disappear while LoadingIcon visible
  • 2020 data not scraping correctly
    • bar chart doesn't align with slider
  • Can we remove extractLatestPerimeters and use preprocessed finalPerimeters instead?

RFP / DONE

  • Finish data loading strategy for all years / years before current
  • Maybe go back and refactor scraper/merger accordingly
    • compress all perimeters for each fire into a single FeatureCollection with each perimeter as a separate Feature
    • consider compressing all perimeters for year into a single FeatureCollection in order to load only one file / year
    • simplify geometry even more...
  • Do not display geojson features with datestamps > currentDate Use getFillColor/LineColor, or some other GeoJsonLayer, as a callback? e.g. getFillColor={feature => feature.datestamp > foo...}
    • Display all fires up to currentDate, not just those for current year
  • RFP
    • publish site to gh-pages / transmote
    • refine prototype
      • enable basic picking + tooltips
      • add playback controls implement in a way that a segment can be cued + played to tell stories, e.g. encircling of Ojai by Thomas Fire - start time - stop time - playback speed - metadata (title, desc, etc)
      • rotate map to oblique view with north to left, south to right
      • load all perimeters, not just last, to show fires growing
    • write
      • inspiration / reason
      • narrative
      • current prototype
      • aspirations
        • features:
          • other datasets
          • zoom to fires (create GIFs for RFP)
            • ojai/thomas
            • paradise/??
            • sonoma/?? - 2017. 2019
          • ...?
        • technical:
          • improve loading
          • reduce bundle size (5.9MB???)
          • all open-source, including data scraper
          • deployment target: desktop-first, but could possibly be adapted to mobile
            • load less data / omit smaller fires
            • focus more on highlighting individual fires and less on open-ended map exploration
    • send to cfp@parametric.press, Subject: [Pitch] The Fires Next Time: California Wildfires Map