🚸 Walkthrough on rasterizing vector polygons into label masks #31
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Initial draft tutorial on preparing vector labels for an image segmentation task. Using shapefiles of flood water extent digitized by UNITAR-UNOSAT over Johor, Malaysia on Sentinel-1 imagery taken 15 Dec 2019.
Fix readthedocs build failure because pyogrio was not installed.
New datashader DataPipe classes to use!
Picking a single Sentinel-1 image from 15 Dec 2019 over Johor, Malaysia corresponding to the mapped flood extent polygons. Perform reprojection from EPSG;4326 to UTM, and transform VV channel from linear to decibel. Did a bit of reorganizing to put the raster section before the vector section, may decide to change it later when the canvas part comes in.
Fix readthedocs `ERROR: Couldn't find cache key for notebook file vector-segmentation-labels.md`. Signed-off-by: Wei Ji <23487320+weiji14@users.noreply.github.com>
A comprehensive library for creating static, animated, and interactive visualizations in Python!
weiji14
force-pushed
the
vector-segmentation-labels
branch
from
e0f9a83 to
ebf26fd
Compare
Point to https://docs.xarray.dev/en/latest/ instead.
Debugging why readthedocs build fails when reproducing the same environment and commands locally works.
See if this helps with fixing the KeyboardInterrupt when plotting the Sentinel-1 map using matplotlib, hopefully 5 minutes is more than enough.
Make it easier to debug errors when builds fail on Readthedocs, see https://jupyterbook.org/en/latest/content/execute.html#execution-tracebacks-in-the-terminal. Also reverting 1313d38 and ada85a8.
Read in the two shapefiles using PyogrioReaderIterDataPipe, and show a quick example of reprojecting and visualizing the vector polygons.
Using template Sentinel-1 xarray.DataArray grid to create blank datashader.Canvas. Show how the metadata information match! Also some small edits to the previous pyogrio section like fixing an incorrectly set projection system.
Show how the painting of vector geopandas.GeoDataFrame polygons onto the datashader.Canvas works! Got some words of caution and where to seek more advice. Also added spatialpandas to the 'docs' extras list of dependencies.
Simple Python interface for Graphviz!
Marie Kondo-ing the section on pre-processing the vector data, because working with two shapefiles was a pain. The merging of two GeoDataFrame objects using BatchMapper was a bad idea as it returned a DataPipe without a length, so DatashaderRasterizer complained. Decided too that focusing on just one vector was enough to teach the concept, and the masters can read the footnotes on more advanced combinatorial pipelines.
Combining two xarray.DataArray objects into a single xarray.Dataset before slicing. Using Zipper to pair the two DataPipes, and then Collator to do the actual combining. Showing the DataPipe graph after the zip stage to give people an overview of the (many) steps done so far. Double checked too that the image and mask looks ok, which led to the discovery of an issue with rounding numbers or something. Had to change the resolution from 80m to 100m because of this, otherwise the rasterized water mask is cut off at the Southern part.
Fixes `ExecutableNotFound: failed to execute PosixPath('dot'), make sure the Graphviz executables are on your systems' PATH`. Decided to use apt instead of conda because it's less hassle.
Just complete the whole vector segmentation tutorial in one go! Really simplified the ending with a minimal xbatcher slicing and conversion to torch.Tensor step only. Gave a shoutout to Edoardo's work at UNOSAT in the end. Decided to rename the file to vector-segmentation-masks. Added more emojis throughout the page, and should be good to go!
weiji14
added a commit
that referenced
this pull request
Oct 2, 2022
Refactoring the xarray collate functions to use `xr.merge` instead of the dictionary style way of appending data variables to an xarray.Dataset. Solution adapted from 7787f8e in #62 that is more robust to images being cut off due to rounding issues as with 6b18934 in #31. Downside is the need to verbosely rename the xarray.DataArray objects, and handle some conflicting coordinate labels.
weiji14
added a commit
that referenced
this pull request
Oct 2, 2022
* ♻️ Use xarray.merge with join="override" in collate functions Refactoring the xarray collate functions to use `xr.merge` instead of the dictionary style way of appending data variables to an xarray.Dataset. Solution adapted from 7787f8e in #62 that is more robust to images being cut off due to rounding issues as with 6b18934 in #31. Downside is the need to verbosely rename the xarray.DataArray objects, and handle some conflicting coordinate labels. * 📝 Minor tweaks to vector segmentation mask walkthrough A few whitespace fixes and fixing some DataPipe references.
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tutorial on creating vector label masks for an image segmentation task. Using flood water extent polygons digitized by UNITAR-UNOSAT over Johor, Malaysia on Sentinel-1 imagery taken 15 Dec 2019.
Preview at https://zen3geo--31.org.readthedocs.build/en/31/vector-segmentation-masks.html
TODO:
datashader.Canvasfrom templatexarray.DataArrayinputReferences: