Geotrellis 3.x #398
Merged
Geotrellis 3.x #398
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Depends on locationtech/geotrellis#3134 |
|
@metasim you can depend on GeoTrellis 3.1.0 and use GeoTrellis projected like now. |
|
@pomadchin Got pulled onto other work.... definitely near the top of my RF todo list. |
* develop: PR feedback. PR feedback. Regression fix. Release notes update. Added forced truncation of WKT types in Markdown/HTML rendering. Ensure default tile size is applied to `raster` reader. Fix nodata doc Doc supervised, set tile size to 256 for visual Update doc to use rf_local_is_in when masking; fix locationtech#351 Close locationtech#310 move reference to static rf_local_is_in python implementation Updated intro section. Added additional raster-read section. Applying pre-partitioning to DataSources. Expanded RasterRefSpec to ensure lazy tiles provide metadata without I/O. Fix unit tests for rf_local_is_in Attempting to keep TravisCI from timing out by using jobs. Add rf_local_is_in function
|
@pomadchin GT 3.1.0 working great! |
* develop: Masking docs: show preview of clipped df Mark final code chunk as executable in masking page Masking docs: show preview of clipped df Attempt to fix "ImportError: libpoppler.so.76: cannot open shared object file: No such file or directory" Attempt to fix "ImportError: libpoppler.so.76: cannot open shared object file: No such file or directory" Mark final code chunk as executable in masking page Bumped to next dev version. 0.8.4 release prep. Misc documentation fixes. Fixes to rf_mask_by_values to compose over IfIn and have SQL docs. Removed inverse flag from rf_mask_by_values before committing to an approach with it. Attempt to create Expression for MaskByValues to enable sql api Fix test for rf_mask_by_values and add SQL test Expand rf_mask_by_value[s] signatures to match python api, expand unit tests Update documentation on masking Update mask functions with inverse arg, add rf_mask_by_values function Add Scala overload taking a literal Array in rf_local_is_in # Conflicts: # pyrasterframes/src/main/python/pyrasterframes/version.py # version.sbt
to verify behavior, and filed locationtech/geotrellis#3153.
* develop: (24 commits) remove unused imports Python unit tests check read pipeline stages Cruft removal. Fix for ML transformer read/write Add failing unit tests for issue 425 ML custom transformer loading is broken PR feedback. Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420. break out commented assert into skipped unit test around masking and deserialization register rf_local_extract_bit with SQL functions Added the ability to do a raster_join on proj_raster types. Fixes locationtech#419. Add landsat masking section to masking docs page Add mask bits python api and unit test Extract bits should throw on non-integral cell types Fix for both masking by def and value; expand code comments; update tests Python regression. Masking improvements and unit tests. Regression Added (disabled) integration test for profiling spatial index effects. Reorganized non-evaluated documentationm files to `docs`. Add failing unit test for mask by value on 0 Updated python tests against spatial index functions. ... # Conflicts: # core/src/main/scala/org/locationtech/rasterframes/expressions/DynamicExtractors.scala # core/src/main/scala/org/locationtech/rasterframes/expressions/transformers/XZ2Indexer.scala # core/src/main/scala/org/locationtech/rasterframes/extensions/Implicits.scala # core/src/test/scala/examples/CreatingRasterFrames.scala # core/src/test/scala/org/locationtech/rasterframes/RasterLayerSpec.scala # core/src/test/scala/org/locationtech/rasterframes/expressions/XZ2IndexerSpec.scala # datasource/src/main/scala/org/locationtech/rasterframes/datasource/raster/RasterSourceRelation.scala # datasource/src/test/scala/org/locationtech/rasterframes/datasource/raster/RasterSourceDataSourceSpec.scala
Signed-off-by: Jason T. Brown <jason@astraea.earth>
Formally adopting Python 3 / abandon Python 2
* develop: (41 commits) Bumped dev version. Release prep. Added `fix/docs.*` to docs build triggers. Atttempted fix for failing pip install in docs CI job. Fix mistakes in merge with develop Fixes to scala classification model example Reconstituted two forms of the mini classification example in Scala. PR feedback Close locationtech#228 and locationtech#331 with discussion on writing page; clean up unsupervised learning output. Various PR improvements Update Python bindings and test Remove incorrect Expression and SQL registry Add documentation, python bindings for rf_agg_approx_quantiles Fix column selection Refine CRS WKT parsing error message and test Update tests to prefer use of rf_agg_approx_quantiles implementation Revert "Add DataFrame tileStats extension" Remove incorrect whitespace before triple backtick Filter sample raster join DF to show non-empty tiles Initial raster join page; expand spatial index discussion ... # Conflicts: # core/src/main/scala/org/locationtech/rasterframes/RasterFunctions.scala # core/src/main/scala/org/locationtech/rasterframes/encoders/StandardSerializers.scala # core/src/main/scala/org/locationtech/rasterframes/expressions/aggregates/TileRasterizerAggregate.scala # core/src/main/scala/org/locationtech/rasterframes/extensions/Implicits.scala # core/src/main/scala/org/locationtech/rasterframes/model/LazyCRS.scala # core/src/main/scala/org/locationtech/rasterframes/util/package.scala # core/src/test/scala/org/locationtech/rasterframes/RasterFunctionsSpec.scala # core/src/test/scala/org/locationtech/rasterframes/RasterJoinSpec.scala # core/src/test/scala/org/locationtech/rasterframes/RasterLayerSpec.scala # core/src/test/scala/org/locationtech/rasterframes/TestData.scala # project/RFDependenciesPlugin.scala # pyrasterframes/src/main/python/pyrasterframes/rasterfunctions.py # pyrasterframes/src/main/python/pyrasterframes/rf_context.py # pyrasterframes/src/main/python/pyrasterframes/rf_ipython.py # pyrasterframes/src/main/python/pyrasterframes/rf_types.py # pyrasterframes/src/main/python/pyrasterframes/version.py # pyrasterframes/src/main/python/tests/RasterFunctionsTests.py # version.sbt
Regressions.
Signed-off-by: Jason T. Brown <jason@astraea.earth>
* develop: Fix locationtech#452. Upgrade rasterframes-notebook container version of rtree. Add map display to raster-read docs with tile and extent
…workaround to locationtech/geotrelli#3184.
* develop: Refactor clip to clamp Use rf_local_clip and other viz functions in supervised learning doc page rf_rescale tests and refinements Add Rescale function to Scala, Python and SQL APIs Add rf_where and rf_standardize functions Add rf_local_min, rf_local_max, and rf_local_clip functions # Conflicts: # pyrasterframes/src/main/python/pyrasterframes/rasterfunctions.py
* develop: Cherry picked: de8464b Expand raster_join test case with null value in left hand tile column Updated raster_join test to filter NA rows. Typo. reproject_and_merge returns null if LHS dimension, extent or crs are null. Additional fixes to type handling of CRS and Extent from Python. Added ability to specify Extent with arbitrarily ordered fields in Python. Fixed misnamed field in Envelope serializer. Add failing unit test for issue 462 Added support for Extents defined with bigint in Python. Failing unit test for st_geometry extents bug Skip failing test for 458 Add failing test for issue 458; failing to render null values in tile column # Conflicts: # build.sbt # core/src/main/scala/org/locationtech/rasterframes/expressions/DynamicExtractors.scala # core/src/main/scala/org/locationtech/rasterframes/expressions/accessors/GetCRS.scala # core/src/main/scala/org/locationtech/rasterframes/expressions/transformers/ExtentToGeometry.scala # pyrasterframes/src/main/python/pyrasterframes/rasterfunctions.py # pyrasterframes/src/main/python/pyrasterframes/rf_types.py
Signed-off-by: Jean-Denis Giguère <jean-denis@anagraph.io> (cherry picked from commit 380e40b)
* develop: rf_tile_to_array_int returns Int, not double. Fix locationtech#473 Disabled column type checking in RasterJoin. Fix for propogating TileLayerMetadata tile dimensions to layer creation. Fix for pyspark AttributeError on sparksession _conf member Regressions. # Conflicts: # core/src/main/scala/org/locationtech/rasterframes/extensions/RasterJoin.scala # core/src/main/scala/org/locationtech/rasterframes/extensions/ReprojectToLayer.scala
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Geotrellis 3.x Upgrade Log
Inttype parameter toGridInttype parameter toCellGridInttype parameter toGridBounds... orTileBoundsGridBounds.toGridTypeto coerce fromInttoLongtype parametergeotrellis.spark.tilingtogeotrellis.layergeotrellis.spark.iotogeotrellis.spark.store...FixedRasterExtentorg.locationtech.rasterframes.util.ShimsExtent.jtsGeomtoExtent.toPolygonTileLayerMetadata.gridBoundstoTileLayerMetadata.tileBoundsgeotrellis-gdaldependencygeotrellis.vectorgeometryorg.locationtech.rasterframes.encoders.StandardEncoders.crsEncodertocrsSparkEncoder(cols, rows)dimension destructuring toDimensions(cols, rows)Tileequality since it's more strictreference.confto usegeotrellis.raster.gdalnamespace.TileDimensionswithgeotrellis.raster.Dimensions[Int].TODO
FixedDelegatingTileobsolete. (cfcombinewhen implementing aDelegatingTileincorrectly handlesNoDatageotrellis#3153)TileDimensionwithgeotrellis.raster.DimensionsLocalRasterSourceupdated to useCellGrid[Long]ReplaceSimpleRasterInfowithGeoTiffInfo(where possible). RevisitRasterExtentvsGridExtent[Long]Replace allFixed*classes.RemoveRasterSourcealtogether.Replacespray-jsonwithcirce.Remaining tasks carried over to #476.