Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

toLayer is broken #357

Closed
metasim opened this issue Sep 19, 2019 · 0 comments
Closed

toLayer is broken #357

metasim opened this issue Sep 19, 2019 · 0 comments
Labels
bug When it really isn't a "feature".

Comments

@metasim
Copy link
Member

metasim commented Sep 19, 2019

Code

val spark = new SparkSession.Builder()
  .master("local[*]")
  .withKryoSerialization
  .getOrCreate()
  .withRasterFrames
import spark.implicits._

val df = spark.read.raster.load("https://raw.githubusercontent.com/locationtech/rasterframes/develop/core/src/test/resources/LC08_RGB_Norfolk_COG.tiff")
  .select(rf_tile($"proj_raster") as "tile", rf_crs($"proj_raster") as "crs", rf_extent($"proj_raster") as "extent")

val crs = CRS.fromString("+proj=utm +zone=18 +datum=WGS84 +units=m +no_defs")

val extent = Extent(364455.0, 4080315.0, 395295.0, 4109985.0)
val layout = LayoutDefinition(extent, TileLayout(2, 2, 32, 32))

val tlm =  new TileLayerMetadata[SpatialKey](
  UByteConstantNoDataCellType,
  layout,
  extent,
  crs,
  KeyBounds(SpatialKey(0, 0), SpatialKey(1, 1))
)
val layer = df.toLayer(tlm)
layer.printSchema()

import org.locationtech.rasterframes.datasource.geotrellis._
val layerDest = Files.createTempFile("gt-", "-layer")
layer.write.geotrellis.asLayer(Layer(layerDest.toUri, "test", 0))

layer.write.geotiff.save("former-layer.tif")

spark.stop()

Result

Exception in thread "main" java.lang.IllegalArgumentException: Need at least one target tile on LHS
	at org.locationtech.rasterframes.extensions.RasterJoin$$anonfun$5.apply(RasterJoin.scala:77)
	at org.locationtech.rasterframes.extensions.RasterJoin$$anonfun$5.apply(RasterJoin.scala:77)
	at scala.Option.getOrElse(Option.scala:121)
	at org.locationtech.rasterframes.extensions.RasterJoin$.apply(RasterJoin.scala:77)
	at org.locationtech.rasterframes.extensions.RasterJoin$.apply(RasterJoin.scala:42)
	at org.locationtech.rasterframes.extensions.RasterJoin$.apply(RasterJoin.scala:34)
	at org.locationtech.rasterframes.extensions.ReprojectToLayer$.apply(ReprojectToLayer.scala:47)
	at org.locationtech.rasterframes.extensions.DataFrameMethods$class.toLayer(DataFrameMethods.scala:211)
	at org.locationtech.rasterframes.extensions.Implicits$WithDataFrameMethods.toLayer(Implicits.scala:57)
	at org.locationtech.rasterframes.datasource.geotrellis.LayerToGeoTiff$.main(LayerToGeoTiff.scala:61)
	at org.locationtech.rasterframes.datasource.geotrellis.LayerToGeoTiff.main(LayerToGeoTiff.scala)
@metasim metasim added the bug When it really isn't a "feature". label Sep 19, 2019
metasim added a commit to s22s/rasterframes that referenced this issue Nov 19, 2019
@metasim
Merge branch 'develop' into astraea/release
9a24f08 
* develop: (79 commits)
  remove unused imports
  Python unit tests check read pipeline stages
  Cruft removal.
  Fix for ML transformer read/write
  Add failing unit tests for issue 425 ML custom transformer loading is broken
  PR feedback.
  Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420.
  break out commented assert into skipped unit test around masking and deserialization
  register rf_local_extract_bit with SQL functions
  Added the ability to do a raster_join on proj_raster types. Fixes locationtech#419.
  Add landsat masking section to masking docs page
  Add mask bits  python api and unit test
  Extract bits should throw on non-integral cell types
  Fix for both masking by def and value; expand code comments; update tests
  Python regression.
  Masking improvements and unit tests.
  Regression
  Added (disabled) integration test for profiling spatial index effects. Reorganized non-evaluated documentationm files to `docs`.
  Add failing unit test for mask by value on 0
  Updated python tests against spatial index functions.
  ...
metasim added a commit to s22s/rasterframes that referenced this issue Nov 22, 2019
@metasim
Merge branch 'develop' into feature/gt-3.0
d223b82 
* develop: (24 commits)
  remove unused imports
  Python unit tests check read pipeline stages
  Cruft removal.
  Fix for ML transformer read/write
  Add failing unit tests for issue 425 ML custom transformer loading is broken
  PR feedback.
  Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420.
  break out commented assert into skipped unit test around masking and deserialization
  register rf_local_extract_bit with SQL functions
  Added the ability to do a raster_join on proj_raster types. Fixes locationtech#419.
  Add landsat masking section to masking docs page
  Add mask bits  python api and unit test
  Extract bits should throw on non-integral cell types
  Fix for both masking by def and value; expand code comments; update tests
  Python regression.
  Masking improvements and unit tests.
  Regression
  Added (disabled) integration test for profiling spatial index effects. Reorganized non-evaluated documentationm files to `docs`.
  Add failing unit test for mask by value on 0
  Updated python tests against spatial index functions.
  ...

# Conflicts:
#	core/src/main/scala/org/locationtech/rasterframes/expressions/DynamicExtractors.scala
#	core/src/main/scala/org/locationtech/rasterframes/expressions/transformers/XZ2Indexer.scala
#	core/src/main/scala/org/locationtech/rasterframes/extensions/Implicits.scala
#	core/src/test/scala/examples/CreatingRasterFrames.scala
#	core/src/test/scala/org/locationtech/rasterframes/RasterLayerSpec.scala
#	core/src/test/scala/org/locationtech/rasterframes/expressions/XZ2IndexerSpec.scala
#	datasource/src/main/scala/org/locationtech/rasterframes/datasource/raster/RasterSourceRelation.scala
#	datasource/src/test/scala/org/locationtech/rasterframes/datasource/raster/RasterSourceDataSourceSpec.scala
metasim added a commit to s22s/rasterframes that referenced this issue Nov 26, 2019
@metasim
Merge branch 'develop' into feature/agg-overview
c6c7fbb 
* develop:
  Fix rf_local_extract_bits argument parsing
  remove unused imports
  Python unit tests check read pipeline stages
  Cruft removal.
  Fix for ML transformer read/write
  Add failing unit tests for issue 425 ML custom transformer loading is broken
  Fixed `ReprojectToLayer` to work with `proj_raster`. Closes locationtech#357 Depends on locationtech#420.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug When it really isn't a "feature".
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@metasim