Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit filetype options for tar_terra_sds() and tar_terra_sprc() #95

Open
Aariq opened this issue Sep 11, 2024 · 4 comments
Open

Limit filetype options for tar_terra_sds() and tar_terra_sprc() #95

Aariq opened this issue Sep 11, 2024 · 4 comments
Assignees

Comments

@Aariq
Copy link
Collaborator

Aariq commented Sep 11, 2024

It would be worthwhile testing with a driver that does not support sub-datasets first to see if the error message is informative—perhaps if it is we don't actually need to do anything

APPEND_SUBDATASET=YES works for GTiff output of both SpatRasterDataset or SpatRasterCollection objects, but other formats will not necessarily work this way. Perhaps these methods should be limited to filetype="GTiff"?

From https://gdal.org/user/raster_data_model.html#subdatasets-domain

Currently, drivers which support subdatasets are: ADRG, ECRGTOC, GEORASTER, GTiff, HDF4, HDF5, netCDF, NITF, NTv2, OGDI, PDF, PostGISRaster, Rasterlite, RPFTOC, RS2, TileDB, WCS, and WMS.

Originally posted by @brownag in #59

@Aariq Aariq self-assigned this Oct 2, 2024
@Aariq
Copy link
Collaborator Author

Aariq commented Oct 2, 2024

Error message is:

── Last error message ────────────────────
    _store_ [writeRaster] cannot append datasets with this file format

That seems explicit enough to not worry about adding guard rails that we'd have to maintain as new drivers get added. If you agree @njtierney, feel free to close this issue.

@njtierney
Copy link
Owner

Can you provide a bit more context around the code that produces this error? Or is it linked in the issue above?

@Aariq
Copy link
Collaborator Author

Aariq commented Oct 2, 2024

Sure, here's an example that produces this error:

targets::tar_dir({ # tar_dir() runs code from a temporary directory.
    targets::tar_script({
    library(geotargets)
        elev_scale <- function(z = 1, projection = "EPSG:4326") {
            terra::project(
                terra::rast(system.file("ex", "elev.tif", package = "terra")) * z,
                projection
            )
        }
        list(
            tar_terra_sprc(
                raster_elevs,
                command = terra::sprc(list(
                    elev_scale(1),
                    elev_scale(2, "+proj=igh")
                )),
                filetype = "MFF2"
            )
        )
    })
    targets::tar_make()
})

The errors don't show up the same when using the reprex package for some reason, but here's what they look like with targets 1.8.0:

▶ dispatched target raster_elevs
✖ errored target raster_elevs
✖ errored pipeline [0.317 seconds]
Error:
! targets::tar_make() error

── Debug target raster_elevs ───────────────────────────────────────────────────────────────────────────────────
    tar_meta(raster_elevs)$error
    tar_workspace(raster_elevs)

── General debugging ───────────────────────────────────────────────────────────────────────────────────────────
    • tar_errored()
    • tar_meta(fields = any_of("error"), complete_only = TRUE)
    • tar_workspace()
    • tar_workspaces()

── How to ──────────────────────────────────────────────────────────────────────────────────────────────────────
    • Debug: https://books.ropensci.org/targets/debugging.html
    • Help: https://books.ropensci.org/targets/help.html

── Last error message ──────────────────────────────────────────────────────────────────────────────────────────
    _store_ [writeRaster] cannot append datasets with this file format

── Last error traceback ────────────────────────────────────────────────────────────────────────────────────────
    No traceback available.

@Aariq
Copy link
Collaborator Author

Aariq commented Oct 11, 2024

Pros of doing our own checking:

  • Opportunity to write a better/different error message
  • Fail faster, before command is run

Cons:

  • Need to maintain our own list of GDAL drivers and versions that support APPEND_SUBDATASET=YES

After seeing your changes in #97 and realizing that APPEND_SUBDATSET requires GDAL ≥ 3.0, I'm more convinced that we should intervene here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants