Sentinel RasterIO Errors

[zags]

Reopening issue #53 because this has not been fixed and is an issue with rio-tiler.

Running the following:

from rio_tiler import sentinel2
import os
os.environ["CURL_CA_BUNDLE"] = "/etc/ssl/certs/ca-certificates.crt"
os.environ["AWS_REQUEST_PAYER"] = "requester"
tile, mask = sentinel2.tile("S2B_tile_20180813_23MQQ_0", 781, 1052, 11, (4, 3, 2), tilesize=256)

gives the error:

RasterioIOError: '/vsis3/sentinel-s2-l1c/tiles/23/M/QQ/2018/8/13/0/B4.jp2' does not exist in the file system, and is not recognized as a supported dataset name.

running gdalinfo on this path gives the error:

ERROR 4: /vsis3/sentinel-s2-l1c/tiles/23/M/QQ/2018/8/13/0/B4.jp2: No such file or directory
gdalinfo failed - unable to open '/vsis3/sentinel-s2-l1c/tiles/23/M/QQ/2018/8/13/0/B4.jp2'.

This path is being generated by rio-tiler. This is only happening on Sentinel; Landsat works fine.

[vincentsarago]

Hi @zags, as mentioned previously this is not a rio-tiler error, the path is generated by rasterio to be GDAL compliant /vsis3/ instead of s3://

If GDAL gives you an error it's because of misconfiguration on your side (either driver missing or missing AWS valid credentials)

[zags]

Can you test the code I posted? Dose it work for you?

[zags]

Listing the S3 bucket, it appears that "sentinel-s2-l1c/tiles/23/M/QQ/2018/8/13/0/B4.jp2" does not in fact exist. The correct URL is "sentinel-s2-l1c/tiles/23/M/QQ/2018/8/13/0/B04.jp2", though this gives other errors with RasterIO.

There is likely a bug in rio-tiler. Please don't close this until you can successfully run the code I have posted.

[zags]

Part of the problem is that bands need to be ("04", "03", "02"), but this doesn't fix everything.

The updated

from rio_tiler import sentinel2
import os
os.environ["CURL_CA_BUNDLE"] = "/etc/ssl/certs/ca-certificates.crt"
os.environ["AWS_REQUEST_PAYER"] = "requester"
tile, mask = sentinel2.tile("S2B_tile_20180813_23MQQ_0", 781, 1052, 11, ("04", "03", "02"), tilesize=256)

gives the following output:

ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0

rio_tiler/sentinel2.py in tile(sceneid, tile_x, tile_y, tile_z, bands, tilesize)
129 _tiler = partial(utils.tile_read, bounds=tile_bounds, tilesize=tilesize, nodata=0)
130 with futures.ThreadPoolExecutor(max_workers=3) as executor:
--> 131 data, masks = zip(*list(executor.map(_tiler, addresses)))
132 mask = np.all(masks, axis=0).astype(np.uint8) * 255

OSError: Read or write failed

[vincentsarago]

alright

tile, mask = sentinel2.tile("S2B_tile_20180813_23MQQ_0", 781, 1052, 11, (4, 3, 2), tilesize=256)

should be

tile, mask = sentinel2.tile("S2B_tile_20180813_23MQQ_0", 781, 1052, 11, bands=('04', '03', '02'), tilesize=256)

the docs says:
https://github.com/mapbox/rio-tiler/blob/a0dbe5675743c1bcb1ff19ef7e5135e945db9fdb/rio_tiler/sentinel2.py#L98
And this is because sentinel bands name can be like 8A. I'll try to make the documentation clearer

sorry I should have caught this error sooner.

[zags]

Thank you for looking into this. Are you seeing the "Read or write failed" error? Any thoughts as to what's causing this?

[vincentsarago]

Yes I am and I have no idea what those are 🤔

[vincentsarago]

hey @zags, I've seen the same errors elsewhere and I think it's related to gdal+openjpeg install.
Can you share your configuration ?

[zags]

This is GDAL 2.1.0.

The lib files that show up for pip install -t xxx rasterio are libgdal-48932867.so.20.3.3 and libopenjp2-00a09f32.so.2.3.0

[vincentsarago]

Alright I think we can close this one because main problem was from GDAL+configuration.
I'm going to open another issue related to the band names