Sentinel2 L1C requester pays

[samsammurphy]

Hi @vincentsarago 👋

It happened, AWS now requires a requester pays header for Sentinel2 L1C 😢 . How do we move forward with rio tiler from here?

This is the message I get when running the first example in the README

from rio_tiler import sentinel2
tile, mask = sentinel2.tile('S2A_tile_20170729_19UDP_0', 77, 89, 8)
# rasterio._err.CPLE_AWSAccessDeniedError: Access Denied

Interestingly when I run the command again I get this

# rasterio._err.CPLE_OpenFailedError: '/vsis3/sentinel-s2-l1c/tiles/19/U/DP/2017/7/29/0/preview.jp2' does not exist in the file system, and is not recognized as a supported dataset name.

which is more inline with what my (previously working) lambda sentinel2 tiler is now saying..

[vincentsarago]

Hi @samsammurphy,
To make it work you need to set AWS_REQUEST_PAYER="requester" environment.
I could add a

with rasterio.Env(AWS_REQUEST_PAYER="requester") as env:

but I'm not sure it's the best way to do

[samsammurphy]

Howdy @vincentsarago

I understand the concern that rio tiler uses will be charged by AWS to access L1C now. It has to be really clear to the users (i.e. to avoids nasty surprises).

From the front end maybe this could be a one time configuration setting

from rio_tiler import sentinel2
sentinel2.aws_requester_payer = 'requester'

This could have the effect setting the AWS_REQUEST_PAYER environment variable before running rio tiler, maybe through rasterio as you mentioned, or perhaps as subprocess command? The latter, however, might not work on all machines (?) and would require python2 vs 3 handling, here's what it might look like anyway..

import subprocess
subprocess.run(['export', 'AWS_REQUEST_PAYER="requester"']) 

[vincentsarago]

FYI
There was a ticket couples years ago about the same subject on rasterio: rasterio/rasterio#692

[samsammurphy]

Cheers!

Mojodna suggests a way of setting the environment variable which would handle the reservation I had with my previous subprocess-based suggestion, namely:

os.environ["AWS_REQUEST_PAYER"] = "requester"

edit
p.s. he suggests this instead of using rasterio.Env.

[vincentsarago]

@samsammurphy I'm going to close this, except if it's still a blocker!