Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get and open should also work with plain links #43

Closed
betolink opened this issue Nov 30, 2021 · 3 comments · Fixed by #119
Closed

get and open should also work with plain links #43

betolink opened this issue Nov 30, 2021 · 3 comments · Fixed by #119

Comments

@betolink
Copy link
Member

right now, get and open require a DataGranule array, if a user already has a list of files from a NASA DAAC, get and open should work the same way.

@andypbarrett
Copy link
Collaborator

Do you have ideas on how to do this? I have just this use case right now.

@betolink
Copy link
Member Author

betolink commented Jan 11, 2022

I think we just need to add handlers for get and open to check if we're dealing with a DataGranule list or simple links, but I was waiting to have the fsspec feature first. All the open and get operations should be done with fsspec and that requires a heavy refactoring of the EDL authentication. I hope to work on this next week.

@betolink
Copy link
Member Author

betolink commented Mar 22, 2022

There is a WIP, we can download granules by passing only a list of files to the GET method but they links have to be HTTPS. We still need to implement the logic to infer the data center and credentials to use if the links are in S3.

if we have a list that looks like this we can now use the Store:

granules = ['https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N57W103_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N59W109_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N58W104_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N55W109_dem.tif',
     'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/ASTGTM.003/ASTGTMV003_N54W103_dem.tif']

store = Store(auth)
files = store.get(granules, local_path="./data/")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants