Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading S3File using RasterDataset #1165

Open
alexandralorenzo opened this issue Mar 9, 2023 · 3 comments · May be fixed by #1399
Open

Reading S3File using RasterDataset #1165

alexandralorenzo opened this issue Mar 9, 2023 · 3 comments · May be fixed by #1399
Labels
datasets Geospatial or benchmark datasets

Comments

@alexandralorenzo
Copy link

Summary

Hello,
First of all thanks for your work.

Could it be possible to add an option from reading files from S3 using RasterDataset?

class BDOrtho(RasterDataset):
    filename_glob = "*.jp2"
    is_image = True
    separate_files = True

TypeError: expected str, bytes or os.PathLike object, not S3File

Kind regards,
Alexandra

Rationale

I need to create my own datasets reader

Implementation

No response

Alternatives

No response

Additional information

No response

@adamjstewart
Copy link
Collaborator

I would love to add support for this. At the moment, RasterDataset is designed only for files on the local filesystem. If I'm understanding it correctly, TorchData would make it easier to integrate different optional data sources. We've talked about porting our datasets to TorchData but haven't gotten a chance to work on this yet. TorchGeo is entirely a volunteer-driven open-source project, so if you have cycles and want to work on this, I would be happy to review PRs or give suggestions. Although I don't have access to AWS, so my debugging and testing abilities may be limited.

@adamjstewart adamjstewart added the datasets Geospatial or benchmark datasets label Mar 9, 2023
@adamjstewart
Copy link
Collaborator

You might be interested in #1399

@adamjstewart adamjstewart linked a pull request Jul 12, 2023 that will close this issue
@adamjstewart
Copy link
Collaborator

Note that we technically support this in 0.5.0, although the user has to pass in a list of files themselves.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasets Geospatial or benchmark datasets
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants