You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The LoadFileMixin is used by rasterio, csv, and xarray nodes for both local and remote files. As written, the it always loads the whole file. The Mixin should be updated to only load the whole file in certain cases, and otherwise open the dataset directly from the source path.
Describe the solution you'd like
Distinguish between local and remote sources. Only read the file into memory if necessary for caching.
Additional Notes
The Rasterio node should be modified to not use the mixin. This will make the read_from_source unnecessary.
The text was updated successfully, but these errors were encountered:
Description
The
LoadFileMixin
is used by rasterio, csv, and xarray nodes for both local and remote files. As written, the it always loads the whole file. The Mixin should be updated to only load the whole file in certain cases, and otherwise open the dataset directly from the source path.Describe the solution you'd like
Distinguish between local and remote sources. Only read the file into memory if necessary for caching.
Additional Notes
The Rasterio node should be modified to not use the mixin. This will make the
read_from_source
unnecessary.The text was updated successfully, but these errors were encountered: