File tree Expand file tree Collapse file tree 1 file changed +26
-0
lines changed
Expand file tree Collapse file tree 1 file changed +26
-0
lines changed Original file line number Diff line number Diff line change @@ -1000,6 +1000,32 @@ separately from Zarr.
10001000
10011001.. _tutorial_copy :
10021002
1003+ Accessing Zip Files on S3
1004+ ~~~~~~~~~~~~~~~~~~~~~~~~~
1005+
1006+ The built-in `ZipStore ` will only work with paths on the local file-system, however
1007+ it is also possible to access ``.zarr.zip `` data on the cloud. Here is an example of
1008+ accessing a ``.zarr.zip `` file on s3:
1009+
1010+ >>> s3_path = " s3://path/to/my.zarr.zip"
1011+ >>>
1012+ >>> s3 = s3fs.S3FileSystem()
1013+ >>> f = s3.open(s3_path)
1014+ >>> fs = ZipFileSystem(f, mode = " r" )
1015+ >>> store = FSMap(" " , fs, check = False )
1016+ >>>
1017+ >>> # cache is optional, but may be a good idea depending on the situation
1018+ >>> cache = zarr.storage.LRUStoreCache(store, max_size = 2 ** 28 )
1019+ >>> z = zarr.group(store = cache)
1020+
1021+ This store can also be generated with ``fsspec ``'s handler chaining, like so:
1022+
1023+ >>> store = zarr.storage.FSStore(url = f " zip:: { s3_path} " , mode = " r" )
1024+
1025+ Note that this is intended for a read-only data source. However, this can be
1026+ especially useful if you have a very large ``.zarr.zip `` file on s3 and only need
1027+ to access a small portion of it.
1028+
10031029Consolidating metadata
10041030~~~~~~~~~~~~~~~~~~~~~~
10051031
You can’t perform that action at this time.
0 commit comments