Skip to content

Commit 3805758

Browse files
authored
Update tutorial.rst to include section about accessing Zip Files on S3
Per discussion here, add information about about accessing zip files on s3: #1613
1 parent b5f79dd commit 3805758

File tree

1 file changed

+26
-0
lines changed

1 file changed

+26
-0
lines changed

docs/tutorial.rst

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1000,6 +1000,32 @@ separately from Zarr.
10001000

10011001
.. _tutorial_copy:
10021002

1003+
Accessing Zip Files on S3
1004+
~~~~~~~~~~~~~~~~~~~~~~~~~
1005+
1006+
The built-in `ZipStore` will only work with paths on the local file-system, however
1007+
it is also possible to access ``.zarr.zip`` data on the cloud. Here is an example of
1008+
accessing a ``.zarr.zip`` file on s3:
1009+
1010+
>>> s3_path = "s3://path/to/my.zarr.zip"
1011+
>>>
1012+
>>> s3 = s3fs.S3FileSystem()
1013+
>>> f = s3.open(s3_path)
1014+
>>> fs = ZipFileSystem(f, mode="r")
1015+
>>> store = FSMap("", fs, check=False)
1016+
>>>
1017+
>>> # cache is optional, but may be a good idea depending on the situation
1018+
>>> cache = zarr.storage.LRUStoreCache(store, max_size=2**28)
1019+
>>> z = zarr.group(store=cache)
1020+
1021+
This store can also be generated with ``fsspec``'s handler chaining, like so:
1022+
1023+
>>> store = zarr.storage.FSStore(url=f"zip::{s3_path}", mode="r")
1024+
1025+
Note that this is intended for a read-only data source. However, this can be
1026+
especially useful if you have a very large ``.zarr.zip`` file on s3 and only need
1027+
to access a small portion of it.
1028+
10031029
Consolidating metadata
10041030
~~~~~~~~~~~~~~~~~~~~~~
10051031

0 commit comments

Comments
 (0)