Skip to content

Commit d0cf6b5

Browse files
add back part
Signed-off-by: Robert Nishihara <rkn@anyscale.com>
1 parent af8eec8 commit d0cf6b5

File tree

1 file changed

+11
-0
lines changed

1 file changed

+11
-0
lines changed

doc/source/data/loading-data.rst

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -652,6 +652,17 @@ Ray Data interoperates with distributed data processing frameworks like `Daft <h
652652
{'col1': 1, 'col2': '1'}
653653
{'col1': 2, 'col2': '2'}
654654

655+
.. _loading_huggingface_datasets:
656+
657+
Loading Hugging Face datasets
658+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
659+
660+
To read datasets from the Hugging Face Hub, use :func:`~ray.data.read_parquet` (or other
661+
read functions) with the ``HfFileSystem`` filesystem. This approach provides better
662+
performance and scalability than loading datasets into memory first.
663+
664+
First, install the required dependencies
665+
655666
.. _loading_datasets_from_ml_libraries:
656667

657668
Loading data from ML libraries

0 commit comments

Comments
 (0)