We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent af8eec8 commit d0cf6b5Copy full SHA for d0cf6b5
doc/source/data/loading-data.rst
@@ -652,6 +652,17 @@ Ray Data interoperates with distributed data processing frameworks like `Daft <h
652
{'col1': 1, 'col2': '1'}
653
{'col1': 2, 'col2': '2'}
654
655
+.. _loading_huggingface_datasets:
656
+
657
+Loading Hugging Face datasets
658
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
659
660
+To read datasets from the Hugging Face Hub, use :func:`~ray.data.read_parquet` (or other
661
+read functions) with the ``HfFileSystem`` filesystem. This approach provides better
662
+performance and scalability than loading datasets into memory first.
663
664
+First, install the required dependencies
665
666
.. _loading_datasets_from_ml_libraries:
667
668
Loading data from ML libraries
0 commit comments