Skip to content

Commit

Permalink
[Doc] Add data ingestion clarification for AIR converting existing py…
Browse files Browse the repository at this point in the history
…torch code example (ray-project#32058)

The example under Ray AI Runtime/Example section directly used native PyTorch datasets for data loading. It's good to clarify that the current approach is for simplicity, the more recommended approach is to use the Ray dataset.

Signed-off-by: Yunxuan Xiao <yunxuanx@Yunxuans-MBP.local.meter>
Signed-off-by: Yunxuan Xiao <xiaoyunxuan1998@gmail.com>
Co-authored-by: Yunxuan Xiao <yunxuanx@Yunxuans-MBP.local.meter>
Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
Co-authored-by: Yunxuan Xiao <yunxuanx@Yunxuans-MacBook-Pro.local>
Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>
  • Loading branch information
4 people authored and edoakes committed Mar 22, 2023
1 parent 1521d1f commit 4a82c3b
Showing 1 changed file with 7 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,14 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "db36ae56",
"metadata": {},
"source": [
"Then we download the data:"
"Then we download the data: \n",
"\n",
"This tutorial assumes that your existing code is using the `torch.utils.data.Dataset` native to PyTorch. It continues to use `torch.utils.data.Dataset` to allow you to make as few code changes as possible. **This tutorial also runs with Ray Data, which gives you the benefits of efficient parallel preprocessing.** See an example of using Ray Data for the CIFAR-10 dataset {doc}`here </ray-air/examples/torch_image_example>`."
]
},
{
Expand Down Expand Up @@ -1300,7 +1303,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.8.10 ('venv': venv)",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
Expand All @@ -1314,11 +1317,11 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
"version": "3.9.6 (default, Oct 18 2022, 12:41:40) \n[Clang 14.0.0 (clang-1400.0.29.202)]"
},
"vscode": {
"interpreter": {
"hash": "3c0d54d489a08ae47a06eae2fd00ff032d6cddb527c382959b7b2575f6a8167f"
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
}
}
},
Expand Down

0 comments on commit 4a82c3b

Please sign in to comment.