-
Notifications
You must be signed in to change notification settings - Fork 678
Open
Labels
community help wantedWe would love the community's help completing this issueWe would love the community's help completing this issueenhancementNew feature or requestNew feature or request
Description
Currently, we dont allow users to decide on their own the number of works, pin_to_memory, etc. We should expose it in the configs.
Recipe example: https://github.com/pytorch/torchtune/blob/7bfb3336446f0d874ab5d4595249839b735b7076/recipes/full_finetune_single_device.py#L578
then in the config we have something like:
dataloader:
batch_size: ...
num_workers: 0 # set to -1 to use all. This may not work in some setups.
pin_memory: False # Set to true to xyz
#TODO: check if it makes sense to expose other arguments
The script in this PR should help to bulk update the configs: #1954
Metadata
Metadata
Assignees
Labels
community help wantedWe would love the community's help completing this issueWe would love the community's help completing this issueenhancementNew feature or requestNew feature or request