Skip to content

Commit

Permalink
Update on "Fixed on_disk_cache issues"
Browse files Browse the repository at this point in the history
Fixed issues with cache locks and cache files overwrites. Required to be compatible with pytorch/data#810




[ghstack-poisoned]
  • Loading branch information
VitalyFedyunin committed Oct 13, 2022
1 parent 4e945e4 commit 07af834
Showing 1 changed file with 0 additions and 2 deletions.
2 changes: 0 additions & 2 deletions torchtext/datasets/cnndm.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,7 @@ def _get_split_list(source: str, split: str):


def _load_stories(root: str, source: str, split: str):

split_list = set(_get_split_list(source, split))

story_dp = IterableWrapper([URL[source]])
cache_compressed_dp = story_dp.on_disk_cache(
filepath_fn=partial(_filepath_fn, root, source),
Expand Down

0 comments on commit 07af834

Please sign in to comment.