Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Commit

Permalink
feat: add exception handling to _unzip method in build_data.py to con… (
Browse files Browse the repository at this point in the history
#4572)

* feat: add exception handling to _unzip method in build_data.py to continue execution even when some files fail to extract

* fix lint error
  • Loading branch information
chiehminwei authored and Pearl Li committed Jun 9, 2022
1 parent c837722 commit 484c6db
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions parlai/core/build_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -380,8 +380,13 @@ def _unzip(path, fname, delete=True):
PathManager.mkdirs(outpath)
continue
logging.debug(f"Extracting to {outpath}")
with zf.open(member, 'r') as inf, PathManager.open(outpath, 'wb') as outf:
shutil.copyfileobj(inf, outf)
try:
with zf.open(member, 'r') as inf, PathManager.open(
outpath, 'wb'
) as outf:
shutil.copyfileobj(inf, outf)
except FileNotFoundError:
logging.error(f"Failed to open ${member} and extract to ${outpath}")
if delete:
try:
PathManager.rm(fullpath)
Expand Down

0 comments on commit 484c6db

Please sign in to comment.