Skip to content

Commit f96d66b

Browse files
Adrien Di Mascioadimasci
Adrien Di Mascio
authored andcommitted
BUG: close hdf store on any error
In `read_hdf`, if the `store.select()` call throws either a `ValueError`, a `TypeError` or a `KeyError` then the store is closed. However, if any other exception is thrown (e.g. an `AssertionError` if there are gaps in blocks ref_loc) , the store is not closed and some client code could end up trying to reopen the store and hit an error like: `the file XXX is already opened. Please close it before reopening in write mode`. Furthermore, the exception is re-raised in all cases. This commit just closes store in all cases. Closes #28430
1 parent a880e42 commit f96d66b

File tree

3 files changed

+21
-1
lines changed

3 files changed

+21
-1
lines changed

doc/source/whatsnew/v1.0.0.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,7 @@ I/O
191191
- Bug in :meth:`DataFrame.to_json` where using a Tuple as a column or index value and using ``orient="columns"`` or ``orient="index"`` would produce invalid JSON (:issue:`20500`)
192192
- Improve infinity parsing. :meth:`read_csv` now interprets ``Infinity``, ``+Infinity``, ``-Infinity`` as floating point values (:issue:`10065`)
193193
- Bug in :meth:`DataFrame.to_csv` where values were truncated when the length of ``na_rep`` was shorter than the text input data. (:issue:`25099`)
194+
- :meth:`read_hdf` now closes the store on any error thrown (:issue:`28430`)
194195

195196
Plotting
196197
^^^^^^^^

pandas/io/pytables.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -404,7 +404,7 @@ def read_hdf(path_or_buf, key=None, mode="r", **kwargs):
404404
)
405405
key = candidate_only_group._v_pathname
406406
return store.select(key, auto_close=auto_close, **kwargs)
407-
except (ValueError, TypeError, KeyError):
407+
finally:
408408
# if there is an error, close the store
409409
try:
410410
store.close()

pandas/tests/io/pytables/test_pytables.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -562,6 +562,25 @@ def check_default_mode():
562562
check("w")
563563
check_default_mode()
564564

565+
def test_store_closed_on_error(self, mocker):
566+
"""check hdf store is closed whichever error occurs"""
567+
mocker.patch("pandas.io.pytables.HDFStore.select").side_effect = AssertionError(
568+
"Gaps in blk ref_locs"
569+
)
570+
df = tm.makeDataFrame()
571+
with ensure_clean_path(self.path) as path:
572+
df.to_hdf(path, "df")
573+
try:
574+
pd.read_hdf(path, "df")
575+
except AssertionError: # this is expected, because of our mock
576+
pass
577+
try:
578+
HDFStore(path, mode="w")
579+
except ValueError as exc:
580+
pytest.fail(
581+
"store not closed properly, got error {exc}".format(exc=exc)
582+
)
583+
565584
def test_reopen_handle(self):
566585

567586
with ensure_clean_path(self.path) as path:

0 commit comments

Comments
 (0)