Fix memory leaks and aborts in H5O EFL decode #2656
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The function that decodes external data files object header messages would call assert() when parsing malformed files, causing applications to crash when linked against the debug library. This change converts these assert() calls to HDF5 error checks, so the messages are sanity checked in both release and debug mode and debug mode no longer crashes applications. Also cleaned up some error handling usage and debug checks. Related to GitHub HDFGroup#2605
Fixes GitHub HDFGroup#2605
derobins
requested review from
lrknox,
byrnHDF,
fortnern,
jhendersonHDF,
qkoziol,
vchoi-hdfgroup,
bmribler,
glennsong09,
mattjala and
brtnfld
as code owners
derobins
commented
Apr 2, 2023
| @@ -111,64 +103,69 @@ H5O__efl_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UNUSED | |||
|
|
|||
| /* Number of slots */ | |||
| UINT16DECODE(p, mesg->nalloc); | |||
| HDassert(mesg->nalloc > 0); | |||
| if (mesg->nalloc <= 0) | |||
| HGOTO_ERROR(H5E_OHDR, H5E_CANTLOAD, NULL, "bad number of allocated slots when parsing efl msg") | |||
There was a problem hiding this comment.
Parsing a malformed file should never result in an assert() call.
There was a problem hiding this comment.
Agree
However, it's an unsigned value, so can't be <0
There was a problem hiding this comment.
I could change that to be != 0, but then that would be broken if anyone changes the type to be signed so they can have a "bad" value.
derobins
commented
Apr 2, 2023
| for (size_t u = 0; u < mesg->nused; u++) | ||
| H5MM_xfree(mesg->slot[u].name); | ||
| H5MM_xfree(mesg->slot); | ||
| } |
There was a problem hiding this comment.
mesg->slot and the names could be allocated but were not cleaned on errors.
qkoziol
approved these changes
Apr 2, 2023
qkoziol
requested changes
Apr 2, 2023
I went ahead and added parens, but I honestly don't think that remembering math > logic is too much to remember. If I get some free time, I'll add a function or macro to hide all the awkward -1 stuff. |
derobins
added
Merge - To 1.12
Priority - 1. High 🔼
mattjala
approved these changes
Apr 3, 2023
| void *ret_value = NULL; /* Return value */ | ||
| H5O_efl_t *mesg = NULL; | ||
| int version; | ||
| const uint8_t *p_end = p + p_size - 1; /* pointer to last byte in p */ |
There was a problem hiding this comment.
Suggest possibly moving this assignment past the assert for p since addition on a NULL pointer is undefined. That said, we don't have a normal check for p != NULL so it won't be caught in release builds as is anyway if this ever happens. Hopefully the assert would catch it during development though.
jhendersonHDF
approved these changes
Apr 3, 2023
derobins
added a commit
to derobins/hdf5
that referenced
this pull request
Apr 12, 2023
* Convert asserts to error handling in efl decode The function that decodes external data files object header messages would call assert() when parsing malformed files, causing applications to crash when linked against the debug library. This change converts these assert() calls to HDF5 error checks, so the messages are sanity checked in both release and debug mode and debug mode no longer crashes applications. Also cleaned up some error handling usage and debug checks. * Free memory on H5O efl decode errors * Add buffer size checks to efl msg decode * Add parentheses to math expressions Fixes GitHub HDFGroup#2605
derobins
added a commit
to derobins/hdf5
that referenced
this pull request
Apr 12, 2023
* Convert asserts to error handling in efl decode The function that decodes external data files object header messages would call assert() when parsing malformed files, causing applications to crash when linked against the debug library. This change converts these assert() calls to HDF5 error checks, so the messages are sanity checked in both release and debug mode and debug mode no longer crashes applications. Also cleaned up some error handling usage and debug checks. * Free memory on H5O efl decode errors * Add buffer size checks to efl msg decode * Add parentheses to math expressions Fixes GitHub HDFGroup#2605
derobins
added a commit
that referenced
this pull request
Apr 13, 2023
* Convert asserts to error handling in efl decode The function that decodes external data files object header messages would call assert() when parsing malformed files, causing applications to crash when linked against the debug library. This change converts these assert() calls to HDF5 error checks, so the messages are sanity checked in both release and debug mode and debug mode no longer crashes applications. Also cleaned up some error handling usage and debug checks. * Free memory on H5O efl decode errors * Add buffer size checks to efl msg decode * Add parentheses to math expressions Fixes GitHub #2605
derobins
added a commit
that referenced
this pull request
Apr 13, 2023
* Convert asserts to error handling in efl decode The function that decodes external data files object header messages would call assert() when parsing malformed files, causing applications to crash when linked against the debug library. This change converts these assert() calls to HDF5 error checks, so the messages are sanity checked in both release and debug mode and debug mode no longer crashes applications. Also cleaned up some error handling usage and debug checks. * Free memory on H5O efl decode errors * Add buffer size checks to efl msg decode * Add parentheses to math expressions Fixes GitHub #2605
brtnfld
pushed a commit
to brtnfld/hdf5
that referenced
this pull request
May 17, 2023
* Convert asserts to error handling in efl decode The function that decodes external data files object header messages would call assert() when parsing malformed files, causing applications to crash when linked against the debug library. This change converts these assert() calls to HDF5 error checks, so the messages are sanity checked in both release and debug mode and debug mode no longer crashes applications. Also cleaned up some error handling usage and debug checks. * Free memory on H5O efl decode errors * Add buffer size checks to efl msg decode * Add parentheses to math expressions Fixes GitHub HDFGroup#2605
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
H5O__efl_decode() would call assert() and not properly clean up memory when parsing malformed external data files messages. This change corrects that behavior and fixes #2605.