Skip to content

Commit

Permalink
Cap hts_getline() return value at INT_MAX
Browse files Browse the repository at this point in the history
On success, hts_getline() returns the length of the string read.
Its return type is int, so when plain int is 32 bits, trouble ensues
for very long lines exceeding 2GiB: the return value wraps to negative
and is misinterpreted as error. Rather than changing the return type
to e.g. ssize_t, clamp the return value for very long lines.

In test/sam.c's test cases, check the return value is indeed the
expected length.
  • Loading branch information
jmarshall committed Jun 10, 2022
1 parent fee3bbb commit 1109c8b
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 3 deletions.
2 changes: 1 addition & 1 deletion hts.c
Original file line number Diff line number Diff line change
Expand Up @@ -1899,7 +1899,7 @@ int hts_getline(htsFile *fp, int delimiter, kstring_t *str)
case no_compression:
str->l = 0;
ret = kgetline2(str, (kgets_func2 *) hgetln, fp->fp.hfile);
if (ret >= 0) ret = str->l;
if (ret >= 0) ret = (str->l <= INT_MAX)? (int) str->l : INT_MAX;
else if (herrno(fp->fp.hfile)) ret = -2, errno = herrno(fp->fp.hfile);
else ret = -1;
break;
Expand Down
2 changes: 1 addition & 1 deletion htslib/hts.h
Original file line number Diff line number Diff line change
Expand Up @@ -676,7 +676,7 @@ int hts_set_opt(htsFile *fp, enum hts_fmt_option opt, ...);
@param fp The file handle
@param delimiter Unused, but must be '\n' (or KS_SEP_LINE)
@param str The line (not including the terminator) is written here
@return Length of the string read;
@return Length of the string read (capped at INT_MAX);
-1 on end-of-file; <= -2 on error
*/
HTSLIB_EXPORT
Expand Down
6 changes: 5 additions & 1 deletion test/sam.c
Original file line number Diff line number Diff line change
Expand Up @@ -1525,7 +1525,11 @@ static void test_text_file(const char *filename, int nexp)
if (in) {
kstring_t str = KS_INITIALIZE;
int ret, n = 0;
while ((ret = hts_getline(in, '\n', &str)) >= 0) n++;
while ((ret = hts_getline(in, '\n', &str)) >= 0) {
size_t len = strlen(str.s);
n++;
if (ret != len) fail("hts_getline read length %d (expected %zu)", ret, len);
}
if (ret != -1) fail("hts_getline got an error from %s", filename);
if (n != nexp) fail("hts_getline read %d lines from %s (expected %d)", n, filename, nexp);

Expand Down

0 comments on commit 1109c8b

Please sign in to comment.